Search Results: "ron"

28 June 2011

Matthew Palmer: SSL Session Caching (in nginx)

Given that I was responsible for putting together the system architecture for Github when they moved away from their previous services provider to Anchor Systems, I keep an eye on what's going on there even though I've gotten out of professional IT. Recently(ish), Ryan Tomayko blogged about some experiments in using Amazon CloudFront as a CDN for static assets. While discussing as-yet unsolved performance issues, he quoted a tweet from Shopify's CEO (who I would have thought would have known better):
Protip to make SSL fast: terminate in your load balancer, that makes SSL session cookies actually work.
No, no, NO -- a million times NOOOOOO! Load balancers are, by their very nature, performance chokepoints. You can't horizontally scale your load balancer, because if you do (by, say, putting another load balancer in front of it -- don't laugh, I've seen it done) you've just re-introduced all the same problems that putting SSL on the load balancer was supposed to accomplish (that is, getting SSL session caching to work). The only thing you accomplish by putting your SSL sessions onto the load balancer is bring the moment your load balancer melts down through excessive CPU usage that much closer. I dunno, maybe Shopify doesn't actually do that much traffic or something, but I certainly wouldn't be signing up for that headache if it were my infrastructure. (Sidenote: yes, I know that you can get "SSL hardware accelerators", that put the basic crypto operations into dedicated silicon, but they're rarely cheap, and you can't horizontally scale those, either. They're effectively a specialised and outrageously expensive form of "scaling by bigger hardware" -- which is a valid method of scaling, but one with fairly terminal limitations that certainly wouldn't make it practical for Github). The problem you're trying to solve by centralising SSL operations is that SSL includes a mechanism whereby a client making a new TCP connection can say "here's the crypto details from last time I connected to you" and the server can just pull out all of the per-connection data that would otherwise take a long time (several TCP round trips and a smallish chunk of CPU time) to setup. This can make SSL connections a lot snappier (in Ryan's opinion, "We spend roughly as much time handshaking as we do generating the average dynamic request."), so they're a definite win if you can do it (and yes, you can do it). Webservers don't typically do this caching by default, but setting up a simple SSL session cache for a single nginx or Apache webserver is pretty trivial. Testing it is simple, too: just install gnutls-bin and run: 
	gnutls-cli -V -r HOSTNAME  grep 'Session ID'
If all three Session IDs are the same, then you've got SSL session caching running. (If you're wondering why I'm using gnutls-cli rather than openssl s_client, it's because openssl is living in the pre-IPv6 dark ages, and I've got IPv6-only services running I wanted to test). So that's great for your little single-server hosting setup, but per-machine caches aren't much use when you're running behind a load balancer, Github style, because each SSL-enabled TCP connection can very well end up on a different server, and a per-machine cache won't help you very much. (If you think the answer to this problem is "session affinity", you're setting yourself up for a whole other world of pain and suffering.) Here's the kicker, though -- the SSL session is just a small, opaque chunk of bytes (OpenSSL appears to store it internally as an ASN.1 string, but that's an irrelevant implementation detail). An SSL session cache is just a mapping of the SSL session ID (a string) to the SSL session data (another string). We know how to do "shared key-value caches" -- we have the technology. There won't be too many large-scale sites out there that aren't using memcached (or something practically equivalent). So, rather than stuff around trying to run a load balancer with enough grunt to handle all your SSL communication for all time, you can continue to horizontally scale the whole site with as many backend webservers as you need. All that's needed is to teach your SSL-using server to talk to a shared key-value cache. Far be it for me to claim any sort of originality in this idea, though -- patches for Apache's mod_ssl to do this have been floating around for a while, and according to the release notes, the 2.4 series will have it all built-in. However, Github uses nginx, to great effect, and nobody's done the work to put this feature into nginx (that I could find, anyway). Until now. I present, for the edification of all, a series of patches to nginx 0.8.x to provide memcached-enabled SSL session caching. Porting them to other versions of nginx hopefully won't be too arduous, and if you ask nicely I might give it a shot. If you don't use memcached (gasp!), my work should at least be a template to allow you to hook into whatever system you prefer. Sharding the memcached calls wouldn't be hard either, but I'm not a huge fan of that when other options are available. I had originally hoped to be able to say "it works at Github!" as a testimonial, but for some reason it hasn't been deployed there yet (go forth and pester!), so instead I'm just releasing it out there for people to try out on a "well, I don't know that it doesn't work" basis. If it breaks anything, it's not my fault; feel free to supply patches if you want to fix anything I've missed.

Sam Hartman: Dream Plug

As part of trying to help with Freedom Box, I gained access to a Dream Plug The Dream Plug is one of the leading possible platforms for Freedom Box. Computationally it s sexy for a low-power device. There s a 1200 mhz arm with 512m of RAM. As a platform to enable people to create novel applications, it s great. It has multiple USB ports, two ethernets, a built-in 802.11B/G access point, bluetooth, audio, ESATA, Micro SD and full-sized SD. So, whether you application needs storage, networking, audio, or some interesting side device, you re covered. With the optional JTAG adapter it s even fairly friendly to developers: there are options for recovering from most failures and full console access. If you re looking at a reference platform that s still nominally embedded, but that allows you to play around thinking about what your application could do, this is a great option. However, especially for Freedom Box, it s important to remember that the reference board a developer wants is not the same as a cost-reduced product on which to actually deploy something for people to buy. The Dream Plug is wrong for every actual deployment I ve imagined. First, it s huge. If you happen to have free power plugs on your wall with lots of space above and below them, it might be an option. If however, you re like everyone I know and have power strips, constrained spacing, etc, then the industrial engineering will disappoint at every turn. I was hoping for something that kind of looked like an Apple Airport Express. It s significantly larger than that in every dimension. Also, the plug is oriented the wrong way for minimizing the fraction of the power strip it takes up. The power supply part of the device can be detached, although even that is way too huge for a power strip, and the cable between the power supply and the computer is fairly short. You can reconfigure the device without its plug nature: running a cord from a normal power outlet to the power supply. But then if you have to detach the power supply for heat management reasons, you get a cord from the outlet to the power supply and another cord from the supply to the device. Add a few USB or ESATA devices and the octopus of cables begins to resemble some arcane mechanism. (So far, no elder gods have appeared though.) The other issue is that you almost never want all the functionality. You pay for the bluetooth, audio and ESATA in terms of cost, heat and space regardless of whether you use them. I don t have a lot of applications that really take advantage of the full array of hardware. The firmware update mechanism is decidedly not targeted at end-users. The version I obtained had a hacked copy of Debian lenny. No mechanism was provided for replacing the image in a safe manner that did not potentially require the optional highly-non-end-user-compatible JTAG board if something got interrupted. You could either unscrew the device and get to the micro SD card containing the image, or run software to replace the image from within Debian lenny. It s possible to configure the boot loader to run an update image off USB or SD until that succeeds, but doing that is also a non-end-user operation. In conclusion, the name is perfect. This is exactly what hackers need to dream about the power of small computers everywhere. However we must not forget that there s a step required to turn dreams into reality. Just as with any fully proprietary product, Freedom Box will require cost reduction steps, semi-custom boards and actual OEMs to truly be usable. The claim in the previous sentence that the Freedom Box may have proprietary elements is disquieting to some. I think we can put together a software stack that is free with the possible exception of some device firmware. However, my suspicion is that anyone who turns that into a fully realized end-user product will add proprietary elements. I suspect some of the results of the cost reduction process such as resulting semi-custom boards will be proprietary. In many cases though, I suspect some proprietary software elements will be introduced.

27 June 2011

NeuroDebian: Neuroscience runs on GNU/Linux

Neuroscience runs on GNU/Linux While everybody else is waiting for Linux to become the standard on the desktop, in the field of neuroscience research we have made this step already so silently that we haven t even realized it ourselves. In the NeuroDebian project we were trying to figure out what software would need to be integrated into Debian to have a maximum impact on enhancing its utility as a research platform. So we ran a survey, asking people to tell us what software they use in their research and what platform they run it on. We also asked people to describe their personal preferences regarding a computing environment and how that might differ from the computing platforms they are institutionally provided with. The main result of this survey was very simple and pretty surprising (even to us): Despite common believe, GNU/Linux is the current standard computing platform in neuroscience and that is both on the big compute clusters and on the laptop and desktop. It turns out that researchers on GNU/Linux are not only the majority, they are also much happier with what they get, and spend less time on non-research maintenance tasks than the guys who are still stuck with Windows. It was also nice to see that Debian-based systems are preferred by neuroscientists among GNU/Linux distributions for their personal computing environments and that even proportions of Red Hat and Debian-based systems together represent the vast majority of GNU/Linux deployments in institutional computing infrastructure. Well done Debian! The results have just been accepted for publication in the journal Frontiers in Neuroinformatics. A link to the manuscript, the original survey form, collected data, as well as supplementary analysis results are available at: http://neuro.debian.net/survey/2011/

Matthew Palmer: SSL Session Caching (in nginx)

Given that I was responsible for putting together the system architecture for Github when they moved away from their previous services provider to Anchor Systems, I keep an eye on what's going on there even though I've gotten out of professional IT. Recently(ish), Ryan Tomayko blogged about some experiments in using Amazon CloudFront as a CDN for static assets. While discussing as-yet unsolved performance issues, he quoted a tweet from Shopify's CEO (who I would have thought would have known better):
Protip to make SSL fast: terminate in your load balancer, that makes SSL session cookies actually work.
No, no, NO -- a million times NOOOOOO! Load balancers are, by their very nature, performance chokepoints. You can't horizontally scale your load balancer, because if you do (by, say, putting another load balancer in front of it -- don't laugh, I've seen it done) you've just re-introduced all the same problems that putting SSL on the load balancer was supposed to accomplish (that is, getting SSL session caching to work). The only thing you accomplish by putting your SSL sessions onto the load balancer is bring the moment your load balancer melts down through excessive CPU usage that much closer. I dunno, maybe Shopify doesn't actually do that much traffic or something, but I certainly wouldn't be signing up for that headache if it were my infrastructure. (Sidenote: yes, I know that you can get "SSL hardware accelerators", that put the basic crypto operations into dedicated silicon, but they're rarely cheap, and you can't horizontally scale those, either. They're effectively a specialised and outrageously expensive form of "scaling by bigger hardware" -- which is a valid method of scaling, but one with fairly terminal limitations that certainly wouldn't make it practical for Github). The problem you're trying to solve by centralising SSL operations is that SSL includes a mechanism whereby a client making a new TCP connection can say "here's the crypto details from last time I connected to you" and the server can just pull out all of the per-connection data that would otherwise take a long time (several TCP round trips and a smallish chunk of CPU time) to setup. This can make SSL connections a lot snappier (in Ryan's opinion, "We spend roughly as much time handshaking as we do generating the average dynamic request."), so they're a definite win if you can do it (and yes, you can do it). Webservers don't typically do this caching by default, but setting up a simple SSL session cache for a single nginx or Apache webserver is pretty trivial. Testing it is simple, too: just install gnutls-bin and run: 
	gnutls-cli -V -r HOSTNAME  grep 'Session ID'
If all three Session IDs are the same, then you've got SSL session caching running. (If you're wondering why I'm using gnutls-cli rather than openssl s_client, it's because openssl is living in the pre-IPv6 dark ages, and I've got IPv6-only services running I wanted to test). So that's great for your little single-server hosting setup, but per-machine caches aren't much use when you're running behind a load balancer, Github style, because each SSL-enabled TCP connection can very well end up on a different server, and a per-machine cache won't help you very much. (If you think the answer to this problem is "session affinity", you're setting yourself up for a whole other world of pain and suffering.) Here's the kicker, though -- the SSL session is just a small, opaque chunk of bytes (OpenSSL appears to store it internally as an ASN.1 string, but that's an irrelevant implementation detail). An SSL session cache is just a mapping of the SSL session ID (a string) to the SSL session data (another string). We know how to do "shared key-value caches" -- we have the technology. There won't be too many large-scale sites out there that aren't using memcached (or something practically equivalent). So, rather than stuff around trying to run a load balancer with enough grunt to handle all your SSL communication for all time, you can continue to horizontally scale the whole site with as many backend webservers as you need. All that's needed is to teach your SSL-using server to talk to a shared key-value cache. Far be it for me to claim any sort of originality in this idea, though -- patches for Apache's mod_ssl to do this have been floating around for a while, and according to the release notes, the 2.4 series will have it all built-in. However, Github uses nginx, to great effect, and nobody's done the work to put this feature into nginx (that I could find, anyway). Until now. I present, for the edification of all, a series of patches to nginx 0.8.x to provide memcached-enabled SSL session caching. Porting them to other versions of nginx hopefully won't be too arduous, and if you ask nicely I might give it a shot. If you don't use memcached (gasp!), my work should at least be a template to allow you to hook into whatever system you prefer. Sharding the memcached calls wouldn't be hard either, but I'm not a huge fan of that when other options are available. I had originally hoped to be able to say "it works at Github!" as a testimonial, but for some reason it hasn't been deployed there yet (go forth and pester!), so instead I'm just releasing it out there for people to try out on a "well, I don't know that it doesn't work" basis. If it breaks anything, it's not my fault; feel free to supply patches if you want to fix anything I've missed.

Philipp Kern: YouTube serving its content over IPv6

In the aftermath of the World IPv6 Day YouTube seems to be serving its content over IPv6 now. Interestingly the frontpage is still served via IPv4 (if you're not in a Google IPv6 whitelisted network). But all the Flash and HTML5 video content is served through IPv6 if available, as the cache servers return proper AAAA DNS records. Apparently that's the case unless your network is blacklisted because of bad IPv6 support and even if Google has some caches at your provider's site (which is the case for Alice DSL in Germany, at least).

I think that's quite some motivation for the providers to at least fix IPv6 connectivity if available and to suppress rogue IPv6 router advertisements in their networks. I had to ensure the former today and the latter is a constant source of grief with the bulk of L2 switches and Wi-Fi access points not being IPv6 ready.

26 June 2011

Cyril Brulebois: Hack orola (1)

Disclaimer: This post only describes what I ve been hacking on those days, and why. I haven t finished yet, and that might be too long of a story for a blog/planet. I m therefore duplicating the problem and the context, and mentioning the parts already documented on the page dedicated to that hack. Challenge: How to send commands to a computer without using an input device? Context: While it s not frequent, I sometimes run into situations where no input devices would work. That can happen because of a new X server with no compatible input devices; or while running a sample Wayland compositor on a console with X still running (not exactly sure what the problem was exactly); or because running gdb on X from within X is a bad idea; etc. Simple, non-bulletproof solutions: Alternative, viable solution 1: Buying a tiny, open, programmable device, like the Arduino Nano, a few components like a selector and a push button might do the trick. Then, some bits of programming with the open source Arduino environment should make it possible to connect that device to a USB port using a mini-B cable. Some bits going through a serial console and a daemon listening on it would make it possible to trigger a predefined set of actions just by selecting a position and hitting the button. Tada! No drawback for that solution, except the need for thinking about bringing that device everywhere. :) Alternative, viable solution 2: While I m by no means a phone addict, I tend to have it with me more or less all the time, and it can be connected through a mini-B USB cable. Oh, but then, maybe I could turn it into a suitable device as described above? That s a Motorola RAZR V3, it runs Java applications, and there s a SDK available for Linux (32 bits) through the Motorola developer network. That might be enough, so I decided to jump through some hoops to see whether that s indeed feasible. Not because I like it much better than the Arduino-based solution, just because I can. My hack orola page explains how I went from the maybe it s possible idea to the yes, now I know how to build, upload, and run a custom application certitude. It still lacks the actual Java code running on the phone, as well as its Unix daemon counterpart, but for a good reason: they need be written! At the moment the following sections are available:

25 June 2011

Mark Brown: I know that place

One slightly unexpected effect of all the travel I ve been doing recently is that I get a real, visceral sense of place from some slightly surprising places when I see them on the big screen. Having an idea of how the place really is, how people behave, how the air feels, how the food tastes, how it smells, a sense of what s around the corner and beyond the camera. A sense of memory, not of imagination or of an idea of what a place should be. San Francisco is one place this happens for and is perhaps not so surprising it is a pretty distinctive place after all and the imagery was already familiar from countless movies and TV shows before I ever went there. What was a bit more surprising was a random suburban area in South Korea; Korea has had a real building boom amid all the growth it s had which has resulted in a certain look and feel to the architecture and the cities being very common though not distinctive of any once place. That s the space where a lot of the more real world Korean films are set, and it s the one I recognise. Don t think I ve ever seen a film that ventured into Itaewon ( ), mind you. It s a much stronger feeling of recognition than I get with anywhere in the UK, I think purely due to the surprise factor. Having a good picture of Edinburgh isn t a great surprise, finding how quickly I ve built one up of places on other continents much more so.

22 June 2011

Martin Zobel-Helas: How YOU can help Debian!

I recently had been ask how persons usually not involved in Debian's development process can help Debian. This is a question that pops up quite often, so I thought I should write down a bit of that.

Help to make Debian a better OS
If you are using Debian, and you want something of your OS changed, open a bug report. This varies from wishlist bugs if you want to have an enhancement of a package over normal bugs for stuff that you think is a real bug up to serious or grave bug, if you found a security bug. Send a mail to submit@bugs.debian.org or use the tool reportbug and describe the problem you have found. Find more information here. Be as verbose as possible when explaining your problem. This will make it easier for the package maintainer to help you and to understand the problem. If you are not sure which package to report the bug against, report it against unknown. The bug will be taken care of, there are guys redirecting those bug reports to the appropriate package!
You could also help to verify bug reports. There are dozens of packages around, that have hundred of open bug reports. It will help the Debian package maintainer if you can tell him a "me too", esp for complex problems, or if you found out how to reproduce a bug.

Help by spreading the word
If you are using Debian, speak about it! If you have problems with Debian, speak about it! If you like Debian, speak about it. Read the debian-user mailing list (or a localized one) and jump in if users have the same problem you had, and help them.
All sort of publicity will help Debian. If there is a small exhibition near to your living, speak there about Debian, and how you are using it. Speak also to Debian, we can help you to announce your presence at that exhibition and provide you with information material in various languages. Good contact point for that is events@debian.org, or one of the debian-events-* mailing lists on lists.debian.org. If you need help, ask for it. Also, you can help Debian manning an exhibition. If you see events in your area, offer to help and don't be shy. Other way to help is working with the publicity team and prepare press announcements, the Debian Project News (DPN), contact journalists or press media if interesting things happen in Debian.

Help Debian to organize stuff
There are many ways to help Debian organize itself. For example the annual Debian Conference DebConf is a big organisation monster, and you don't need to be developer to help with that. Sometimes it's as easy as helping in the video team taping the conference, help at the front desk with registration, sorting badges or speaking to the caterer about needed foods. We also have miniDebConf or so called Debian Bug Squashing parties from time to time. Your company could provide office rooms, you could provide crash space for developers to sleep or even by sponsoring some beverages or food. Also helping around exhibition is a good idea. If your company is willing to print some flyers or posters this can help us.

Help by translating or writing documentation
Debian's website and all of the software Debian delivers should be available in all languages around the world! Good starting point for that is http://www.debian.org/international/ and the Debian Internationalization Mailing list. Also writing or extending documentation is a job everyone can do. If you are using a piece of software heavily and miss documentation, speak to the Debian package maintainer (you can find our at http://packages.debian.org/$yourpackage) and start submitting bugs with documentation.

Help by donating
There are actually many ways to help by donating (not only by money). Surely Debian will accept money donations via one of it's official representation (ffis, SPI, debian.ch, ...). On the other hand donating can be as simple as allowing your employees to work some specified time on the week on Debian! Or you donate machine hardware (probably not your old ones that you used five years and which are not under warranty now any more, sorry...), bandwith or colocation in your datacenter. Speak to the hardware donations team if you want to know the current needs.

Conclusion
I only wrote down a very few areas where you can help Debian, and there are plenty more! Don't hesitate to jump in to help. If you don't understand stuff: ask! But be prepared that you will be pointed to URLs where the stuff you ask for is documented. Helping Debian sometimes starts with reading tons of documentation (and i am sure you will find errors in that documentation to fix!), but after a while it makes a lot of fun to work for and with Debian. Find your own area to work on within Debian, and don't think you can't help. Even graphic designers, lawyers or clerks can help Debian!

I started using Debian around 15 years ago and became Debian Developer around 6 years ago. Within the last six years I had been in various positions inside Debian (listmaster team member, volatile team member, release team member and Stable Release Manager, Debian Sysadmin Team member) and got to those just by jumping in where help needed.


Craig Small: First Look at Python

When the programming language Python came out while it seemed like an interesting idea, I didn t really see the point. I already know Perl, PHP and C (plus a few others) why learn another language? So until recently, I didn t Recent frustrations with PHP and how loose it is with things like variables made me have another look at Python and what it could do for me. As just learning a language by itself it pretty boring I decided to write some small programs in the language and used the framework Turbogears to do it. First of all, learning the basics of python from perl or PHP is dead simple. The idea of what a language looks like is not too different between these three so it was a snap to write simple stuff in python. So, what does yet-another language give me? It s not one-way though, there have been some challenges. Quite likely these are more my deficiencies than the languages. Despite the minor problems, python really does give you much much more than PHP and some more than Perl. For me and what I am intending on writing it is worth learning yet another language.
Related articles

21 June 2011

Michal Čihař: phpMyAdmin code documentation

For some time, code documentation for phpMyAdmin had no official place. The reason for this was phpdoc eating way too much memory on daily runs, so I've disabled the cron job. Today I gave a try to DocBlox and it seems to work fine, so I've re-enabled daily builds. The new (daily updated) documentation is available at same location as the original one was: http://www.phpmyadmin.net/phpdoc/ You can also check build failures in the build.log: http://www.phpmyadmin.net/phpdoc/build.log

Filed under: English Phpmyadmin 0 comments Flattr this!

MJ Ray: Re: Cyberunions Podcast: Building links with the Co-operative Movement

John Atherton sent me a link to cyberunions discussing open-source and worker co-ops. In general, it s a very good show and worth a listen by anyone from the union movement, in my opinion. It s available in Ogg as well as MP3, which I think is also a plus point. There were a couple of points where I was ranting at the radio, though The first was the choice to try Skype alternatives. Leaving aside the danger of playing the alternatives game, I can understand why they chose that (the change of corporate control to Microsoft has been in the headlines recently and the phones affect them but not their listeners or readers), but open internet phones are a bit fiddly because they try to move quite a lot of data (audio) in real-time and a lot of domestic network devices don t behave correctly with the protocols that they use to do it. I agree with their conclusion that the problem is probably networking, but I wonder why they didn t log in to their routers, look at any debugging output and try to find the setting to change? Or try switching on debugging in the software and see what s wrong, or send it to someone who can help? Instead, it seemed like Linphone and Ekiga got blamed for not doing Skype-style router-busting without asking. They don t mention whether they tried switching on the STUN option mentioned in the Linphone FAQ and I don t see a email on linphone-users asking for support but I don t read every mail on that list. Am I being unfair, expecting them to seek support? Maybe but I m not sure. If you got the software from a store, what would you do? Would you throw it away and tell people that the stupid thing doesn t work? Moreover, would you use your podcast to say it doesn t work because users need to be clever enough for that obscure tool? That s a great way to scare people off unnecessarily. Wouldn t you usually think you might be overlooking some detail and ask for support, from its maker or the place where you got it? I think you d ask. I d probably ask, even though I m available for hire to fix FOSS problems and it s not the proudest feeling when I have a script-reading call-centre worker solve my mistake. Of course, with most from-store software, you ve paid an artificially-inflated box price, but the bigger cost is your time and that s the same for downloaded free and open source software, so why give it less of a chance to work? I also think they had easier options, like switching more of their website over to free and open source software, like getting rid of the rubbish comment system which doesn t work if you don t let it run javascript on your browser. I ve had a bit of discussion with them since the show and it seems my first suggestion may no longer work (typical!), so we ll see how that goes. Back to the show: the next segment did a good job of linking FOSS to the co-operative movement. I really think they should call it Free and Open Source Software and not just open source because I feel the freedom should be important to the labour movement. Also, calling the whole thing open source seems like calling all of the political left Revolutionary Socialism : inaccurate at best. I don t often express this objection these days (I ve bigger targets to work on), but it s important when trying to inform new audiences, else they only see part of our rainbow, a less radical part. Pretty interesting to suggest Orbea bikes as Mondragon s most famous brand. I d pick Eroski or Caja Laboral. What about you? Then it moved on to a pretty good description of UK co-ops, mentioning co-ops that I m a member of (the Co-operative Group) or buy from (John Lewis and Suma). I was surprised that it seemed to stop short and didn t mention any of the co-ops that work with free software, or with any software in fact. Of course, I d love software.coop to be mentioned, but there are other interesting and very different ones like BristolWireless.net and OSAlliance.com. Globally, many of the worker tech co-ops are linked through the tech-coop mailing list. This was the second part where I was shouting at the speakers. After all, Debian is a great voluntary project making a fantastic GNU/Linux distribution, but it is not a worker co-op by a long way and I suspect some of my fellow developers would be horrified to hear that suggested. Why not mention some actual tech worker co-ops? Finally, before the listener feedback, suggestions of tools which union activists could use. There was some minor confusion about exactly which version or name was really FOSS and a surprising claim about openoffice into LibreOffice (I thought Oracle were still screwing that up with Apache s help but I may have missed some news), but it was a good list of door-opening suggestions. I hope some activists will take them up. It seems there s an earlier show about open social networks, which is a topic I ll look at after I finish my series of posts about mailing lists. So, please let me know if you have comments on this show, that show, open social networks or mailing lists which you think I might find interesting, or if there s anything above you d like me to expand upon (or correct often, correct ).

Michal Čihař: New features in phpMyAdmin

As most of the GSoC projects in phpMyAdmin go on quite well, we're already merging some of the new features to our master branch. You can try them all on our demo server and you are welcome to report any issues found :-). First big part which has been changed is rewriting of server status page done by Tyron Madlener. It now provides more information, provides real time charts and much more. This change also included change in charts produced in other places in the code, so they are now more interactive and allow easy exporting. The browse mode got improved as well, you can now reorder columns (by Aris Feryanto) and there are new options for displaying GIS data (by Madhura Jayaratne). While speaking about GIS, I have to mention another new feature - rendering of GIS data, either as polygons or on OpenStreetMap map (you can see it by clicking on "Visualize GIS data" link on browse page, it is not possible to link it directly due to XSS protection). Many smaller changes are being done by Thilanka Kaushalya to convert various parts of user interface to AJAX. And last but not least we have nice support for editing stored procedures and functions. Support for events and triggers will follow soon. All these are being done by Rouslan Placella. I hope you will like these new features and thanks to all our GSoC students who are doing great job.

Filed under: English Phpmyadmin 0 comments Flattr this!

20 June 2011

Raphaël Hertzog: apt-get, aptitude, pick the right Debian package manager for you

This is a frequently asked question: What package manager shall I use? . And my answer is the one that suits your needs . In my case, I even use different package managers depending on what I m trying to do. APT vs dpkg, which one is the package manager? In the Debian world, we re usually thinking of APT-based software when we re referring to a package manager . But in truth, the real package manager is dpkg. It s the low-level tool that takes a .deb file and extracts its content on the disk, or that takes the name of a package to remove the associated files, etc. APT is better known because it s the part of the packaging infrastructure that matters to the user. APT makes collection of software available to the user and does the dirty work of downloading all the required packages and installing them by calling dpkg in the correct order to respect the dependencies. But APT is not a simple program, it s a library and several different APT frontends have been developed on top of that library. The most widely known is apt-get since it s the oldest one, and it s provided by APT itself. Graphical APT front-ends update-manager is a simple frontend useful to install security updates and other trivial daily upgrades (if you re using testing or sid). It s the one that you get when you click in the desktop notification that tells you that updates are available. In cases, where the upgrade is too complicated for update-manager, it will suggest to run synaptic which is full featured package manager. You can browse the list on installed/available packages in numerous ways, you can mark packages for installation/upgrade/removal/purge and then run in one go all the recorded actions. software-center aims to be an easy to use application installer, it will hide most of the packaging details and will only present installed/available applications (as defined by a .desktop file). It s very user friendly and has been developed by Ubuntu. Of the graphical front-ends, I use mainly synaptic and only when I m reviewing what I have installed to trim the system down. Console-based GUI APT front-ends In this category, I ll cite only aptitude. Run without parameter, it will start a powerful console-based GUI. Much like synaptic, you can have multiple views of the installed/available packages and mark packages for installation/upgrade/removal/purge before executing everything at once. Command-line based package managers and APT front-ends This is where the well known apt-get fits, but there are several other alternatives: aptitude, cupt, wajig. Wajig and cupt are special cases as they don t use libapt: the former wraps several tools including apt-get, and the latter is a (partial) APT reimplementation (versions 1.x were in Perl, 2.x are now is C++). You re welcome to try them out and find out which one you prefer, but I have never felt the need to use something else than apt-get and aptitude. apt-get or aptitude? First I want to make it clear that you can use both and mix them without problems. It used to be annoying when apt-get did not track which packages were automatically installed while aptitude did, but now that both packages share this list, there s no reason to avoid switching back and forth. I would recommend apt-get for the big upgrades (i.e. dist-upgrade from one stable to the next) because it will always find quickly a relatively good solution while aptitude can find several convoluted solutions (or none) and it s difficult to decide which one should be used. On the opposite for regular upgrades in unstable (or testing), I would recommend aptitude safe-upgrade . It does a better job than apt-get at keeping on hold packages which are temporarily broken due to some not yet finished changes while still installing new packages when required. With aptitude it s also possible to tweak dynamically the suggested operations while apt-get doesn t allow this. And aptitude s command line is probably more consistent: with apt-get you have to switch between apt-get and apt-cache depending on the operation that you want to do, aptitude on the other hand does everything by itself. Take some time to read their respective documentation and to try them.
Click here to subscribe to my free newsletter and get my monthly analysis on what s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

11 comments Liked this article? Click here. My blog is Flattr-enabled.

Nathan Handler: Google Summer of Code Report #2

A lot of work has taken place since my last report. First, since my original mentor, Matt Zimmerman, is going to be moving, he helped arrange for Stefano Zacchiroli to fill in for him temporarily. I have had a call with Stefano, and I am looking forward to getting to work with him more over the next couple of weeks. Shortly after my first report, we realized that having the dashboard only display bugs severely limited its usefulness. In our original design, we had it pulling a list of bugs with specific usertags from the BTS. This forced all information to be added to bugs, which we viewed as a good thing since it made the information available to the Debian Maintainers without requiring them to familiarize themselves with DEX. The issue was that when looking back at the ancient-patches project or forward to the big merges project, a large portion of the work will be triaging and determining whether or not the changes are even applicable in Debian. If they aren t, there is no reason for a bug to be filed on the BTS. As a result, I spent a fair amount of time re-writing most of the dashboard. It still supports finding bugs on the BTS that have been tagged with a specific DEX usertag, but it is now task-based rather than bug-based. A task can be a bug number, a package name, a patch name, or anything. It is merely something that needs to be done for a project. Each task will have a name, an assignee, a status, a note, and a bug associated with it. There are two ways to define a list of tasks for a project. The first is to tag bugs on the BTS with a user of debian-derivatives@lists.debian.org and a tag with a format of dex-<distribution>- <project> (i.e. dex-ubuntu-ancient-patches). The script will convert the bugs to tasks and automatically set all of the fields. The second method is to specify the list of tasks in a plain text file on the server the scripts are being run on. Eventually, this list will be able to be specified via a web interface. Currently, you are only able to specify the task name in the file. This means that all of the other fields for a task created in this manner are initially blank. I am still deciding on the best way to specify additional information about each task in the file. Once I decide on a method, it should not be too difficult to modify the script to parse and store that additional information. For the ancient patches DEX project, everyone involved had to update a status file stored in a VCS on alioth. This was rather annoying, so we decided early on that we want to be prevent this from being necessary for future projects. This means that the dashboard needs a way for users to modify the information on it. For tasks that are based on bugs, this is rather easy; you simply use the BTS email interface to adjust the bug. However, for other tasks, we needed the ability to modify them on the dashboard. I have managed to get this partially implemented. Currently, all cells on the table are text input fields (with css being used to hide the boxes). This means that you can click on any cell and edit its contents. Hitting the submit button will execute a script to save these changes. This is the tricky part. The dashboard is a collection of html files generated by a python script. The script is meant to be run by cron every n minutes. When you update something via a web interface, you usually expect to see the changes instantly. This means that we can t wait for the python script to run again and re-generate the html files. As a result, I currently have it setup so that when you submit the form, it will go and modify the html files to reflect the changes. The next time cron runs the python script, the html files will be re-created, and the changes made by the user will be lost. Once I come up with a method for storing additional information about a task, it shouldn t be too hard to make changes made by a user via the web interface permanent. I also did some work on a script to generate graphs. There are two types of graphs I hope to be able to display on the dashboard. One graph will help show which individuals have done the most work on a project. I have not started working on this graph yet. The second type of graph is meant to show the number of open tasks versus time. This will allow people to quickly see the progress of a project, estimate how long the project will take to complete, and notice if things start to slow down. I have some initial code for this graph using matplotlib. However, due to the many changes that were made to the dashboard, I had to hold off a bit on this graph, as it relies on certain information provided by the dashboard. I hope to sort out some of the last few issues with this graph, such as getting the axes to display properly, and then merge it with the dashboard sometime in the next few weeks. Allison Randal approached me a while ago to see if my Summer of Code project might be in a state where it could be used to assist with a Python 2.7 DEX project that she was working on. For a while, I did not think that it would be ready. However, last week, I helped her get the related bugs usertagged appropriately, and they now show up in the DEX dashboard. While the dashboard is still not complete, it does allow her and everyone else working on the project to clearly see the tasks (bugs) that they need to focus on and the current status of each of them. I plan to try and incorporate any feedback I receive from people working on this project into my designs. All of my code is available in a git repository on alioth. Keep in mind that it should not be considered stable at this point. Further status updates will be posted on my blog, and I will try to keep the wiki page updated. I am still trying work out where we will host the actual dashboard. In the meantime, you can try out my testing version of it. During the next few weeks, I will be focusing my efforts on coming up with a method for storing additional task information. That will allow me to support permanently storing changes made by users via the web interface. I also hope to finish the first graph script and merge it with the dashboard. Finally, I hope to find a permanent home for the dashboard, as having it on my personal server is not a viable long-term solution. Finally, if anyone is interested in getting involved with DEX, feel free to join the debian-derivatives mailing list, join #debian-derivatives on OFTC, or contact me via email or on IRC (nhandler).</project></distribution>

Rudy Godoy: Compute Clusters Integration for Debian Development and Building Report 2

Hello, this is the 2nd report regarding my project. I'll offer a brief
summary of how are we doing and then the details, so you don't have to
read everything at all.
I have to start saying that over the last two weeks I made a lot of
progress on the project. The most important part: Debian ARM image
integration with Eucalyptus is now cleared. The work to be done for the
next two weeks is pretty much fully coding to achieve what we've committed:
supporting Debian ARM images for Eucalyptus.
What was done so far
--------------------
- I've have identified and mapped what needs to be modified over the
  eucalyptus source for supporting ARM. I'm puting the details on the
  next section. I think this is the most crucial part that is now
  sucessfully achieved.
- I started a Project Log[1] on the wiki, so anyone can check and
  eventyally use the information I've already gathered. The log is being
  updated constantly. You can also check what I've done and what I'm
  working on.
- We got feeback[2] from Eucalyptus developers and packaging
  team regarding developers and packaging team regarding the ultimate
  source for debian/ packaging bits. This was quite important because
  in the past I've managed to build Eucalyptus from source using the debian/
  included in the upstream tarball. However the people in charge
  told us that they were using a different one, which indeed was
  different and resulted in different output than I expected. I'm
  still working on fixing the bits to get a clean build.
- I've joined the pkg-eucalyptus team and I've been given commit rights
  to the repo[2].
1- http://wiki.debian.org/SummerOfCode2011/BuildWithEucalyptus/ProjectLog
2- http://lists.alioth.debian.org/pipermail/pkg-eucalyptus-maintainers/2011-June/000289.html
What I'll be working over the next two weeks
--------------------------------------------
- Heavily code on the different bits that I've identified and need to be
  changed. They are:
  C code:
  - node/handlers_kvm. c,h  - handles nc creation. nc's are node
  clusters for Eucalyptus, the code is in charge of node creation. It
  prepares the "machine" that is able to run a compatible image.
  - util/data. c,h  - handles instance creation. The code over there
  implements the functions node/handlers_* use, like instance creation
  and libvirt's API parameter passing. Since libvirt's parameters for
  ARM vary from x86's ones, we need to support that.
  Perl code:
  - tools/gen_kvm_libvirt_xml - wrapper tool for creating KVM/QEMU
  libvirt's XML Domain definition. This needs to be heavily
  changed/updated - redesigned, to support ARM architecture.
- Depending on the previous task progress: Modify the Debian packaging
  bits for Eucalyptus and push them to the pkg-eucalyptus's team
  repository. This is not crucial for my project's success but I'd like
  to work with Debian official builds. I've yet to commit the
  Eucalyptus team version to the debian/ dir. I expect to have fixed
  the bits to get a successful and clean build, then commit.
Other plans
-----------
- I expect a slow down on the work pace over the second week due end of
  semester exams on the week from July 3 to July 9. After that week I'm
  on vacations and will be fully dedicated to the project.
- Code changes to Eucalyptus source will be pushed upstream. I would
  not like to keep a separate repo for this. I'll talk about this with
  Steffen. Changes to Debian bits will be pushed to the appropiate repos.
Other bits
----------
When the project started I noted it was too vague for some people and
it's results/output were unclear. This concerned me. However, while the
project started and we got lot of feedback things got much clear and I'd
like to take the opportunity to explain it a bit.
Currently, developers targeting ARM devices are struggling[1] with having
a development environment. There are missing bits that sometimes are not
in place and result on hard time for them. Also, due mobile devices
growth and vendors using Linux-based[2] operating systems it's a nice
opportunity for Debian to keep it's well known fame of: supporting every
arch over the sun. How this project fits, well by offering the ability
for developers to run ARM-based Debian images over x86/x86_64 CPU's
would impact the way they could be more productive and for Debian to be
more adopted as development environment.
I can say that I'm very enthusiast of this opportunity and I'm
certain that the outcome will result in more adoption. Even the Ubuntu
people is now pushing more and more efforts for ARM official support,
due the facts I've stated.
1- http://lists.debian.org/debian-arm/2011/06/msg00020.html
2- http://events.linuxfoundation.org/events/android-builders-summit/back

19 June 2011

Leo 'costela' Antunes: Importing an Outlook PST into IMAP

Well, every once in a while we re forced to do something that isn t particularly interesting or pleasant. Last week it happened again: I had to import a few pretty big PSTs (most 2Gig, one 10Gig, with about 100.000 emails) into our dovecot IMAP. Doing this with Outlook itself was out of the question: it took way too long, even on the local network (many hours for a 1G file) and was prone to hanging and crashes, which were obviously a pain to debug and start over.
Thunderbird was unfortunately not much better, since at least in our tests it didn t import the read status of the emails (they were all marked as unread) and also wasn t particularly good at handling folders with strange names, containing . , / or some more obscure characters. We had used it before for smaller files, where manually dealing with the problems was acceptable, but this time it required something a bit more elaborate, if we were to keep our sanity. Enter libpst. It includes the handy readpst utility which dumps all emails in usable formats in a directory tree, one directory per folder. Unfortunately the Debian version is somewhat outdated and doesn t support the newer Outlook formats, so I did some packaging and even a little bit of patching. It seems Thunderbird also uses this library, which would explain why it didn t handle the Read-Status (haven t confirmed this though; just read it somewhere). The last step was this not-so-little script, which uses the dumped directories from readpst and imports them in IMAP. It would have probably been a bit more elegant to use libpst directly, but I unfortunately didn t have the time to mess around with that. I did have to mess around a lot with encodings though, ergo the unholy chaos with unicode()s and str.encode()s thrown around like rice at a wedding (I could never really wrap my head around charset problems; the subject boggles my mind to this very day). 
#!/usr/bin/env python2.7
#-*- encoding: utf-8 -*-
 
import os, sys, time
import re
 
from argparse import ArgumentParser
from getpass import getpass
 
import imaplib
from email.header import decode_header
from mailbox import mbox
 
from twisted.mail import imap4 # for their imap4-utf7 implementation
 
parser = ArgumentParser(description="""Recursively import mbox files in a directory 
                                        structure to an IMAP server.\n
                                        The expected structure is that generated by 
                                        'readpst -r'.""")
parser.add_argument('-s', dest='imap_server', default='localhost', help='IMAP server to import emails to')
parser.add_argument('-u', dest='imap_user', required=True, help='user for logging in to IMAP')
parser.add_argument('-p', dest='imap_passwd', help="will be prompted for if not provided")
parser.add_argument('-c', dest='charset', default='utf8', help='charset in which the folders are stored (for versions older than 2003)')
parser.add_argument('-f', dest='force', action='store_true', help='import mail even if we think it might be a duplicate')
parser.add_argument('-m', dest='mappings', help='a JSON file with mappings between folder names and mailbox names (no slashes or dots)')
parser.add_argument('folder', nargs='+', help="the base folders to import")
 
args = parser.parse_args()
 
if not args.imap_passwd:
    args.imap_passwd = getpass()
 
 
if args.mappings:
    import json
    folderToMailbox = json.load(open(args.mappings,'r'))
else:
    folderToMailbox =   
 
 
 
def mailboxFromPath(path):
    paths = []
    for p in path.split(os.path.sep):
        p = folderToMailbox.get(p, p)
 
        # only other invalid char besides '/', which can't be created by readpst anyway
        p = p.replace('.','')
 
        paths.append(p)
 
    return '.'.join(paths)
 
def imapFlagsFromMbox(flags):
    # libpst only sets R and O
    f = []
    if 'R' in flags or 'O' in flags:
        f.append(r'\Seen')
    if 'D' in flags:
        f.append(r'\Deleted')
    if 'A' in flags:
        f.append(r'\Answered')
    if 'F' in flags:
        f.append(r'\Flagged')
    return '('+' '.join(f)+')'
 
def utf7encode(s):
    return imap4.encoder(s)[0]
 
def headerToUnicode(s):
    h = decode_header(s)[0]
    try:
        if h[1]: # charset != None
            try:
                return unicode(*h)
            except LookupError:
                return unicode(h[0],'utf8','replace')
        else:
            return unicode(h[0], 'utf8')
    except UnicodeDecodeError:
        try:
            return unicode(h[0], 'cp1252') # the usual culprits for malformed headers
        except UnicodeDecodeError:
            pass
 
        try:
            return unicode(h[0], 'latin1') # the usual culprits for malformed headers
        except UnicodeDecodeError:
            pass
 
        return unicode(h[0], 'ascii', 'ignore') # give up...
 
def main():
    imap = imaplib.IMAP4_SSL(args.imap_server)
    imap.login(args.imap_user, args.imap_passwd)
 
    imap.select()
    for base in args.folder:
        print "importing folder "+base
        for root, dirs, files in os.walk(base):
            if 'mbox' in files:
                folder = unicode(os.path.relpath(root, base), args.charset)
                mailbox = mailboxFromPath(folder)
                print u'importing mbox in  0  to  1 '.format(folder, mailbox)
                mailbox_encoded = utf7encode(mailbox)
                m = mbox(os.path.join(root, 'mbox'), create=False)
                for msg in m:
                    if imap.select(mailbox_encoded)[0] != 'OK':
                        print "creating mailbox "+mailbox
                        r = imap.create(mailbox_encoded)
                        if r[0] != 'OK':
                            sys.stderr.write("Could not create mailbox: "+str(r))
                            continue
                        imap.subscribe(mailbox_encoded)
                        imap.select(mailbox_encoded)
 
 
                    # skip possibly duplicated msgs
                    query = 'FROM " 0 " SUBJECT " 1 "'.format(
                                utf7encode(headerToUnicode(msg['from']).replace('"','')),
                                utf7encode(headerToUnicode(msg['subject']).replace('"',r'\"'))
                            )
                    if msg.has_key('date'):
                        query += ' HEADER DATE " 0 "'.format(utf7encode(msg['date']))
                    if msg.has_key('message-id') and msg['message-id']:
                        query += ' HEADER MESSAGE-ID " 0 "'.format(utf7encode(msg['message-id']))
 
                    r = imap.search(None, '( 0 )'.format(query))
                    if r[1][0] and not args.force:
                        print "skipping "+mailbox+": '"+headerToUnicode(msg['subject'])[:20]+"' (mid: "+str(msg['message-id'])+")"
                        continue
 
                    r = imap.append(mailbox_encoded, '', imaplib.Time2Internaldate(time.time()), str(msg))
                    if r[0] != 'OK':
                        sys.stderr.write("failed to import  0  ( 1 ):  2 ".format(msg['message-id'], msg['date'], r[1]))
                        continue
                    num = re.sub(r'.*APPENDUID \d+ (\d+).*', r'\1', r[1][0])
                    r = imap.uid('STORE', str(num), "FLAGS", imapFlagsFromMbox(msg.get_flags()))
 
                    if r[0] != 'OK':
                        sys.stderr.write("failed to set flags for msg  0  in  1 ".format(num, mailbox))
 
    imap.logout()
 
if __name__ == '__main__':
    main()
Another useful automation is using the mappings file, which looks something like 
 
"Deleted items": "Trash",
"Sent items": "Sent"
to automatically import in the right destinations, in case you have a standardized structure.

18 June 2011

John Goerzen: Music Powerful and Fun

Sometimes there s nothing quite so heartwarming as hearing a 4-year-old that doesn t think anybody is listening. When Jacob is all alone in his room, sometimes he will sing. Maybe it s a song he knows, or fragments of a song he knows. Or maybe it s something he just made up on the fly. It might be about trains, cats, letters, or who knows what. Maybe it will crack him up and he ll keep singing and then laugh. Or maybe it will stop in 30 seconds and he ll go on to something else. Brief and out of tune though it may be, it is still one of the most beautiful sounds I know and I love to hear it. Jacob loves to have other people sing, too. Sometimes I will make up songs while we drive: We are driving, we are driving, we are driving down the road; we are driving down the old country road and we re making some dust, yes a lot of dust Or Jacob likes me to sing the songs in the Winnie the Pooh book he has (the original, pre-Disney version, which has the words for the songs but not a tune, so I make up a little bit different one each time). Very occasionally he will sing with me, but if he does, he doesn t want anybody to comment about it. I sing with the Kansas Mennonite Men s Chorus. We re a group of about 300 singers (all men, though not all Kansan and not all Mennonite) that sing for charity. We ve raised over $600,000 for charity so far. I still remember the first time I went to a KMMC practice 3 years ago. It was my first year singing, and the first practice of the season. We started singing the first song, and wow what a powerful moment. Even at its imperfect first practice state, hearing 300 people sing quietly is a powerful sound and when they get to the top of their lungs, it s indescribable. No CD can ever quite do that justice. Jacob and Oliver come to KMMC concerts sometimes, but this year they took a special interest. Oliver pointed to, hm, perhaps 300 men and said dad for each one. (He was quite far away and probably couldn t pick me out specifically.) Jacob, for the first time, sat quietly engrossed in the concert until we got to Dry Bones (a song involving surprise and lots of homemade instruments) which made him laugh out loud. Then he would hide under his seat from the applause. Last week, KMMC went on tour. This was my first choir tour, and Terah came along while the boys spent time with grandparents in Kansas. We sang in Bloomington, IL, and then in Goshen, IN at the fairly new Sauder Concert Hall. That hall is a few years old, and has been reviewed as once of the best concert halls in the world acoustically. It seats 1100 people and was perfect for this. The 85 or so singers from KMMC that went on tour joined the Indiana and Ohio Men s Choruses to make a combined group of about 200 singers. And what an experience that was. I learned later this was the first event to completely pack Sauder Concert Hall. We ended the concert singing the Mennonite Anthem a souped-up version of the doxology. (You can see a video of a different group singing it to get an idea of what it was). This is often joked about as being part of Mennonite DNA. You can gather a random group of Mennonites and ask to hear 606 (its number in, well, an old hymnbook that we don t use anymore) and you can probably get a fairly well-sounding rendition, from memory, a cappella, complete with 4-part harmony, at the drop of the hat. Mennonite youth have been known to sing it in train stations, airports, soccer stadiums, and with kazoos. At Goshen, we had 200 men on stage, plus a brass band, plus 1100 people in the audience facing us. They were, of course, invited to sing along. (That s one of the rules of 606: you can t exclude anybody because even if you tried, they d still sing, because how can you resist?) So I was up there on stage, singing a song I love, and hearing the 1100 people in the audience sing it to me. I could glance out over the front rows and saw the smiles on so many faces as they sang, and what a moving moment that was. The excitement and thrill of it all hit me so much that I could barely finish the song. The numbers aren t all in yet, but I m guessing that we raised over $15,000 for charity last weekend (the choir members paid their own expenses in advance, so 100% of the money raised goes to international relief efforts.) A 4-year-old singing alone in his room is really heartwarming. I have to say that 1300 people singing a song that touches each one of them is a pretty close second.

Joachim Breitner: Better PDF screenshots with gtk 3

Half a year ago, I experimented with gtk+ and cairo to extract a PDF screenshot out of a progam, i.e. a screenshot of the window that is not a table of colored dots as a regular PNG file would be, but rather a vector graphic that scales perfectly and has selectable text. The main use that I see is documentation: A scalable screenshots looks much better in print, and even in an online-PDF-file, having the full-text-search also find the text in the images is useful. My experiments back then were successful in a proof-of-concept way, but they were very ugly hacks. With the new version 3 of gtk+, though, things become much easier, as only one cairo context is used, instead of each widget creating its own context to draw on. By now, the first gtk-3 applications have reached Debian unstable, so I thought it would be time to re-do the experiment. And the result is very pleasing: The pdf screenshot program comes as a gtk module that you can load by setting an environment variable before launching a gtk-3 program. The application will run as usual, but there is an additional PDF screenshot button. If you click it, you get a Save File dialog (with a nice, live-updated preview of the screenshot, as an additional gimmick) that lets you save a PDF file. Its as easy as that. You can check out the resulting PDF file of gucharmap and a screencast of me creating that file, including copy n pasting from the resulting document.
<video controls="true" height="592" preload="none" src="http://www.joachim-breitner.de/various/pdf_screenshot_3.ogv" tabindex="0" width="800"> Your browser does not support the video tag, but you can still download and view the screencast with the above link. </video>
PS: I would offer the video as well on YouTube, but it seems it does not accept Ogg Theora videos. I hope all of you can make use of the above links. Update: By popular demand, I also created a SVG version. It even looks better because the embedded pixmaps have a higher resolution probably gtk is confused by the resolution of the PDF cairo surface. The code is almost trivially the same. I guess I could add other formats (PNG, PS) as well. Now I just have to find out how to tame GtkFileChooser to offer this feature better.

Joachim Breitner: Better PDF screenshots with gtk 3

Half a year ago, I experimented with gtk+ and cairo to extract a PDF screenshot out of a progam, i.e. a screenshot of the window that is not a table of colored dots as a regular PNG file would be, but rather a vector graphic that scales perfectly and has selectable text. The main use that I see is documentation: A scalable screenshots looks much better in print, and even in an online-PDF-file, having the full-text-search also find the text in the images is useful. My experiments back then were successful in a proof-of-concept way, but they were very ugly hacks. With the new version 3 of gtk+, though, things become much easier, as only one cairo context is used, instead of each widget creating its own context to draw on. By now, the first gtk-3 applications have reached Debian unstable, so I thought it would be time to re-do the experiment. And the result is very pleasing: The pdf screenshot program comes as a gtk module that you can load by setting an environment variable before launching a gtk-3 program. The application will run as usual, but there is an additional PDF screenshot button. If you click it, you get a Save File dialog (with a nice, live-updated preview of the screenshot, as an additional gimmick) that lets you save a PDF file. Its as easy as that. You can check out the resulting PDF file of gucharmap and a screencast of me creating that file, including copy n pasting from the resulting document.
<video controls="true" height="592" preload="none" src="http://www.joachim-breitner.de/various/pdf_screenshot_3.ogv" tabindex="0" width="800"> Your browser does not support the video tag, but you can still download and view the screencast with the above link. </video>
PS: I would offer the video as well on YouTube, but it seems it does not accept Ogg Theora videos. I hope all of you can make use of the above links.

17 June 2011

Timo Jyrinki: Betawikiss tuhannen euron porkkana

--- cut ---
Vuoden 2007 lopussa Siebrand asetti lokalisointitavoitteet MediaWikille.Tavoitteet olivat todella kunnianhimoisia. N ytt kin silt , ett n ihin ei p st . Emme kuitenkaan aio luovuttaa ilman taistelua. Viel on vajaa viikko aikaa saavuttaa nuo tavoitteet. Sin voit auttaa saavuttamaan tavoitteet suomen kielen osalta.

Yhteisty ss Stichting Open Progressin kanssa pystymme tarjoamaan sinulle kannustusta. Tarjoamme 1000 euroa jaettuna kaikkien k nt jien kesken, jotka ovat tehneet yli 500 uutta k nn st MediaWikiin ja sen laajennuksiin vuoden loppuun menness .

My s muut Betawikin projektit kaipaavat apua. Kiinnostaisiko sinua k nt vaikkapa FreeCol-peli tai Mantis-virheenseurantaj rjestelm . Yst v llinen ja yhteisty kykyinen ymp rist saattaa sinut vauhtiin sek auttaa sinua kehittym n suomentajana parantamalla k nn ksi yhdess .

Betawiki l ytyy osoitteesta http://translatewiki.net.
--- cut ---
(l hde)

Katso my s http://lokalisointi.org/.

Next.

Previous.