This GUADEC there will be a couple of sessions on Friday afternoon from 2pm about Wayland. I ll be giving a presentation with a brief introduction to what Wayland is, what new features we ve worked on in the last cycle as well as what s planned for the next one. As this is GUADEC i ll of course be covering how we re doing with getting Wayland integrated into GNOME. There will also be a Wayland panel discussion where you can ask your tricky questions of myself, Owen Taylor, Robert Bragg and Kristian H gsberg to get things started i ve got some already prepared! And if that s not enough Wayland for you, on Monday we ll be BoF ing between 10am and 2pm in room A218. It would be great to see you there.

So, this weekend I headed up to Uncle Steve's and assisted in the laying of the flagstones for the patio for the BBQ - there's even at least one bit of photographic evidence of me in a non-supervisory capacity. Was a good weekend, and fun was had by all, even if we couldn't really express it due to Complete Exhaustion by the end of it. Only another few weekends and it'll be the Debian UK Party 2013 which will, as it is every year, be absolutely awesome. I'll leave the thanks for the beer and food donations until after the party :)

Continuing my HOWTO series, in this article, I will explore how to set up syslog-ng to process JSON in various scenarios. We will touch the subject of interoperability, and have a look at multiple scenarios, in the hope that readers will find something similar to their own needs. There are many different ways in which one can plug JSON into their logging system, and it is very likely that I will not be able to cover it all within this HOWTO. However, I will explain the basics, and how to plug the pieces together, which should at least be a useful foundation to build even more complex systems onto.In essence, we will look at the following scenarios: raw JSON transport, JSON over syslog, JSON and syslog payload in the same stream. The documentation will cover both syslog-ng 3.4 and 3.5, where the 3.5 parts will be clearly marked as such. The 3.3 series do not have a JSON parser, and as such, is insufficient for most needs explored in this guide.In the default setting, the JSON parser will parse the MESSAGE part of an incoming log event, and place every key into a macro of the same name. If the incoming JSON is structured, it the keys will be converted to dot notation format: In this case, one can access the value as $ deeper.level within a syslog-ng template.The parsed keys can be prefixed using the prefix() option of the json-parser. One can also change what the parser will receive as input by using the template() option.See the documentation for more information on these.Transporting raw JSONThe simplest scenario is to take a transport like TCP, and simply push raw JSON structures through, one each line, and have syslog-ng parse that to its internal representation. One can do all kinds of things with the data then: transport it further as JSON, store it in files, in a database - the possibilities are endless, but we will explore two scenarios.JSON on TCP, using syslog field namesThe first one is receiving JSON on TCP, using syslog field names, store the parsed stuff in a file (using the conventional format), and in mongodb.

source s_tcp_json   tcp(port(10514) flags(no-parse));  ;
parser p_json   json-parser();  ;
destination d_file   file("/var/log/remote.log");  ;
destination d_mongodb   mongodb(collection("remote_log"));  ;
log  
 source(s_tcp_json);
 parser(p_json);
 destination(d_file);
 destination(d_mongodb);
 ;

This needs a bit of an explanation: we use the no-parse flag on the source, because by default, the tcp() source expects a message that conforms to the old syslog protocol. We have raw JSON, and we need to tell the source that. The other elements should be self explanatory, I believe, with the exception of json-parser(): we give it no parameters, so it will not modify the keys in the JSON it receives, keeping them intact, as-is.For this reason, incoming messages need to have the same keys as syslog-ng normally would use internally: DATE, HOST, MESSAGE, and so on, like in the following example: JSON on TCP, custom field namesA more likely scenario is when we receive JSON, but not in a layout syslog-ng uses internally. In this case, we need to rewrite the message a little, and possibly use templates, depending on what kind of output we wish to use.Lets assume the incoming JSON has these fields: As you can see, this has similar fields, but not quite, and FACILITY is missing too.In this case, we'll use a different configuration:

source s_tcp_json   tcp(port(10514) flags(no-parse));  ;
parser p_json   json-parser(prefix(".json."));  ;
destination d_file   file("/var/log/remote.log"
                          template("$ .json.timestamp  $ .json.source  $ .json.app [$ .json.id ]: $ .json.msg \n"));  ;
destination d_mongodb   mongodb(collection("remote_log")
                                value-pairs(
                                  pair("PROGRAM" "$ .json.app ")
                                  pair("HOST" "$ .json.source ")
                                  pair("PID" "$ .json.id ")
                                  pair("MESSAGE" "$ .json.msg ")
                                  pair("DATE" "$ .json.timestamp ")
                                  pair("PRIORITY" "$ .json.prio ")
                                  pair("FACILITY" "auth")
                                ));  ;
log  
 source(s_tcp_json);
 parser(p_json);
 destination(d_file);
 destination(d_mongodb);
 ;

We use the prefix() option of json-parser(), to avoid collisions with the default syslog-ng namespace. Then, we use a template for the file destination, and value-pairs() for the MongoDB one. Doing all this, we get similar results as we did in the first scenario.JSON over syslogAn even likelier scenario is when we have JSON payload in the message part of a normal syslog message. This is useful because the logs themselves remain syslog compatible, and can be transported through devices that know no better.Thankfully, if we do not tell syslog-ng that we do not want any parsing on the input, the json-parser will receive the MESSAGE part of the log, so if we have JSON payload there, things will just work:

source s_tcp_json_payload   tcp();  ;
parser p_json   json-parser();  ;
destination d_file   file("/var/log/remote.log");  ;
destination d_mongodb   mongodb(collection("remote_log"));  ;
log  
 source(s_tcp_json_payload);
 parser(p_json);
 destination(d_file);
 destination(d_mongodb);
 ;

Again, this assumes that the JSON payloads uses field names compatible with syslog-ng's defaults. Adapting it to use templates or value-pairs can of course be made, as shown in an example above.As an example, here's how a syslog message with JSON payload would look like:

<38>2013-07-22T13:45:54 localhost prg00000[1234]:  "MESSAGE": "foo", "PROGRAM": "bar" 

This would, of course, override the MESSAGE and PROGRAM fields, but the rest would be kept intact.Mixed JSON and syslog streamIn order to parse JSON that is intermingled with normal syslog streams, we can use a new feature of syslog-ng 3.4: junctions. These allow us to create branches, and tell syslog-ng something along these lines:

If there's an incoming message, that begins with @json:, parse it as JSON, otherwise treat it as a normal syslog message.

We can, of course, skip the part that mandates an in-stream marker, and things will just work, but in that case, for each JSON message, an error will get logged too, which is undesirable. Therefore, if one wants to intermingle JSON and syslog traffic on the same wire, it's best to do so with the JSON having a marker prefix. The marker, however, is not going to be part of the string the json parser receives, so setting it to an opening curly bracket will not work, unless the template is modified accordingly.The config to do this looks like this:

block parser mixed-json-parser()  
 channel  
  junction  
   channel  
    parser   json-parser(marker("@json:"));  ;
    rewrite   set-tag(".json");  ;
    flags(final);
    ;
   channel  
    flags(final);
    ;
   ;
  ;
 ;
source s_mixed  
 tcp();
 parser   mixed-json-parser();  ;
 ;

A more elaborate example can be found in a post by Bazsi.Closing thoughtsThat covers most of the use cases I saw for the JSON parser, if there are cases uncovered, or questions unanswered, I can be reached via twitter, email and a whole host of other ways, and I will happily expand this tutorial if there's enough demand for more.

I have finished preparing the beta 2 of the next release of OpenStack. It is currently only available from out Git on Alioth (in /git/openstack), and directly from my jenkins repository, which creates Wheezy backports for it:

deb ftp://havana.pkgs.enovance.com/debian havana main
deb http://archive.gplhost.com/debian havana-backports main

As for every OpenStack release, a large number of Python modules needed to be packaged and are waiting in the FTP master NEW queue to be approved: oslo-sphinx, python-django-discover-runner, python-hacking, python-jsonrpclib, python-lesscpy, python-neutronclient, python-nosehtmloutput, python-requestbuilder, python-termcolor, sphinxcontrib-httpdomain and sphinxcontrib-pecanwsme. Let s hope that they will be approved before the next beta release in september (time where OpenStack Havana will be in feature freeze). The total number of packages maintained by the OpenStack team (on which I really am the only active maintainer for the moment ), there s 53 packages, plus these 11 packages waiting in the NEW queue. That s a big number of package, and I wouldn t mind some help One thing that annoyed all the community is that Quantum, the OpenStack network virtualization module, had to be renamed as Neutron, because of a trademark from Quantum (you probably remember the Quantum Fireball hard drives? Well, it s the same company ). Another good news is that my openstack-debian-images package has just been approved and landed in Sid. With it, you can automate the process of building a Debian image for OpenStack with a simple shell command (there s a man page that I wrote with it: read it if you need to build images). It is made of a single shell script to build the image, using kpartx, parted, mbr, debootstrap, extlinux and friends. I tried to keep it simple, not involving a huge amount of components. With the release of cloud-init 0.7.2-3, I have fixed a few bugs (3 important bugs, out of which 2 where RC bugs), thanks to contributions in the debian-cloud@lists.debian.org mailing list. This includes adding new init.d scripts, so we now have support for user data. This doesn t only benefits OpenStack images, but anyone willing to start virtual machines in the cloud (nowadays, every cloud implementation needs cloud-init installed in the virtual images). This means you can include a script in the metadata of the virtual machine you start it, and it will be executed at startup. If everything goes as planned (that is, no new RC bug), I will upload an update of cloud-init to backports in 5 days (there is already a version there, but it doesn t have the necessary init.d scripts to execute the user data scripts), and openstack-debian-images in 9 days. Then it will be possible to build OpenStack images with absolutely all the tools available from Wheezy (and backports). I hope to be able to discuss that during Debconf 13.

As a Sunday diversion, I wrote 150 lines of code and turned git-annex into a podcatcher! I've been using hpodder, a podcatcher written in Haskell. But John Goerzen hasn't had time to maintain it, and it fell out of Debian a while ago. John suggested I maintain it, but I have not found the time, and it'd be another mass of code for me to learn and worry about. Also, hpodder has some misfeatures common to the "podcatcher" genre:

• It has some kind of database of feeds and what files have been downloaded from them. And this requires an interface around adding feeds, removing feeds, changing urls, etc.
• Due to it using a database, there's no particularly good way to run it on the same feeds on multiple computers and sync the results in some way.
• It doesn't use git annex addurl to register the url where a file came from, so when I check files in with git-annex after the fact they're missing that useful metadata and I can't just git annex get them to re-download them from the podcast.

So, here's a rethink of the podcatcher genre:

cd annex; git annex importfeed http://url/to/podcast http://another/podcast

There is no database of feeds at all. Although of course you can check a list of them right into the same git repository, next to the files it adds. git-annex already keeps track of urls associated with content, so it reuses that to know which urls it's already downloaded. So when you're done with a podcast file and delete it, it won't download it again. This is a podcatcher that doesn't need to actually download podcast files! With --fast, it only records the existence of files in git, so git annex get will download them from the web (or perhaps from a nearer location that git-annex knows about). Took just 3 hours to write, and that's including full control over the filenames it uses (--template='$ feedtitle)/$ itemtitle $ extension '), and automatic resuming of interrupted downloads. Most of what I needed was already available in git-annex's utility libraries or Hackage. Technically, the only part of this that was hard at all was efficiently querying the git repository for a list of all known urls. I found a pretty fast way to do it, but might add a local cache file later on.

Given a model:
class PASPPoint(models.Model): point = models.PointField() when = DateTimeField() user = models.ForeignKey(User) objects = models.GeoManager()
The following Query:
counts = PASPPoint.objects.filter( user=user, when__year= 2013 , when__month= 4 , ).extra( day : date_trunc( day , when) ).values( day ).order_by().annotate(count=Count( id ))
will error out with a SyntaxError. It didn t hit me until some PostgreSQL debugging that this was because when is a SQL-ish word. D oh. Changing when to timestamp fixed the problem.

FISL (pronounced FIZZ lay by the locals) is a large Free and Open Source software gathering in Porto Alegre, Brazil. While the primary audience of that conference is the Latin America "libre software" community, a number of overseas speakers also participate in that conference. This is my contribution to them: a short guide for international guests visiting the F rum Internacional do Software Livre.

Planning Before you fly out, make sure you look up the visa requirements for your country of citizenship. Many western countries will require a visa and you will need to visit the local Brazillian embassy ahead of time to get one. Next, have a look at the list of recommended immunizations. As with most destinations, it is recommended that your routine immunizations be up to date, but there are also other specialized ones such as Yellow Fever that are recommended by the Brazillian government. You should therefore visit a travel clinic a few weeks ahead of time. Other than that, I suggest reading up on the country and keeping an eye on the various travel advisories.

Arrival You will be flying at the Porto Alegre airport. If you need to exchange overseas money for Brazilian Reals, you can do that there. You'll probably also want to pick up a power adapter at the airport if you intend to charge your laptop while you're in the country Brazil has both 127V and 220V outlets using Type N sockets. Privacy note: using the free airport wifi will require giving your Passport details as part of the registration process.

Language If you don't speak Portuguese, expect a few challenges since most of the people you'll meet (including taxi drivers, many airport workers, some hotel staff) won't speak English. I highly recommend getting a phrase book before you leave and printing paper maps of where you are planning to go (to show to taxi drivers when you get lost). Native Spanish speakers seem to get by speaking Spanish to Portuguese speakers and understanding enough Portuguese to hold a conversation. I wouldn't count on it unless your Spanish is quite good though. Also, the official conference blog posts get eventually translated to English, but there is a delay, so you may want to subscribe to the Portuguese feed and use Google Translate to keep up with FISL news before you get there.

The conference FISL is a large conference and it has a very "decentralized" feel to it. From the outside, it looks like it's organized by an army of volunteers where everyone is taking care of some portion of it without a whole lot of top-down direction. It seems to work quite well! What this means for you as a foreign speaker however is that you're unlikely to be provided with a lot of information or help finding your way around the conference (i.e. no "speaker liaison"). There is a separate registration desk for speakers but that's about all of the attention you'll receive before you deliver your talk. So make sure you know where to go and show up in your assigned room early to speak with the person introducing you. If your talk is in English, it will be live-transated by an interpreter. It's therefore a good idea to speak a bit more slowly and to pause a bit more. Other than that, the organizers make an effort to schedule an English talk in each timeslot so non-Portuguese speakers should still be able to get a lot out of the conference. FISL was a lot of fun for me and I hope that some of these tips will help you enjoy the biggest FLOSS gathering in the southern hemisphere!

Last week, the second milestone of the Havana developement branch of Ceilometer has been released and is now available for testing and download. This means the first half of the OpenStack Havana development has passed! New features Ten blueprints have been implemented as you can see on the release page. I'm going to talk through some of them here, that are the most interesting for users. The Ceilometer API now returns all the samples sorted by timestamp. This blueprint is the first one implemented by Terri Yu, our OPW intern! In the same spirit, I've added the ability to limit the number of samples returned. On the alarming front, things evolved a lot. I've implemented the notifier system that will be used to run actions when alarms are triggered. To trigger these alarms, Eoghan Glynn (RedHat) worked on the alarm evaluation system that will use the Ceilometer API to check for alarm states. I've reworked the publisher system so it now uses URL formatted target for publication. That now allows to publish different meters to different target using the same publishing protocol (e.g. via UDP toward different hosts). Sandy Walsh (RackSpace) have been working on the StackTach like functionality and added the ability for the collector to optionally store the notification events received. Finally, Mehdi Abaakouk (eNovance) implemented a TTL system for the database, so you're now able to expire your data whenever you like. Bug fixes Thirty-five bugs were fixed, though most of them might not interest you so I won't elaborate too much on that. Go read the list if you are curious. Toward Havana 3 We now have 30 blueprints targeting the Ceilometer's third Havana milestone, with some of them are already started.I'll try to make sure we'll get there without too much trouble for the 6th September 2013. Stay tuned!

The first wheezy based beta release of Debian Edu was wrapped up today. This is the release announcement: New features for Debian Edu 7.1+edu0~b0 released 2013-07-27 These are the release notes for for Debian Edu / Skolelinux 7.1+edu0~b0, based on Debian with codename "Wheezy". About Debian Edu and Skolelinux Debian Edu, also known as Skolelinux, is a Linux distribution based on Debian providing an out-of-the box environment of a completely configured school network. Immediately after installation a school server running all services needed for a school network is set up just waiting for users and machines being added via GOsa , a comfortable Web-UI. A netbooting environment is prepared using PXE, so after initial installation of the main server from CD, DVD or USB stick all other machines can be installed via the network. The provided school server provides LDAP database and Kerberos authentication service, centralized home directories, DHCP server, web proxy and many other services. The desktop contains more than 60 educational software packages and more are available from the Debian archive, and schools can choose between KDE, Gnome, LXDE and Xfce desktop environment. This is the fifth test release based on Debian Wheezy. Basically this is an updated and slightly improved version compared to the Squeeze release. ALERT: Alpha based installations should reinstall or downgrade the versions of gosa and libpam-mklocaluser to the ones used in this beta release. Software updates

• Switched roaming workstation profiles from wicd to network-manager for network configuration, as wicd didn't work any more.
• Changed version numbers of patched gosa and libpam-mklocaluser packages to make sure our locally patched versions will be replaced by the official packages when they are released from Debian. Those installing alpha version need to reinstall or manually downgrade gosa and libpam-mklocaluser.
• Added bluetooth tools to the default desktop (bluedevil, blueman).
• Added tools for sharing the desktop on KDE (krdc, krfb).
• Added valgrind to the default installation for easier debugging of crash bugs.

Other changes

• Fixed artwork package to work with gnome, no longer break desktop=gnome installations.
• Adjusted installer to now work when forced to use a proxy with the netinst CD.
• Fixed code detecting and setting/loading hardware specific setup/firmware to work more robust out of the box.
• Adjusted Kerberos setup to detect realm and server settings at install time instead of dynamically at run time. This avoid a crash with krb5-auth-dialog on diskless workstations without a DNS name.
• Worked around misfeature in network-manager not calling the dhclient exit hooks, causing automatic proxy configuration and automatic host name setting at run time to work again.
• Fixed feature setting the default Iceweasel start page from URL fetched from LDAP, to allow schools to set the global default by updating the dc=skole,dc=skolelinux,dc=no LDAP object.
• Changed default host name on all networked machines to be unique (generated from MAC or reverse DNS) after boot.
• Adjusted partition sizes to make sure they are big enough.

Known issues

• Grub is missing the new artwork.
• KDE fail to understand the wpad.dat file provided, causing it to not use the http proxy as it should.
• Chromium also fail to use the proxy.

Where to get it To download the multiarch netinstall CD release you can use

• ftp://ftp.skolelinux.org/skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-CD.iso
• http://ftp.skolelinux.org/skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-CD.iso
• rsync -avzP ftp.skolelinux.org::skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-CD.iso .

The MD5SUM of this image is: 55d5de9765b6dccd5d9ec33cf1a07109
The SHA1SUM of this image is: 996a1d9517740e4d627d100de2d12b23dd545a3f To download the multiarch USB stick ISO release you can use

• ftp://ftp.skolelinux.org/skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-USB.iso
• http://ftp.skolelinux.org/skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-USB.iso
• rsync -avzP ftp.skolelinux.org::skolelinux-cd/wheezy/debian-edu-7.1+edu0~b0-USB.iso .

The MD5SUM of this image is: d8f0818c51a78d357de794066f289f69
The SHA1SUM of this image is: 49185ca354e8d0543240423746924f76a6cee733 How to report bugs http://wiki.debian.org/DebianEdu/HowTo/ReportBugs

My last running update was sent after I had a great April month with two successful trail races (35km then 44km). The next target was Mont-Blanc marathon in Chamonix valley in early July, so it's time for me to send a new update before summer. May and June was mostly training. Huge training. Very long distances with 375km in May and over 430 in June (breaking my former monthly record by over 50 kilometers). Yes, that means over 14 kilometers every day..:-) I indeed achieved that with my "go to work partly by running" habits, which are now deeply included in my daily schedule. Much more convenient than bus ride (with the typical random schedules of RATP, which French people translate by "Rentre Avec Tes Pieds" or "Come back home by walking"....which is exactly what I'm doing). With such training (sometimes including extra rides in the forests, with short but steep ups and downs)....I was well prepared for THE early summer goal : Marathon du Mont-Blanc in Chamonix (not Ultra Trail du Mont-blanc : this one, I'm not ready for....yet).. My first mountain trail race, and what race! Imagine 42km in the moutain lovers heaven, namely the Chamonix Valley, with the Mont-Blanc, Aiguille Verte, Aiguille du Midi, Aiguilles Rouges, etc. above your head. Of course, not only in the valley but goind up and down "slightly"....in short 2200 meters positive climb..:-) I ran it with a female friend of mine, who I met in local races and through runners web forums. Sabine has about the same running skills than me and we developed a good and nice friendship, sharing our love for running in the nature. So, during May and June, we decided to run this race together as we were both registered for it. A first trial in a local 35km trail race two weeks before the MMB was very successful. We ended up in 3h40 and Sabine was ranked 5th female runner. And, moreover, we had a tremendous pleasure by crossing the finishing line together. So, we did it again in Chamonix. And, hell, we did it well: I had set the goal to 6 hours and 30 minutes and we completed the race in..... 6 hours and 34 minutes. Much much better than I and she would have expected. And, believe me, crossing the finishing line was again a special moment, particularly because the crowd at the end of this race make it look like l'Alpe d'Huez on the Tour de France. And, this on a wonderful sunny day, arriving at Planpraz, facing the Mont-Blanc. If you've ever been to Chamonix, you know what I mean. So, first moutain trail and, hell, not the last one! For those of you who can read French...or just want to see some pictures of a running bubulle, here is the link to the story. And these 42 kilometers were indeed so well done that the day after, while coming down from the famous Aiguille du Midi with Elisabeth (who came with me to Chamonix, of course)...I decided to run down from the cable car intermediate stop (2300m altitude) to Chamonix (1050m). Over 1200 meters down in 48 minutes... :-). Just a bit slower than the cable car....:) The remaining of July saw me again running back and forth to work and I'm indeed about to break my monthly record, eventually reaching 450 kilometers in one month and only two days NOT running in the whole month. Yeah, crazy. The upcoming Sunday, August 4th, just before going to Debconf 13, I'll spend a few days in Lanslebourg and will be running the EDF Cenis Tour trail, for 50 kilometers and 2600 meters positive climb. This time, a very "small" race, with no more than 150 runners. So, that will be another experience by running nearly entirely alone between 1400 and 2700 meters altitude. I'll post the outcome (or you'll probably hear about it at DC13....). See you, hopefully not in 3 months, for another update about running Bubulle.

This is time again for my annual report about the number of developers per country. This is now the fifth edition of this report, after the 2012, 2011, 2010 and 2009 editions. So, here we are with the July 2013 version, sorted by the ratio of *active* developers per million population for each country.

Act: number of active developers
Dev: total number of developers
A/M: number of active devels per million pop.
D/M: number of devels per million pop.
2009: rank in 2009
2010: rank in 2010
2011: rank in 2011 (June)
2011: rank in 2012 (June)
2013: rank now

Code	Name	Population	Act Dev	Dev	Act/Million	Dev/Million	2009	2010	June 2011	June 2012	July 2013
fi	Finland	5259250	19	31	3,61	5,89	1	1	1	1	1
ie	Ireland	4670976	12	15	2,57	3,21	13	9	6	2	2
ch	Switzerland	7870134	20	27	2,54	3,43	2	2	2	3	3
mq	Martinique	396404	1	1	2,52	2,52			3	4	4
se	Sweden	9088728	22	37	2,42	4,07	3	6	7	5	5
no	Norway	4973029	12	15	2,41	3,02	5	4	4	6	6
nz	New Zealand	4331600	10	15	2,31	3,46	4	3	5	7	7
de	Germany	81471834	165	235	2,03	2,88	7	7	9	9	8	*
lu	Luxemburg	503302	1	1	1,99	1,99	8	5	8	8	9	*
at	Austria	8217280	16	26	1,95	3,16	6	8	10	10	10
fr	France	65350000	100	130	1,53	1,99	12	12	11	11	11
au	Australia	22607571	31	59	1,37	2,61	9	10	12	12	12
be	Belgium	11071483	14	17	1,26	1,54	10	11	13	13	13
uk	United-Kingdom	62698362	73	114	1,16	1,82	14	14	14	14	14
nl	Netherlands	16728091	19	40	1,14	2,39	11	13	15	15	15
ca	Canada	33476688	32	61	0,96	1,82	15	15	17	16	16
dk	Denmark	5529888	5	10	0,9	1,81	17	17	16	17	17
es	Spain	46754784	35	56	0,75	1,2	16	16	19	18	18
it	Italy	59464644	36	51	0,61	0,86	23	22	22	19	19
cz	Czech Rep	10190213	6	6	0,59	0,59	21	20	21	21	20	*
lt	Lithuania	3535547	2	2	0,57	0,57	28	19	20	22	21	*
us	USA	313232044	173	378	0,55	1,21	19	21	25	24	22	*
il	Israel	7740900	4	6	0,52	0,78	24	24	24	25	23	*
hu	Hungary	10076062	5	10	0,5	0,99	18	25	26	20	24	*
hr	Croatia	4290612	2	2	0,47	0,47	20	18	18	26	25	*
lv	Latvia	2204708	1	1	0,45	0,45	26	26	27	27	26	*
bg	Bulgaria	7364570	3	3	0,41	0,41	25	23	23	23	27	*
uy	Uruguay	3477778	1	2	0,29	0,58	22	27	28	28	28
jp	Japan	127078679	35	50	0,28	0,39	30	28	29	29	29
pl	Poland	38441588	10	14	0,26	0,36	29	29	30	30	30
cr	Costa Rica	4301712	1	1	0,23	0,23	31	30	31	31	31
by	Belarus	9577552	2	2	0,21	0,21	35	36	39	39	32	*
sg	Singapore	5183700	1	1	0,19	0,19				33	33
pt	Portugal	10561614	2	4	0,19	0,38	27	32	32	34	34
gr	Greece	10787690	2	3	0,19	0,28	33	38	34	35	35
sk	Slovakia	5477038	1	1	0,18	0,18	32	31	33	36	36
ar	Argentina	40677348	6	8	0,15	0,2	34	33	35	32	37	*
rs	Serbia	7186862	1	1	0,14	0,14					38	*
tw	Taiwan	23040040	3	3	0,13	0,13	37	34	37	37	39	*
br	Brazil	192376496	21	23	0,11	0,12	36	35	38	38	40	*
cu	Cuba	11241161	1	1	0,09	0,09		38	41	41	41
kr	South Korea	48754657	4	6	0,08	0,12	39	39	42	42	42
gt	Guatemala	13824463	1	1	0,07	0,07					43	*
ve	Venezuela	28047938	2	2	0,07	0,07	40	45	50	49	44	*
ec	Ecuador	15007343	1	1	0,07	0,07		40	43	43	45	*
co	Colombia	45566856	3	4	0,07	0,09	41	44	46	47	46	*
cl	Chile	16746491	1	2	0,06	0,12	42	41	44	44	47	*
za	South Africa	50590000	3	10	0,06	0,2	38	48	48	48	48
ru	Russia	143030106	7	8	0,05	0,06	43	42	47	45	49	*
mg	Madagascar	21281844	1	1	0,05	0,05	44	37	40	40	50	*
ro	Romania	21904551	1	2	0,05	0,09	45	43	45	46	51	*
my	Malaysia	28250000	1	1	0,04	0,04			49	50	52	*
pe	Peru	29907003	1	1	0,03	0,03	46	46	51	51	53	*
tr	Turkey	74724269	2	2	0,03	0,03	47	47	52	52	54	*
ua	Ukraine	45134707	1	1	0,02	0,02	48	53	58	59	55	*
th	Thailand	66720153	1	2	0,01	0,03	50	50	54	54	56	*
eg	Egypt	80081093	1	3	0,01	0,04	51	51	55	55	57	*
mx	Mexico	112336538	1	1	0,01	0,01	49	49	53	53	58	*
cn	China	1344413526	10	14	0,01	0,01	53	53	57	56	59	*
in	India	1210193422	9	9	0,01	0,01	52	52	56	57	60	*
sv	El Salvador	7066403	0	1	0	0,14			36	58	61	*
			957	1535	62,35%

A few interesting facts:

• Rankings are very stable: the first 7 countries are the same
• Germany takes Luxemburg over with 10 more active developers than last year
• Finland is still ranked 1st but loses two active developers. Still, it would need to lose 6 developers to lose rank 1
• By losing one active developer, Hungary drops from 20th to 24th
• Similar loss for Bulgaria, dropping from 23rd to 27th
• Belarus doubles its number of developers from 1 to 2 and therefore jumps from rank 39 to 32
• Serbia gains its first developer (indeed, the person registered with the "yu" ISO code and I arbitrarily decided that would count for Serbia.....please correct me if I'm wrong) and shows up at rank 38
• Argentina sadly loses two active developers and rops to rank 37
• Venezuela doubles its number of developers from 1 to 2 and slightly climbs up
• Even though both of them gained one developer, the two most populated countries in the world, China and India, remain very low. China would indeed need 2730 developers to reach the ratio of Germany and India would need 2470.
• Again, one new country : Guatemala. Let's take this as an achievement of another DebConf in Central America, last year.
• Ukraine shows up again and I'm partly responsible for that as I processed Boris Pek's application as DD!
• This year, DebConf will be hosted in one of the top-ranked countries, namely Switzerland. Guys, 9 more developers and you can overtake Finland!
• We have 37 more active Debian developers and 45 more developers overall.
• The ratio of active developers increases from 61.74% to 62.35%

Here's something I just spent too much time figuring out... If you have an Australian "gas" or "LPG bottle"; the type you'd pick-up at any "service" or "petrol" station in Australia, it has a female POL connector. I'm not sure what POL stands for; possibly related to some company who made connectors long ago and it became standard. If you have an American "propane" device such as a grill, turkey fryer or smoker, or indeed something that plugs into a "propane bottle" you would pick-up at a "gas station" -- it will have a "Type 1", "ACME", "ACME Type 1", "QCC", "QCC1" or "QCC Type 1" connector (each seems to be an interchangeable term for the other). The appliance side of this has a big (usually black) plastic nut that is apparently safer or something. Once you have figured this out, you can buy something like the "Grillpro Model 11051 Universal Fit Propane Tank Adapter" and I can attest, based upon empirical evidence, that things will "just work". Of course your other choice might be just to replace the regulator, hoses, etc.; however if everything seems super-glued together the adapter is much easier. In other empirical evidence you won't find anywhere else on the internet : the Vizio M3D550KD LCD television works just fine on 240V, despite not being rated for it on the back.

Due to popular demand this weeks report contains another awesome ascii screenshot. I added some lines to /etc/hurd/runsystem.sysv to inspect the environment very early in the boot process:

start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] exec init proc auth
[...]
+ echo my pid is 1
my pid is 1
+ ps Ax
+ head
USER       PID TT STAT     TIME COMMAND
-            0  ? R<mo  0:00.01 /hurd/proc
root         1  - Sslow 0:00.02 /bin/sh /etc/hurd/runsystem
-            2  - Sp    0:00.00 /hurd/init root=device:hd0s1 console=com0
-            3  ? D<p   0:00.01 root=device:hd0s1 console=com0
-            4  - S<o   0:00.18 ext2fs --readonly --multiboot-command-line=root
-            5  - S<o   0:00.02 /hurd/exec
-            6  - S<o   0:00.00 /hurd/auth
root         7  - S<o   0:00.04 /hurd/term /dev/console device console
root         8  - S<o   0:00.00 /hurd/pflocal
+ pstree -p
sh(1)-+-init(2)-+-auth(6)
                 -ext2fs(4)-+-exec(5)
                             -pflocal(8)
                             -procfs(12)
                             -term(7)
       -pstree(11)
       -root=device:hd0s1(3)

Those who are familiar with the Hurd might note that /etc/hurd/runsystem.sysv runs as PID 1. At the end of this script /sbin/init is exec(2)uted, so sysvinit gets run as PID 1. This is the result of a fairly small patch series. It still took me like two days to come up with it since bootstrapping a Hurd system (or any microkernel OS, or like bootstrapping in general) is complicated. If you look at the screenshot above you will note the /hurd/pflocal process running with PID 8. This is the server providing pipe(2)s and Unix sockets. Before that server is started, one cannot use pipes to connect two processes with pipes like I did with ps and head (I actually lied a little here, because the way Debian/Hurd is set up, the pflocal server is started on demand courtesy of a Hurd feature called "passive translators", very similar to what the cool kids call "socket activation" today). Process 0 is the /hurd/proc server. It maps Mach tasks to the concept of Unix processes. Before that server is started, the notion of a process does not even exist. Pid 4 is the root filesystem. Without that, no other server can be read from the disk. But if the root filesystem is to be started first, it cannot be registered as a Unix process at the proc server, because that has not been started yet. So the first few processes are started in a weird way and order, and since they need to talk to each other, they need to register at each other. No Unix concepts exist yet, only Mach concepts like tasks and messages can be used. The boot sequence is roughly this (the numbers in parenthesis are the PIDs):

1. Grub loads GNU Mach(3), the statically linked root filesystem translator (4) and the /hurd/exec (5) server (the exec server loads executables into Mach tasks, implementing most parts of execve(2)).
2. The rootfs translator is started.
3. The rootfs initiates the Hurd server bootstrap by starting /hurd/exec and /hurd/init (2).
4. /hurd/init starts /hurd/proc (0) and /hurd/auth (6).
5. The translators perform several rendezvouses to introduce themselves to each other.
6. The proc servers state with respect to the already running processes is fixed.
7. /hurd/init starts /etc/hurd/runsystem (1) which starts the pager (that pages out memory to disk) and later execs /sbin/init.
8. From this point on the initialization of Debian/Hurd is no different from Debian/Linux.

Besides the pid one issue I spent my time on:

• A prototypical patch for the other sysvinit related issue, /sbin/killall5 wreaking havoc on a Hurd system: http://lists.gnu.org/archive/html/bug-hurd/2013-07/msg00226.html
• I updated my patch series binding the mtab translator to /proc/mounts: http://lists.gnu.org/archive/html/bug-hurd/2013-07/msg00216.html
• I updated my patch series for daemonizing support for console-client: http://lists.gnu.org/archive/html/bug-hurd/2013-07/msg00246.html

Next week I hope to fully address the killall5 issue and to clean up, polish and submit my patch series for sysvinit. I will then revise my mtab translator prototype which is currently in its third iteration. As always, if there is time left, I will look at the network related issues.

We are happy to announce that the schedule for the upcoming DebConf13 in Vaumarcus, Switzerland, is available! It is still possible for some changes to occur, as we always try to accept new proposals, even those submitted during the conference. DebConf13 talks will mostly happen in two rooms simultaneously, except for a few plenaries which will be presented in the main room with no parallel event. A third room will be available for discussion groups and ad-hoc sessions. As usual, we ve tried to cluster related activities in sequence, even though it was not always possible. Besides that there will be many tracks covering specific topics, which are:

• Debian for the cloud
• Building and porting
• Community outreach
• Real time communications
• Debian blends and derivatives
• Quality assurance
• Debian boot
• Debian in education
• Debian teams

There will also be two special tracks, since DebConf is not only about hacking, talks and discussions:

• Social activities, which includes a poetry night, the traditional cheese and wine party, our group photo, a day trip, the conference dinner etc.
• Debian 20^th birthday, an event open to the general public to celebrate Debian s 20^th birthday (details to come soon!).

As in previous years, we provide various ways to access the schedule:

• Plain nice html export, available for everyone without needing a login.
• The same as above, for people registered in our Pentabarf conference management system: This one has one extra feature compared to the non-login version: You can rate an event and its speaker(s) after you attended it. Please take the opportunity to do so, your feedback is very important both for the speaker(s) and for the organizing team!
• An iCal file
• An xCal file
• An XML file

the DebConf team

Dear lazy web, Friend of mine ask me for help to find out bottleneck of performance problems he s dealing with.
Basically it looks like the most problematic are I/O operations and main reason for that are thousands of read/write operations on directory having about 150k files. Solution is easy right? Spread those files among some hashed subdirectories and we re fine.
But it is not so easy. There are about 200 bash/perl scripts operating on those files.
So it s not so simple as mv aaaaaaaaaaaaa.txt a/aa/aaaaaaaaaaaaaa.txt and so on.
Those scripts were developed for the past 10 or more years so yes noone really knows how and why do they work. So I m thinking about another approach which should be much easier and cheaper than rewriting hundreds of scripts. Something like proxy filesystem between those scripts and files.
So we could still move those files to hashed subdirectories but show them as they are in single directory for the scripts. Anyone knows about such filesystem? Or is it something that has to be written? And another question is it worth to go this way? Is there a chance that some kind of FUSE proxy filesystem with really fast hashed subdirectories will be faster than filesystem with one directory with a lot of files? Anyone tried such approach?

I started writing code when I was somewhere around 1st grade, hacking some rudimentary BASIC on a TRS-80. Since then, I ve used more programming languages than I can count; everything from C to Prolog, Python to Haskell, and probably a dozen others. I enjoy diversity in programming languages and tools. I ve had an interest in both web and mobile development for some time, but have done little with either. Reflecting on that lately, I realize that both platforms severely restrict my freedom to use the tools I choose. Even on an old x86 with 640K of RAM, I had choices; BASIC, Pascal, and C would work under DOS. (Plus assembler, but I had yet to learn that at the time.) On Android, my choices are Java or Java. (OK, or something that compiles to Java, but those selections are still limited.) Plus it seems like I d have to use Eclipse, which seems to have taken the kitchen sync crown from Emacs long ago, and not in a good way. On iOS, the choices are Objective C. And on the web, it s JavaScript. The problem with all this is that there s little room for language innovation on those platforms. If predictions about the decline of development on the PC pan out, how will the next advance in programming languages gain traction if the hot spots for development are locked down to three main languages? Will we find ourselves only willing to consider languages that can compile to Java/Dalvik bytecode? And won t that put limits on the possible space for innovation to occur within? Yes, I know about things like Mono and the Android Scripting Environment, which offer some potential for hope, but I think it s still safe to say that these platforms are pretty closed to development in unofficial languages. I don t know of major apps in the Android market written in Python, for instance. I know of efforts such as Ubuntu s, but even they simply lock down development to a different toolkit. There is no choice there of GTK vs. Qt, or such things. Sadly, I don t think it has to be this way. My phone has way more power than my computer did just a few years ago, and that computer was capable of not just executing, but even compiling, code written in all sorts of different languages. But I don t know if there s much we can do to change things. Even with native code on Android, plenty still has to go through Dalvik (at least that s my understanding). If you wanted to write a Python app with a full graphical touch interface on Android, I don t think this would be all that easy of a task.

Weblate 1.6 has been released today. It comes with lot of improvements, especially adds voting for suggestions, improves import performance and brings more helpful admin interface. This is also first release with crowdfunded feature, but there are still some features to fund! Full list of changes for 1.6:

• Nicer error handling on registration.
• Browsing of changes.
• Fixed sorting of machine translation suggestions.
• Improved support for MyMemory machine translation.
• Added support for Amagama machine translation.
• Various optimizations on frequently used pages.
• Highlights searched phrase in search results.
• Support for automatic fixups while saving the message.
• Tracking of translation history and option to revert it.
• Added support for Google Translate API.
• Added support for managing SSH host keys.
• Various form validation improvements.
• Various quality checks improvements.
• Performance improvements for import.
• Added support for voting on suggestions.
• Cleanup of admin interface.

You can find more information about Weblate on it's website, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Ready to run appliances will be soon available in SUSE Studio Gallery. Weblate is also being used https://l10n.cihar.com/ as official translating service for phpMyAdmin, Gammu, Weblate itself and others. If you are free software project which would like to use Weblate, I'm happy to help you with set up or even host Weblate for you.

Filed under: English Phpmyadmin Weblate 0 comments Flattr this!

There will come a time when you believe everything is finished. That will be the beginning. Louis L Amour

Last year, I wrote about the difficult times in my life and hope for future, but never really explained why. I have written little since, because there is little I can write without a bit of explanation. It is partly because of the complexity of the task of telling my story without telling too much of others stories. But it is important I tell this story. So often on the Internet, we hear only the brave face, the positive things that happen. This story involves tears and difficulty. And also we often see only anger and bitterness. This story involves joy and celebration. One afternoon last year, I was working as usual (I work from home) when two sheriff deputies arrived on my doorstep. They gave me paperwork showing that my now ex-wife had filed for divorce, had asked them to serve me the paperwork, and that she had been given temporary possession of the house. I had 2 hours to gather up clothes and a little computer equipment (there was a list) and leave the house. Thus began the most difficult time in my life. I went from reading a bedtime story and singing a bedtime song to my young boys every night to seeing them only a little, from living in the house my grandparents and dad had lived in to having no particular plan for where I d sleep that night, from thinking I had a good idea of what the future held to not knowing when, if ever, I d ever be back home. I worried about how the boys would fare (they have done well so far). And it was incomprehensible; I couldn t find answers to why? . In the time since, the divorce became final, I did return home, the boys spend more time with me, and a new normal emerged. At the time, it seemed like a sudden, deep winter blizzard. I couldn t see very far down the road, spring seemed far off, and I couldn t see very well either forward or backward. But I was determined to find positives in the situation. It started almost immediately; I had never been a person to talk about pain, but just a few hours after the divorce was filed, I knew I needed to talk to someone about it, and did. A week later, I shared about it in church. Amazing friends, locally and all over the world, provided support and encouragement. I had less total time with the boys than before the divorce, but more time with just the three of us, and we used it to play together at home, spend days in town, and even take a train trip to Santa Fe, where none of us had ever been before. I realized how much I could forgive, and that my ex-wife probably did the best she could with the persistent legacy of difficult life events that happened to her long before she met me. I understood this, and was never angry, just sad, for everyone. I have always known nobody is perfect, myself included, but can be hard on myself when I m less than perfect. I forgave myself, too, realizing that I did my best to help in the most unfamiliar of waters, and although I sometimes didn t get it right, my conscience is clear because my heart was in the right place and I tried, very hard. Most incredibly, I became a person with a deep sense of inner peace. I always tried to work hard to set life on a good path; I got good grades in school, am a good employee, and have a strong set of values. But where courts are involved, there s a strong sense of powerlessness. At times, there was nothing I could do to make life better for my boys or for me. I finally had to let go of taking on responsibility for all that on my own shoulders. I simply knew that things would be OK, and in fact were OK, and that there is nothing in life that really deserves worries. That doesn t mean worries are never present, but that mostly they are subdued, like a radio quietly playing in another room. When they aren t, I can sit down at the piano, play my penny whistle, sing, walk to my creek, talk to friends, or any of so many things that let them melt away. I stopped searching for happiness and peace, and let those things find me. In religious terms, my faith became not just an intellectual one, but also a spiritual one. An atheist friend asked me, Just what does religion mean to you anyway? My answer: The certainty that spring always comes, for everyone that understands this.

People speak of misfortunes and sufferings, remarked Pierre, but if at this moment I were asked: Would you rather be what you were before you were taken prisoner, or go through all this again? then for heaven s sake let me again have captivity and horseflesh! We imagine that when we are thrown out of our usual ruts all is lost, but it is only then that what is new and good begins. While there is life there is happiness. There is much, much before us. Tolstoy, War and Peace

By Thanksgiving, I had much to be thankful for. Some of it under my nose waiting to be rediscovered after years of distraction, such as the glorious Kansas sunrises. Some things were new, such as roasting a turkey all by myself (or, rather, with Jacob and Oliver) for the first time, and having it come out absolutely perfectly. And some were just the things of everyday life: that I lived in my own house again, that I could walk out to the creek at the edge of my property whenever I wanted, that I could play piano, that the sounds of laughter and little running feet again could often be heard on my wood floors. That s not to say everything was easy; the courts sometimes made decisions sometimes I didn t think were in the boys best interests, legal things dragged on and on, but in the end, peace endured. Happiness endured. I found myself thinking at Thanksgiving that it was the best Thanksgiving ever. Not much later, I considered myself happier than I ve ever been. I was focusing on the daily gifts of life, marveling at the sunrises, looking forward to life s next adventure, confident that it would be far better than the last. And then, to my complete surprise, I found myself in a relationship again. On top of all the wonderful things happening in my life, I met Laura. I never could have imagined a friend so wonderful, a relationship so loving and joyful, something to treasure so deeply. I can look back at events and shake my head in amazement and wonder, that I found myself happier than I d ever been, and then this wonderful relationship on top of that. I have been blessed to have the life I do, and feel almost embarrassingly lucky. I share this story because friends that had been through divorce years ago shared their stories. They gave me hope. And if I didn t share this story in this public way, I would be squandering an opportunity to find more positives from what happened. I hope that this helps, somehow, someone that is in pain know that there is beauty in the valley, and spring always comes, every single year. And I share it because happiness like this can t be repressed for long. Tolstoy was right. While there is life, there is happiness. I ll end with a story from last Thanksgiving. It was 6:10AM that weekend. I was still asleep, and heard this: Jacob, yelling from his room: Dad? Dad! Me, groggy: Yes, Jacob? Jacob: Can I go down and look at the Christmas tree? Me: Sure, and you can turn it on too. Jacob: Great! At that point, I could have gone back to sleep. I was really short on sleep that morning, and Jacob would have been fine. But I gave him a couple of minutes, then I went downstairs too. He was curled up on the piano bench, looking at the tree. I quietly turned up the downstairs thermostat, got a chair, put it next to the piano bench, and sat down by the tree too. Jacob crawled over onto my lap and snuggled up for awhile. Neither of us said anything. Then: Dad? Yes? This is the best Thanksgiving ever. And he gave me a big hug. And he was right. Yes, he was RIGHT! That will be the beginning. Spring comes!