Let s assume you are a sysadmin and have to debug a daemon giving bad performance on one machine, but not on the other. Of course, you did not setup either machine, have only basic knowledge of the said daemon and would really love to watch that awesome piece of cinematographic art with a bunch of friends and a couple of beers. So it s like every day, right? The problem with understanding running setups is that you often have to read configuration files. And when reading one is not enough, you have to compare two or more of them. Suddenly, a wild problem occurs: order and indentation do not matter (unless they do), comments are often just beautiful noise and why the hell did that guy smoke/drink/eat while explicitly setting ALL THE OPTIONS to their defaults before actually setting them as he wanted. If you are using diff(1), you probably love to read a lot of differences, which are none in reality. Want an example? [foo]
bar = bar
foo = foo and # settings for foo
[foo]
# foo is best
foo = foo
# bar is ok here, FIXME?
bar = bar and [foo]
foo = x
bar = x
[foo]
foo = foo
bar = bar are actually the same, at least for some parsers. XTaran suggested using something like wdiff or dwdiff, which often helps, but not in the above case. Others suggested vimdiff, which is nice, but not really helpful here either. As there is a problem, and I love to solve these, I started a small new project: cfgdiff. It tries to parse two given files and give a diff of the content after normalizing it (merging duplicate keys, sorting keys, ignoring comments and blank lines, you name it). Currently it can parse various INI files, JSON, YAML and XML. That s probably not enough to be the single diff tool for configuration files, but it is quite a nice start. And you can extend it, of course ;)

0 comment(s) this blog is flattr enabled

In case you are using busybox-static like me to create custom initramfses, here a little warning: The current busybox-static in unstable lost its ability to find builtins with no /proc/self/exe, so if you use it make sure you either have all builtins you need up until mount /proc (including mount) and after you umount all file systems as explicit symlinks or simply create a /proc/self/exe -> /bin/busybox symlink...

I will never understand the point of using autotools, cmake or whatever configure system, when later the code uses an hardcoded list of architectures to determine the size of a pointer Unfortunately for porters this pattern is quite common. Update: As people keep asking, the way to check for the size of a given type is explained in the autoconf manual. To check for the size of the pointer, the following entry has to be added to configure.ac: AC_CHECK_SIZEOF(void *) On a 64-bit system, this will lead to the following entry in config.h:

/* The size of void *', as computed by sizeof. */
#define SIZEOF_VOID_P 8


Today as it was raining and I couldn t do much gardening, psmisc version 22.21 was released. The files are located up on sourceforge at https://sourceforge.net/projects/psmisc/files/latest/download or at your favorite distro repository soon. Once again, thanks to all patch submitters, bug reports and translators for all their help in getting this out. Apologies to the translation teams for having two alpha versions. So what does psmisc 22.21 bring you? Amongst a lot of minor bug fixes it has:

• If you started a process and then spawned some threads and then decided to change the names of the threads, pstree would show the old name, it now shows the correct new name
• pstree has a new flag (-N) for namespace support, thanks Aristeu for the patches
• Previously fuser -M flag only worked if it was before -m, now it can be either order

The Debian psmisc package should be out in the next few hours.

An issue which I always find depressing but sadly unsurprising in discussions of process with software is the frequent disregard for human elements; indeed often the goal people have in creating process is to try to control and eliminate human elements. Little thought is given to what is going to motivate people to do what s asked and if they are going to follow it at all, or in the spirit it was intended. One example I ve seen several times is the idea that some engineers work on irrelevant things and that this can be fixed by requiring every commit to be tied to the project plan or a bug so off project work is obvious. This isn t really attacking the problem so much as putting a roadblock in place to try to avert it; the real problem is normally people not communicating about what they re doing and what s important but those problems are really hard to address. Sadly what tends to happen is that people work around the roadblock and cause some collateral damage; for example devaluing the bug tracker by referencing irrelevant bugs or creating meaningless bugs solely to allow commits, while continuing to behave like they did originally. This is often worse than the original situation. This sort of issue is one of the things I appreciate most about working on Linux there s quite a bit of process but because of the way it has been evolved the incentives are usually right to make sure they are followed in the spirit in which they were intended. For example the patch submission process is essentially just best practices for making sure changes go to people who care in a form which makes it easy for them to work with it; there is a bunch of tooling around it which built on those practices (and in turn influenced the practices) but it all comes back to that basic thing of getting attention for changes and making them easy to work with. As a result people mostly do the right thing (or close enough) so everything runs smoothly. The trick is to remember to standardize and write things down when it s needed notice the good practice and spread its adoption. The key difference here seems to be if process is viewed as something to solve problems for people or if it is viewed as a way to solve problems with people. If the problem is with the people then the view starts off negative and it s perhaps unsurprising that there is little consideration of how they will react. Sometimes it s not so much that a problem is seen with people as that the people doing don t have any real interest in the process (write only timesheets often bear no relation to reality for example) but there is an assumption that they are going to just do what they re asked (but instead they for example don t fill in their time sheets or don t provide accurate information). The end result is the same, their needs don t get considered and their actions end up being counterproductive. Starting to avoid these problems is fairly straightforward take a step back and consider why the people expected to carry out the process are going to want to do so. How does it help them? How will they see it helping others? Generally what s in it for them? If these are difficult questions to answer then there may be a problem it is likely people will ignore the process or do it badly without a lot of active enforcement, but perhaps that s OK for the situation. Beware of using metrics to provide incentives, metrics are gameable and that game can cause problems without care. But do think about people; making software is all about people.

I've started backporting some Perl modules to wheezy-backports - for starters, libbread-board-perl, which is now waiting in BACKPORTS-NEW. At work I've recently been trying to automate the deployment of our platform, and was originally trying to use Carton to manage the CPAN dependencies for us. It seems like it ought to be possible to make this work using CPAN-only tools. However, in practice, I've seen two strong negatives with this approach:

• it's a lot of work for developers to manage the entire dependency chain, and
• it takes forever to get the environment running.

Consider, when you spin up a fresh VM, you need to build Perl from source, and then compile every CPAN module you depend on. This includes all the modules needed to run all the test suites. That's not going to be fast. All that, and you still need a solution that works with the distro's package management, because you still need to install all the build dependencies. So, I'm trying a new approach - if someone else benefits from the packages I backport, even better.

For those of you who didn't yet know it: My mother is a painter. A serious, professional, respected painter. But she sometimes goes to the funny side as well Of course, with all due professionalism! So, she gave us this great gift: She took one of our pictures from DebConf12 (from the "Conference Dinner" night), and painted it. Real size even! So, next time you come to our house, even if we are not around to greet you, we will be glad to welcome you to the Residence!

This is the 2nd point release of OpenStack Havana (this is the name of the current stable release of OpenStack). It was out on Thursday (US time), and I uploaded it on Friday (Chinese time). Unfortunately, I realized that the latest python-keystoneclient didn t support the token and endpoint command line options, effectively breaking Keystone itself, and all automated endpoint registration in all core packages. So after fixing Keystone and openstack-pkg-tools, I had to re-upload a 2nd Debian release. However, after this glitch, the packages passed successfully our CI testing suite: my friends at eNovance and I run the tempest functional test suite each time there s this kind of major update, so that we can validate the packages are working as expected. And all went well after the new keystone command line options were fixed in the postinst scripts. As a side note, I fail to see consistency on deprecating these interfaces. While we still have the old glance index thing in the Glance client (this is really old), keystoneclient just broke backward compatibility, when keystone is quite recent. So, it looks to me that it wasn t a great idea to just remove token and endpoint this way (though I can understand the need of consistency). Hopefully, this will all be fixed anyway, when we move to the unified python-openstackclient command lines instead of per-project clients. I realized today that the pkg-openstack team now maintains more than 100 packages. This of course includes general purpose Python modules, which probably will move to the Python module team if it one day supports Git (life is too short: I do not wish to learn such a deprecated technology as SVN). Though this is still a lot of packages, and some more are coming: as per the OpenStack TODO list wiki page, Ironic (a replacement for nova-baremetal), TripleO (OpenStack on OpenStack), Tuskar (contols where things are deployed with TripleO) and Designate (DNS as a Service) are already packaged, and I am waiting for the Icehouse (this is the name of the next stable release) beta3 to be out to upload them to Experimental. This is scheduled for the 6th of March. I believe I am currently nearly up-to-date with the current global-requirements.txt (eg: python dependencies of OpenStack), pending a few Python module upgrades, so hopefully, packaging OpenStack Icehouse beta3 will smooth.

AltOS 1.3.2 Bug fixes and improved APRS support Bdale and I are pleased to announce the release of AltOS version 1.3.2. AltOS is the core of the software for all of the Altus Metrum products. It consists of firmware for our cc1111, STM32L151, LPC11U14 and ATtiny85 based electronics and Java-based ground station software. This is a minor release of AltOS, including bug fixes for TeleMega, TeleMetrum v2.0 and AltosUI . AltOS Firmware GPS Satellite reporting and APRS improved Firmware version 1.3.1 has a bug on TeleMega when it has data from more than 12 GPS satellites. This causes buffer overruns within the firmware. 1.3.2 limits the number of reported satellites to 12. APRS now continues to send the last known good GPS position, and reports GPS lock status and number of sats in view in the APRS comment field, along with the battery and igniter voltages. AltosUI TeleMega GPS Satellite, GPS max height and Fire Igniters AltosUI was crashing when TeleMega reported that it had data from more than 12 satellites. While the TeleMega firmware has been fixed to never do that, AltosUI also has a fix in case you fly a TeleMega board without updated firmware. GPS max height is now displayed in the flight statistics. As the u-Blox GPS chips now provide accurate altitude information, we ve added the maximum height as computed by GPS here. Fire Igniters now uses the letters A through D to label the extra TeleMega pyro channels instead of the numbers 0-3.

Init system I am happy that Debian's Technical Committee was able to reach a very good decision with the choice of systemd as the default init system for the Linux ports. In particular, I was very pleasantly surprised by the lucidity of Russ Allbery's analyses and, above all, his patience even with people that, in my understanding, were just being trolls or trying to cause confusion and disrupt a process that was already chaotic. Russ, thank you for your exemplary role. (Also, you type fast or what? You fill whole e-mails seemingly faster than I can read!)

Unexpected union of forces It seems to me that Mark Shutterworth's decision of (perhaps?) embracing Debian's decision in the init system is a good thing. I think that one way where Ubuntu did a great thing with respect to Debian was their initial attitude towards cherry-picking the best of breed of Free Software from Debian's archive and polishing that to create a really "for human beings" operating system. I hope that this means that, at least in part, money (Mark's?) can be channeled in the best of the interests so that we all gain from it. (And, to be honest, I see no problem in making money with Free Software and I welcome a well done integration job).

More polarizing things I feel that, now, I may even have some hope of sending another potentially polarizing bug to the Tech Ctte (that is ffmpeg vs. libav) and have faith that some rational decision would be made. I was not even aware of it, but the ffmpeg folks even featured a news item in their page of me having filed the RFP bug mentioned above.

DebConf 14 If everything goes as expected, I hope to attend this year's DebConf. In fact, I am anticipating this, since I would love to, e.g., talk with Joey Hess and discuss a better way of having GNU parallel and the parallel program in moreutils have amicable co-installability among many, many other things with respect to my packages. And, quite probably, work on packaging ffmpeg with other interested people (which seem to be numerous).
Edit: Fix markdown link pointing twice to the ffmpeg site. Thanks Marius Gedminas for pointing this out.

I love Free Software, and the people behind it. Even though I have been a free software user and developer for more than a decade now, there's still more to be amazed at, still more to contribute, and still more to learn. I love the Free Software community, because it never ceases to amaze me. It's not just about the technical side, but the social, human side of it, too!During the past year, I came to another milestone in my career: as of july first, I am a Free Software developer, and enjoying every moment of it. This is a feat that could not have been done without there being demand for what I do, without the free software community as a whole. For that, I owe you all a big thank you, you make it possible for me to make my dream come true. I can only hope that through my work, and through my example, I can contribute back enough.To this day, I take delight in contributing to free software, be it a Lisp on Python (or projects built in it), any of my own projects, syslog-ng, Google Summer of Code mentoring, Debian work - any and all of that are great fun. Even processing NEW is (I really should do that more often)! It's an endless source of not only entertainment and knowledge, but inspiration too.Whenever I read something as amazing as Russ Allbery's mail on the Debian technical committee impasse, I realize just how amazing people we have in our community, and how much we all can learn from them, and how much we owe them. This is yet another reason why I feel that contributing to Free Software is a passion many can share, because it teaches us good values too, atop of being technically excellent.I love Free Software, because without it, I would be lost. Thank you.

Barely one and half month after the release of the Wheezy version of the Debian Administrator s Handbook, I m proud to announce that two translations have been completed: French and Spanish. And, as promised in the last liberation campaign, the French translation has been published as a paperback by Eyrolles (see the cover above). Check out the article on debian-handbook.info for more information.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

I m not sure if it is just a bad motherboard, cable, USB device or just bad luck, but when I reboot the computer the UPS doesn t appear. It s not even seen in lsusb and of course that makes nut-server fail which makes nut-client complain that it cannot work out the UPS status.The solution was to unplug and plug the cable in then reset nut-server and all was good. But playing with cables seems so last century, surely there was a better way? Some Googling showed up a way to Reset USB without a reboot but the directories the entry mentioned didn t exist on my computer; though there was something very close to it. Perhaps that would work? A few tweaks later and I had a working UPS monitoring cable! The changed script is below: I m not sure if it is just a bad motherboard, cable, USB device or just bad luck, but when I reboot the computer the UPS doesn t appear. It s not even seen in lsusb and of course that makes nut-server fail which makes nut-client complain that it cannot work out the UPS status.The solution was to unplug and plug the cable in then reset nut-server and all was good. But playing with cables seems so last century, surely there was a better way? Some Googling showed up a way to Reset USB without a reboot but the directories the entry mentioned didn t exist on my computer; though there was something very close to it. Perhaps that would work? A few tweaks later and I had a working UPS monitoring cable! The changed script is below: Thanks to Eli for the original idea which got me onto the right path.

Andrzej's C++ blog has a nice series on type erasure. I found it interesting to read and have learnt some things from it. For instance, I vaguely guessed, but did not fully realize, that std::function<> is of course a performance hit, since the way how std::function<> is implemented makes it impossible for the compiler to do anything useful with that function call. Inlining is totally out of the question, since we have just erased the type information. You might say, "of course!", but for me it was sort of a revelation. The series is in four parts: part I, part II, part III, part IV.

Today was another busy day. Zoe had a check up at her dentist first thing this morning. This dentist was such a good find, and although it's a trek (it's over an hour round trip for a 15 minute appointment) I think it's well worth it to instill a positive view of dentists into Zoe, and it's all covered by insurance, so really the only expense is time and petrol. After we got home, Zoe was a bit sick of being in the car, and wanted to watch TV. I'm trying to restrict TV to just the late afternoon, while I'm preparing dinner, so I thought this was a good opportunity to break out the Goldie Blox set that I haven't introduced her to yet. That went pretty well, but Mum and Dad (aka Grandma and Grandpa) arrived early for lunch, so we didn't finish working our way through the book. I think it was more of a hit than Robot Turtles is at the moment. We all jumped in the car, and drove down to Bulimba, and caught the cross river ferry over to the CityGlider bus and had Yum Cha for lunch. That went pretty well, and we returned the same way. Zoe had a little play in the park at Bulimba with her grandparents while I popped over to the florist, and then we came home and Zoe gave them a demonstration of the kinetic sand and then Grandma and Grandpa went home and I managed to convince Zoe to take a late nap. We didn't get much of an opportunity for something Science Friday-related today, with all of the other activities. Zoe didn't wake up from her nap until close to 5pm, just after my girlfriend arrived for dinner. Zoe watched a bit of TV and then the three of us walked down to collect pizza for dinner, and that was pretty much how the evening ended. I let Zoe stay up a little bit later, because she'd napped so late.

After so many attempts to have my shiny Movistar Peru (Internet M vil) 3G ZTE MF193 modem to work out-of-the-box in Debian jessie (unstable) with NetworkManager, the word frustration was hitting on my head. Even trying to do the right thing, led me to craziness. I gave up on fanzines and decided to take the old-school route. Release wvdial and friends! Trying different combinations for wvdial.conf was no heaven for sure, but I ve found this wonderful guide from Christian Ams ss from Vienna, Austria! that really made a difference. Of course he s talking about the MF180 model but you get the idea. So I m sharing what was different for the MF193. Basically, I ve done the eject and disable CD-ROM thing already, but still no progress. I ve also tried using wvdial to send AT commands to the evasive /dev/ttyUSBX device. Starting from scratch confirmed that I ve done such things properly indeed. I was amused by the fact that I could use screen to talk to the modem! (yo, all the time wasted trying to have minicom and friends play nice) So, let s get to the point. After following this procedure, you should be able to use NetworkManager to connect to the Interwebs using the 3G data service from Movistar Peru.

1. Step 1 follow the guide
2. Step 2 Here I had to use /dev/ttyUSB4
3. Step 3 follow the guide
4. Unplug your USB modem
5. Plug your USB modem. This time you should see only /dev/ttyUSB 0,1,2 and /dev/gsmmodem should be missing (not sure if this is a bug). Now /dev/ttyUSB2 is your guy.
6. Step 4 use /dev/ttyUSB2
7. Run wvdial from CLI it should connect successfully.
8. Stop wvdial
9. Select the Network icon on GNOME3, click on the Mobile Broadband configuration you have, if not create one.
10. Voil . Happy surfing!

I m pasting my wvdial.conf, just in case.

[Dialer Defaults]
Modem = /dev/ttyUSB2
Username = movistar@datos
Password = movistar
APN = movistar.pe
Phone = *99#
Stupid Mode = 1
Init2 = AT+CGDCONT=4,"IP","movistar.pe"

I ve been hacking (on and off) on a small bit of code (written in Hy) called Acid. So, all of this is *really* fluid, and I m going to change it s API and code (it s currently just one massive hack), but I think the idea has kinda jelled enough for me to chat a bit about. Acid is a DSL for writing event-driven Hy. Currently it s just using global events (that ll change soon), and it s a monumental hack (not sure how much I can do there), but it works pretty well. It s built on top of tulip / asyncio, which is an amazing new Python 3.4 standard library package for working with async code. Although asyncio is designed around network-based usecases, I ve been trying to shoehorn in an event system on top of it. It s going well so far. A basic Acid script looks something like:

(trip
  (on :startup
    (every 1 second (emit :clock-pulse nil)))
  (on :clock-pulse (print "."))
  (emit :startup nil))

which will print a dot ( . ) every second forever. Something a bit more advanced (fetch the MBTA Red line T information once a minute forever)

(defn get-endpoint-url [line]
  (.format "http://developer.mbta.com/lib/rthr/ 0 .json"
    (get  :red-line  "red"
          :blue-line "blue"  line)))
(trip
  ;; OK. Let's do some work with MBTA feeds.
  (on :update-feed
      ;; let's just update feeds on a cron.
      (print (.json (.get requests (get-endpoint-url event))))
      (emit :feed-updated event))
  (every 1 minute
         (emit :update-feed :red-line)))

Clearly, both of these examples are just as clear as their straight functional counterparts, but I m keen to see what I can do with temporal recursion for long-running daemons. I think my first job will be porting snitch's codebase to Acid. Let me know if you have any ideas!

HP recently announced that they are no longer offering free firmware updates for their ProLiant line of servers. I have a HP ProLiant G7 N36L MicroServer at home (running Debian stable, naturally) which I got back in early 2011. While it has, and continues, to serve me well, I'll have to reconsider my options when I go looking for a replacement. This is a pity because I really like the MicroServer form factor. As recently as November 2013, HP released a BIOS update to fix random lockups when installing Windows 8.1/2012 R2 on a MicroServer (I'd never ever be running these operating systems, but the point still stands). I'd be worried if I couldn't upgrade my system for fear of uncovering a bug in firmware that I couldn't get an update for. Non-free firmware is bad, but in certain circumstances, it can be tolerated. Non-free firmware behind a paywall is worse, and can never be tolerated.

An NSA employee reminded me that I've been on IRC for 20 years.