Weblate 1.7 has been released today. It comes with lot of improvements, especially in translation speed and many new features. Full list of changes for 1.7:

• Please check manual for upgrade instructions.
• Support for checking Python brace format string.
• Per subproject customization of quality checks.
• Detailed per translation stats.
• Changed way of linking suggestions, checks and comments to units.
• Users can now add text to commit message.
• Support for subscribing on new language requests.
• Support for adding new translations.
• Widgets and charts are now rendered using Pillow instead of Pango + Cairo.
• Add status badge widget.
• Dropped invalid text direction check.
• Changes in dictionary are now logged in history.
• Performance improvements for translating view.

You can find more information about Weblate on it's website, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Ready to run appliances will be soon available in SUSE Studio Gallery. Weblate is also being used https://l10n.cihar.com/ as official translating service for phpMyAdmin, Gammu, Weblate itself and others. If you are free software project which would like to use Weblate, I'm happy to help you with set up or even host Weblate for you. Further development of Weblate would not be possible without people providing donations, thanks to everybody who have helped so far!

Filed under: English SUSE Weblate 1 comments Flattr this!

not much activity around RC bugs on my side in the last weeks; & the ones I worked on were all in the Debian Perl Group:

• #722332 libcommon-sense-perl: "libcommon-sense-perl: common::sense runs into trouble with Perl 5.18, eg. no say()."
  ensure same version of perl at build and runtime (pkg-perl)
• #723917 src:libcatalyst-dispatchtype-regex-perl: "FTBFS: Module::Build version 0.4004 required--this is only version 0.4003 at Build line 41."
  fix build dependencies (pkg-perl)
• #723927 src:libdancer-plugin-auth-extensible-perl: "FTBFS: POD error"
  add patch to fix POD (pkg-perl)
• #723933 src:libemail-mime-perl: "FTBFS: test failures in t/ct_attrs.t"
  upload new upstream release (pkg-perl)
• #724109 src:libmoosex-getopt-perl: "libmoosex-getopt-perl: FTBFS: Tests failures"
  upload new upstream release (pkg-perl)
• #724111 src:libtemplate-declare-perl: "libtemplate-declare-perl: FTBFS: Tests failed"
  upload new upstream release (pkg-perl)
• #724196 src:libmldbm-sync-perl: "libmldbm-sync-perl: FTBFS: make[1]: *** No rule to make target /usr/local/lib/perl5/5.6.1/i686-linux/Config.pm', needed by Makefile'. Stop."
  re-create Makefile in clean target (pkg-perl)
• #724235 src:libio-compress-lzma-perl: "FTBFS: IO::Compress::Base version 2.061 required--this is only version 2.060"
  fix alternative dependencies (pkg-perl)
• #724303 src:libvideo-fourcc-info-perl: "libvideo-fourcc-info-perl: FTBFS: fails t/99kwalitee.t: no license information"
  don't run author tests during build (pkg-perl)
• #725393 libhttp-server-simple-psgi-perl: "libplack-perl and libhttp-server-simple-psgi-perl: error when trying to install together"
  add versioned Breaks/Replaces (pkg-perl)
• #725585 src:libhttp-request-params-perl: "libhttp-request-params-perl: FTBFS: Tests failed"
  add patch from CPAN RT (pkg-perl)

on an unrelated note: following the discussion on debian-project, I finally revoked my old opengpg key 0x00F3CFE4 which served me well from 1998 until 2009. since then, I've only been using 0x8649AA06 anyway ...

Despite the fact that I've had very little time to either read or write reviews, I haven't stopped buying a few books here and there. And, of course, Powell's Indiespensable keeps coming. Matt Bell In the House Upon the Dirt Between the Lake and the Woods (mainstream)
Steven Brust Orca (sff)
Steven Brust Dragon (sff)
John W. Campbell Collected Editorials from Analog (non-fiction)
Jane Fletcher The Walls of Westernfort (sff)
Mira Grant How Green This Land, How Blu This Sea (sff)
Tate Hallaway Tall, Dark, & Dead (sff)
Diana Wynne Jones Howl's Moving Castle (sff)
Rebecca Lee Bobcat and Other Stories (mainstream)
Victor Lombardi Why We Fail (non-fiction)
George R.R. Martin & Gardner Dozois (ed.) Songs of Love and Death (sff anthology)
Seanan McGuire In Sea-Salt Tears (sff)
Seanan McGuire Late Eclipses (sff)
Justina Robson Selling Out (sff)
John Scalzi Muse of Fire (sff)
Mark Slouka Brewster (mainstream)
Tor.com (ed.) The Stories (sff anthology)
Carrie Vaughn Kitty and the Midnight Hour (sff) This is a wide variety of random stuff, including some stuff that was free (the Tor.com anthology, which is every piece of short fiction they've published and is about 4000 pages, the Campbell editorials from the Internet Archive) and a lot of stuff that was on sale. As well as another two shipments of Indiespensible. The Brust books I've already read and are already in the family, but I hooked a friend of mine on the Vlad Taltos series, and those were the only two that I both didn't have and that weren't available on Kindle. So I picked them up while he was visiting so that he could read the whole series.

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (86.18 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Package Tracking System^U Distro Tracker Marko Lalic implemented quite a few interesting features in the last weeks of the Google Summer of Code (support of teams most notably). Unfortunately he didn t deploy (yet) the latest changes on pts.debian.net. Given the good work he made over the summer, I marked him as successful in his GSOC. Hopefully he will stick around and continue to contribute, he promised to try to handle some mass renaming that we agreed upon. Effectively, after much bike-shedding, I decided that the software would be called Distro Tracker . Once those last-minute cleanups are done, I plan to request tracker.debian.org to Debian System Administrators. This means that it will be deployed in parallel to the current PTS at least until we re at feature parity in the new codebase. The new codebase should be much more easy to get started with, so I should do some promotion and invite people to contribute to it possibly by writing some short how to get started documentation. I started by creating a dedicated wiki page: http://wiki.debian.org/qa.debian.org/distro-tracker
Misc packaging I got two REJECTs from ftpmasters this month (one for galette, one for dolibarr). I took care of fixing the various issues in galette and the package has been promptly accepted afterwards. For dolibarr, I mentored the upstream maintainer about the various problems and got him to fix it. It took a bit more time and the package is thus still in NEW. I packaged wordpress 3.6, and then wordpress 3.6.1 (security update). python-django also had multiple security updates this month, I took care of one or two uploads but Luke Faraone dealt with most of them (including backports to Squeeze!). I packaged Publican 3.6.1 and uploaded dh-linktree 0.4 to fix a FTBFS issue introduced with Perl 5.18. Of anecdotal importance, but I also filed bug #721849 after seeing how much energy was spent to ensure debian/rules didn t contain an improper copyright statement. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

For a while I have been eyeing with the TI eZ430 Chronos watch, a slightly bulky sports watch that not only comes with a built-in thermometer, barometer and accelerometer, but also a freely programmable micro controller (from the TI MSP430 family hence the name). In addition it has a small radio chip included and can communicate wirelessly with the computer, or possibly other devices. All in all, a very geeky toy. A week ago I managed to buy one quite cheaply via eBay (30 including shipping). It took me a while to get an overview of the various resources about the device. Here is what I have learned.

• Hardware: I have the new edition (the included debug board and wireless access point are white). This means that I cannot update the software of the watch via the wireless connection, and it means that most software written for it does not support the new altimeter sensor (yet).
• Development: Debian is a very suitable development platform; the packages mspdebug and gcc-msp430 provide all I need to compile (make config && make) and upload (mspdebug rf2500 "prog ../foo.txt") new software.
• Code: Besides the original software that comes with the device (which supports the new hardware), there are two main lines of community-driven developments: OpenChronos and openchronos-ng. The former is (as the name suggests) older and maintained mostly by forks in various places. The latter is newer, its UI deviates further from the original software and seems to be slightly more active.
• Communication: Both of these have their own mailinglists (openchronos, where I did not get a reply yet, and ti-chronos-development, where my messages are still waiting for moderation). I found only one IRC channel, #openchronos on freenode, with similarly little activity.

After I found that out, I tried to hack a bit on the various software. The first issue I had (and still have) is that I cannot use the buttons of the device while it is attached to the debug board. They do work if forget to take out the battery, but supposedly that is bad for the battery. So my compile-upload-test-cycle becomes a compile-upload-detach-insert-battery-test-remove-battery-attach-cycle. I could not find out how to emulate button presses using mspdebug. Any suggestions are welcome. The other problem was the new altimeter hardware: Neither OpenChronos nor openchronos-ng support that yet. I originally planned to port the driver from the official software to a random OpenChronos fork when I read that Richard Rondu did that to openchronos-ng yet. I found the code in a month-old merge-request. I tried it, it worked, and I started to clean up the code a bit and add a variometer mode to it, e.g. displaying the current rate of climb my ultimate goal is to use this as a surprisingly cheap variometer for paragliding. After a day of hacking and quickly lifting the watch from the room floor to the room ceiling and back I believe that it works okish, although quite a few parameter can benefit from more tweaking: sample rate (currently 20Hz), length of history (currently 2s) and the linear regression method (currently simply the difference between the average of the samples of the current second with the samples of the last second). As always, patches welcome, the code is in my fork on sf.net.
Next to do would be to enable some buzzing noise depending on the rate of change; during flight I don t want to decipher a low-contract LCD screen. But it is questionable whether the buzzer is loud enough we ll see. I was also happy to hear that there are other zE430-hackers in Karlsruhe, one of them for example tries to make the alarm clock sleep-cycle-aware.

Here in the US, the Affordable Care Act is finally going into effect, with accompanying drama. I managed to get signed up today at healthcare.gov. After not having health insurance since 2000, I will finally be covered starting January 1 2014. Since my income is mosty publically known anyway, I thought it might be helpful to blog about some details. I was uninsured for 14 years due to a combination of three factors:

1. Initially, youthful stupidity and/or a perfectly resonable cost/benefit analysis. (Take your pick.)
2. Due to the US health insurance system being obviously broken, and my preference to avoid things that are broken. Especially when the breakage involved insurers refusing to cover me at any sane level due to a minor and easily controlled pre-existing condition.
3. Since I'm not much motivated by income levels, and am very motivated to have time to work on things that are important to me, my income has been on average pretty low, and perhaps more importantly, I have intentionally avoided being a full-time employee of anyone at any point in the past 14 years (have rejected job offers), and so was not eligible for any employee plans which were the only reasonable way to be covered in the US. (See point #2.)

So, if you're stuck waiting in line on healthcare.gov (is this an entirely new online experience brought to us by the US government?), or have seen any of the dozen or so failure modes that I saw just trying to register for a login to the site, yeah, it's massively overloaded right now, and it's also quite broken on a number of technical levels. But you can eventually get though it. Based on some of the bugs I saw, it may help to have an large number of email addresses and use a different one for each application attempt. It also wouldn't hurt to write some programs to automate the attempts, because otherwise you may have to fill out the same form dozens of times. And no, you can't use "you+foo@bar.com" for your email; despite funding the development of RFC-822 in the 80's, the US government is clueless about what consititutes a valid email address. But my "favorite" misfeature of the site is that it refuses to let you enter any accented characters, or characters not in the latin alphabet when signing up. Even if they're, you know, part of your name. (Welcome back to Ellis Island..) I want to check the git repository to see if I can find the backstory for these and other interesting technical decisions, but they have forgotten to push anything to it for over 3 months. The good news is that once you get past the initial signup process, and assuming you get the confirmation mail before the really short expiration period of apparently less than 1 hour (another interesting technical choice, given things like greylisting), the actual exchange is not badly overloaded, and nor is it very buggy (comparatively). I was able to complete an application in about an hour. The irony is that after all that, I was only able to choose from one health insurer covering my area on the so-called "exchange". (Blue Cross/Blue Shield) I also signed up for dental insurance (it was a welcome surprise that the site offers this at all) and had a choice of two insurers for that. The application process was made more uncertian for me since I have no idea what I'll end up doing for money once my current crowdsourced year of work is done. The site wants you to know how much income you'll have in 2014, and my guess is anywhere between $6000 (from a rental property) and about what I made this year (approx $25000 before taxes). Or up, if I say, answered the Google pings. The best choice seemed to be to answer what I made this year, which is also close to what I made last year. So, I'll be paying around $200/month for a combination of not very good health insurance, and not very good dental insurance. There is around $750/year of financial aid to people at my guesstimated 2014 income level, which would drop that to $140/month, but I will let them refund me whatever that turns out to be in a lump sum later instead. For comparison, I am lucky to spend rather less renting a three bedroom house situated in 25 acres of woods.. It's strange to think that all of this is an improvement to the system here in the US, especially given all the better options that could have been passed instead, but it seems that it probably is. Especially when I consider the many people around me who are less fortunate than myself. If you'd like a broader perspective on this, see Tobias Buckell's "American healthcare was already socialized by Reagan, we re just fighting about how to pay for it".

XMir's been delayed from Ubuntu 13.10. The stated reason is that multi-monitor support isn't sufficiently reliable. That's true, but it's far from the only problem that XMir still has:

• It's still broken on some single-monitor systems

Some machines have display corruption. There are writes to freed memory. To be fair, some of the behaviour that's been seen has been down to underlying bugs in the Xorg drivers that were never triggered under normal use but are hit by XMir. Others are down to implicit assumptions made in the drivers that XMir happens to violate. The problem is that there doesn't appear to have been enough room in the schedule to deal with these interactions, presumably because nobody accounted for the inevitable "This thing we thought would be easy turns out to be difficult" part of the project.


• The input driver bug still isn't really fixed

I've mentioned this bug before. It's now marked "Fix released" which is kind of true but not really. Mir now tells XMir that the VT is changing before the VT is changed, but it doesn't wait for that to complete before switching the VTs. Until XMir is scheduled and runs, it's still receiving input events. In most circumstances that window is small, so there's no real risk of triggering it accidentally.

There's one corner case where it might still cause problems. Simply running isn't enough - XMir has to make progress through its event loop. That'll only happen if the X server is processing its wakeup events, and it's possible to effectively stall that by submitting a sufficiently awkward drawing request to the server. The X server will appear to freeze, and if the user then attempts to work out what's going on by switching to a VT and logging in, those input events will still be going to XMir. It's left as an exercise to the reader to construct ways to take advantage of this.

This can't happen in Xorg because the VT switch is blocked until the input devices have been released. Mir could have done the same, but doesn't because of a conscious design decision - in the Ubuntu Phone world, clients stop doing things when they're told to. Ubuntu Desktop is expected to behave the same way.

This is an unfortunate situation to be in. Ubuntu Desktop was told that they were switching to XMir, but Mir development seems to be driven primarily by the needs of Ubuntu Phone. XMir has to do things that no other Mir client will ever cope with, and unless Mir development takes that into account it's inevitably going to suffer breakage like this. Canonical management needs to resolve this if there's any hope of ever shipping XMir as the default desktop environment.


• It's still missing features

XMir doesn't support colour profiles. XRandR properties aren't exposed, so there's no way to control TV output encoding or overscan. There's still no hardware cursor support. Switching to XMir now would reduce functionality without providing any user-visible gain.

It's clear that XMir has turned into a larger project than Canonical had originally anticipated, but that's hardly surprising. There's only one developer with previous X experience working on it full-time, and the announced schedule provided no opportunity to deal with unexpected problems. As if that weren't enough, there's obvious conflict between Ubuntu Desktop and Ubuntu Phone when it comes to developer time and required functionality. The one hardware vendor who's willing to take a public stand has demonstrated that they have no interest in supporting XMir, despite Canonical assuring people that they were already engaging in productive negotiations with GPU manufacturers.

Multiple monitor support may be the single most obvious feature where XMir is lacking, but it's not the only one. Technically and politically, this code is a long way from being ready for prime time. Without significant community contribution, Canonical need to focus on prioritising it appropriately rather than expecting a tiny number of developers to perform miracles. Or, alternatively, they could just drop XMir entirely and focus on Unity 8.

comments

Today the Nagios Plugins Development Team released the long awaited version 1.5 of the Nagios Plugins. It contains several bug-fixes as well as new features. The project moved to a new home at nagios-plugins.org, the SCM and issue tracker moved to github. If you want to contribute to the project, please have a look into the updated development documentation. Many thanks guys, you rock! The package was uploaded already into Debian unstable and it should hit the Backports repository soon. If you are impatiently, you could install the package from our repository.

Enca 1.15 has been just released. It does not bring any new features, just a little of cleanup and officially moves it's code to GitHub. If you don't know Enca, it is an Extremely Naive Charset Analyser. It detects character set and encoding of text files and can also convert them to other encodings using either a built-in converter or external libraries and tools like libiconv, librecode, or cstocs. Full list of changes for 1.15 release:

• Documentation improvement
• Development moved to GitHub
• Do not use deprecated autoconf macros

Still enca is in maintenance mode only and I have no intentions to write new features. However there is no limitation to other contributors :-). You can download from http://cihar.com/software/enca/.

Filed under: Enca English 2 comments Flattr this!

The BBC coverage of the UCI Women Road Race World Championship wasn t starting until 3pm, BBC Radio 5 Live had football and Sports Extra was playing an advert loop (really BBC?), Eurosport wasn t covering the race at all, RAI Sport 2 had coverage which was fine while I was watching the TV, but my Italian isn t good enough to follow the commentary and I wanted to get some other stuff done. So the obvious thing is to have the computer watch for changes to the great http://live.cyclingnews.com/ ticker and read them out, right? Well, it was to me. Here s the script I used:

#!/usr/bin/rc
# Send updates to live cycling news reports to the speech synth
# initialise
url=http://live.cyclingnews.com/
if (~ $1 *:*)  
  url=$1
 
last=""
while (true)  
    # download the update and strip html - if you don't have html2text try
    # sed -e '1,/<ol/d;s/<br[^>]*>/n/g;s/<[^>]*>//g;s/ / /g;s/[[:space:]]*$//'
    next= () curl -s $url   html2text -width 9999   sed -e 's/^ *[0-9]*. //'  
    # look for some added lines in the update section - timezone will need changing for non-European races
    say= () diff -u < echo $last  < echo $next  
        sed -ne '/^+/!d;s///;s/**//g;/ CES*T/,/^ *$/p'   head -9  
    # feed them to the speech synth in reverse order
    echo $say '
      Race Update'   tac   spd-say -e
    # update variables, pause and loop
    last=$next
    # set this to a prime number over 60, like 61, 67, 71, 73, 79, 83, 89
    sleep 67
 

Any questions, comments or improvements? I could have done more, like not saying Race Update even when there was no update, but I wanted to start listening as soon as possible!

Just saw this ad today at my Institute's door. Rough translation: Interim programmer sought Requisites:

• High school
• One year of experience
• Theorical and practical exams on the area
• Familiarity handling electronic equipment, such as computers, perforators/punchers, terminals and similar.

[Yes, this is a joke post. I don't know how anybody is using such an obsolete job description. A job description at least three decades out of date] [Well, yes, I understand it, because I knowhow that particular union works. But I cannot accept it is for real] [...And no, please don't apply for the job. You will be miserable in the unlikely event you are chosen]

Continuing last month progress report onOUnit2. The release is just a few days away, I am testing real life application and the core of the work is already in the VCS. The basic features:

• better configuration setup
  • environment variable
  • command line options
  • configuration files
• improved output of the tests:
  • allow vim quickfix to jump in the log file where the error has happened
  • output HTML report
  • output JUnit report
  • systematic logging (verbose always on), but output log in a file
• choose how to run your test:
  • run tests in parallel using processes (auto-detect number of CPU and run as many worker processes)
  • run tests concurrently using threads
  • use the old sequential runner
• choose which test to run with a chooser that can do smart selection of tests:
  • simple: just run test in sequence
  • failfirst: run the tests that failed in the last run first and skip the success if they are still failing
• some refactoring:
  • bracket: use a registration in the context but easier to use
  • remove all useless functions in the OUnit2 interface
• non-fatal section: allow to fail inside non-fatal section without quitting immediately the whole test
• allow to use OUnit1 test inside OUnit2 (smooth the transition)
• timer that makes tests fail if they take too long, only when using the processes runner (I was not able to do it cleanly using threads and sequential)
• allow to parametrize filenames so that you can use OUNIT_OUTPUT_FILE=ounit-$(suite_name),log and have $(suite_name) replace by the test suite name
• create locks to avoid accessing the same resources within a single process or the whole application (typically to avoid doing a chdir while another thread is doing a chdir elsewhere)
• create a in_testdata_dir function to locate test data, if any

Still remaining to do, but quite straightforward:

• sys admin (website, release process)
• update the whole documentation

Some things that I decided not to do for OUnit 2.0 release:

• introduce a 'cached' state to avoid rerunning a test if you can programmaticaly determine that the result will be the same.

The main development is now done, but before releasing I decided to test it first on a real scale application. The first big migration to OUnit2, will be the OASIS test suite. This is a pretty big test suite (100+ tests) that takes a fair amount of time to run. I hope that during the next week I will be able to port the whole test suite and come back with some timing results. You can follow my progress on porting OASIS to OUnit 2.0, in github. Help wanted If you have a long standing issue with OUnit, this is the time to submit a bug. OUnit BTS If you want to try the dev version of OUnit:

 $> darcs get http://forge.ocamlcore.org/anonscm/darcs/ounit/ounit

Patches always welcome. You can read the documentation of the devel version the website.

It s been a while since I posted anything to my personal site, but I figured I should update with the news that I m leaving Brighton (and the UK) after nearly nine years living by the seaside. I ll be sad to leave this city, which is the greatest place to live in the country, but I have to opportunity to go explore the world and I d be crazy to let it pass me by. So what am I doing? In ten short days, I plan to sell all my possessions bar those I need to live and work day to day and will be moving to Spain for three months. I m renting a flat in Madrid, where I ll continue to work for my software development business and set about improving both my Spanish and my fitness. If you want to follow my adventures, or read about the reasons for my change, then check out the Experimental Nomad website. The post A New Chapter appeared first on David Pashley.com.

One of the earlier examples of students using MIT computer resources to lay the groundwork for a later commercial endeavour, Zork was originally written in a LISP derivative called MDL. This was later tuned into the Zork Implementation Language, a domain specific language that was compiled to target the Z-machine rather than a specific piece of hardware. Combined with machine-specific Z-machine interpreters, this allowed rapid porting of games to a wide range of platforms - the only thing that needed to be rewritten was the interpreter, and that could be reused for any future games running on the same hardware. Infocom were eventually acquired and killed off, but fan interest in their games continued. New Z-machine interpreters were written in order to allow their games (including Zork) to be run on platforms that Infocom had never targetted. One of the best known is Frotz. This has the advantage of being (a) portable and (b) including a "dumb" UI that makes no assumptions about the availablity of any vaguely useful functionality. Like, say, a Curses library. So, Frotz seemed like the natural choice when this happened. But despite having a set of functionality that makes it look much more like an OS than a boot environment, UEFI doesn't actually expose a standard C library. The EFI Application Development Kit solves this particular design decision. Porting Frotz ended up involving far more fixing up of Frotz bugs that tripped up -Werror than anything else. One note, though - make sure you include DevShell in the list of required packages at build time, otherwise file i/o will mysteriously fail. The tying of file i/o to the shell protocol unfortunately means that Frotz can't be directly launched by the firmware. The Boot to Zork images therefore contain a UEFI shell in the standard boot location (\EFI\BOOT\BOOTX64.EFI) which is executed when the firmware attempts to boot the device. The shell then looks for a file called "startup.nsh" in the root directory of the boot device and executes it. Unfortunately this doesn't actually set the shell equivalent of the current device, and so just launching Frotz from startup.nsh fails when Frotz can't open the Zork data file. The solution for this is simple, if ugly - the script walks through the list of devices, looking for one that contains ZORK1.DAT in the root directory. It then changes to that device and launches Frotz. If Frotz exits, it then resets the system. This could be avoided by doing some more work and turning Frotz into a more UEFI-native application. Teaching Frotz to make native UEFI calls would avoid the requirement for the shell protocols, and the firmware provides a mechanism to obtain the path of the currently running executable which would avoid the need to explicitly locate the device. But I'm lazy and this was a "I'm spending the day on a plane" project initially inspired by a Sazerac-fuelled conversation during the UEFI plugfest, not a demonstration of UEFI best practices. UEFI Boot To Zork and the source code to the modified Frotz can be downloaded here.

comments

On a regular basis, I find myself the odd-man-out when it comes to talking about how to work with Python on Debian systems. I'm going to write this and post it so that I might be able to point people at my thoughts without having to write the same email in response to each thread that pops up. Turns out I don't fit in with the Debian hardliners (which is to say, the mindset that pip sucks and shouldn't exist), nor do I fit in with the Python hardliners (which is to say apt and dpkg are out of date, and neither have a place on a Development machine). I think our discourse on this topic has become petty and stupid in general. Let's all try to step back and drop a bit of the attitude. pip doesn't suck, and neither does apt. The truth is, both sides are wrong. As with any subject, the real answer here is much more nuanced than either side presents it. I'm going to try and present my opinion on this, in the way that both my Pythonista self and my Debianite self see the issue. Hopefully I can keep this short, to the point, and caked with logic. The case for dpkg (the Debianite in me) In defense of dpkg and apt, imagine having to install python-gnome2 on all your systems when you install. It'd be hell on earth. Imagine having a user try to do this. It's insane to assume that end-users will be using pip for this purpose. pip is fun and all, but it's also installing 100% untrusted code to your system (perhaps as root, if you're using pip with sudo for some reason), and hasn't been reviewed for software freeness, which is something Debian (and Debian users) take seriously. This isn't even to mention the hell that pip wreaks on dpkg controlled files / packages. Try to remember how much of your system running (yes, right now) is running because of Python or Python modules. Try to imagine how much of a pain in the ass it'd be if you couldn't boot into GNOME to use nm-applet to connect to wifi to pip install something. I'm sure even the most extreme pip'er understands the need for Operating System level package management. Debian also has a bigger problem scope - we're not maintaining a library in Debian for kicks, we're maintaining it so that end user applications may use the library. When we update something like Django, we have to make sure that we don't break anything using it (although, to be honest, the fact that we package webapps is an entire rant for later) before we get to update it to the newest release. Hell, with a few coffees, I could automate the process of releasing a .deb with a new upstream release, 100% unattended. I won't, however, since this is an insane idea. Let's go over a brief list of things I do before uploading a new package:

1. Review the entire codebase for simple mistakes.
2. Review the entire codebase for license issues.
3. Review the entire codebase for files without source, and track down (and include source for) any sourceless files (such as pickle files, etc).
4. Get to know the upstream, get to know open bugs, write something using the lib, in case I need to debug later.
5. Install the package.
6. Test the package.
7. Work out any Debian package issues (this is easy).

Now, a brief list of things I do before I update a package:

1. Review the changes between the last uploaded version (in diff format, if it's sane, otherwise get the VCS and review), ensure all the above are still OK.
2. Review for Debian-local issues (such as how it will upgrade, using piuparts, and adequate, etc).
3. Check to make sure it won't break any reverse dependencies.
4. Review for bugfixes that I might need to bring back to the stable release.
5. Figure out if I should (or even can) backport the package, if API is stable.
6. Review for bugs (upstream or in Debian) that I need to mention in the debian/changelog.

Clearly, this isn't a quick-and-dirty task. It's not a matter of getting a package updated (technically), it's a much more detailed process than that. This is also why Debian is so highly regarded for its technical virtuosity, and why the ISS decided to deploy Debian in space, despite other commercial distros such as Red Hat, or Ubuntu, and community distros, such as Fedora or Arch. It's also not Debian's job to package the world in the archive. This is an insane task, and it's not Debian's place to do it. We introduce libraries as things need them, not because we wrote some new library that someone may find slightly useful at some point in the future. maybe. Upstream developers and language communities (not only Python here) tend to lose sight of why we're doing this in the first place, which is our users. This isn't some sort of technical pissing contest to see who can distribute the software in the best way. Debian-folk always keep end users as our highest priority. I quote the Debian Social Contract, when I say that Our priorities are our users and free software. No one's trying to get developers to use dpkg to create software. In fact, as you'll see below, I actively discourage using system modules for development. The case for pip (the Pythonista in me) In defense of pip, the idea that Debian will keep the latest versions of packages is insane. The idea that we can keep pace with upstream releases is nuts, and the idea that every upstream release on pypi is ready to ship is bananas. b-a-n-a-n-a-annas. As a developer, I don't want to support every release, and I surely don't want other people depending on some random snapshot. Often times, I'll put stuff up on pypi as a preview, or to release often, and solicit feedback without having to give out instructions on using a git checkout (it's also easier to have them try a version from pypi so I can cross-ref the git tag to reproduce issues when they file them) pypi is easy, ubiquitous and works regardless of the platform, which means less of my development time is spent packaging stuff up for platforms I don't really care about (Arch, Fedora, OSX, Windows), even though I value feedback from users on those systems. The effort it takes to release something is limited to python setup.py sdist upload, and it's in a place (and in a shape) that anyone can use it without having 10 sets of platform-local instructions. Even ignoring all the above, when I'm writing a new app or bit of code, I want to be sure I'm targeting the latest version of the code I depend on, so that future changes to API won't hit me as hard. By following along with my dependencies' development, I can be sure that my code breaks early, and breaks in development, not production. Upstreams also tend to not like bug reports against old branches, so ensuring I have the latest code from pypi means I can properly file bugs. Lastly, I prefer virtualenv based setups for development, since I'm usually working on many things at once. This often means version mismatches in libraries, which brings in API changes (another whole rant here as well). I don't want to keep installing and uninstalling packages to switch between the two projects, and using a chroot(8) means a lot of overhead and that it's disconnected from my development environment / filesystem, so I resort to virtualenv to isolate my Development environment. Final notes I don't want to keep arguing about this. Just accept that the world's a big place and that there exist use-cases that both apt and pip need to exist and work in the way they're working now. At the very least, try and understand there exist smart people on both sides, and no one is trying to screw anyone over or keep their own little private club to themselves. Hopefully, going forward, we can make sure that the integration between these two tools gets better, not worse. Help make this dream a reality. Contribute to a productive tone, not a destructive one. In short:

• Use pip without sudo always. Don't tell people to use sudo.
• Use apt or dpkg when deploying system-wide.
• Understand people are going to package, and they will be more concerned about software using your library then keeping your library up to date.
• Understand Debian Developers and package maintainers have to do a lot of work when updating or sponsoring an upload.
• Understand upstream developers can't be bothered to fix every issue with every release (release early, release often) with some snapshot you introduced into unstable.
• Use pip and virtualenv in development setups, so we can upgrade your app when we upgrade the lib.

I ve been running several Deb-o-Matic instances on several servers since ages. This service, meant to provide a way to automate the build of Debian source packages, has been the starting point for a lot of Italian developers to start packaging, and eventually become Debian or Ubuntu developers. I can t exactly tell how many packages Deb-o-Matic servers have built so far, but it has been the primary build and test environment for at least three Debian Developers since four years. This service was hosted for free since the beginning, but the good things always come to an end. My provider decided they cannot keep our servers running (its core business is totally unrelated to hosting servers), and asked me to shut them down. I started with the two kfreebsd-* machines, and soon I will shut down i386 and amd64 as well. When all services will be down, I ll take care of removing the debian.net domains I created for them. Deb-o-Matic development won t stop, but developers who used Deb-o-Matic daily will have to find a different solution to build and test their packages.

A client has recently asked for my advice on web editing software. There are lots of programs out there for editing web sites and according to a quick Google search there are free Windows programs to do most things that you would want to do. The first thing I m wondering about is whether the best option is to just get a Linux PC for web editing. PCs capable of running Linux are almost free nowadays (any system which is too slow for the last couple of Windows versions will do nicely). While some time will have to be spent in learning a new OS someone who uses Linux for such tasks will be able to use fully-featured programs such as the GIMP which are installed as part of the OS. While it is possible to configure a Windows system to run rsync to copy a development site to the production server and to have all the useful tools installed it s much easier to run a few apt-get or yum commands to install the software and then copy some scripts to the user s home directory. The next issue is whether web editing is the best idea. Sites that are manually edited tend to be very simple, inconsistent, or both. Some sort of CMS seems to be the better option. WordPress is a CMS that I m very familiar with so it s easy for me to install it for a client, while I try and resist the temptation to force my favorite software on clients there is the issue that I can install WordPress quickly which therefore saves money for my client. WordPress is a CMS that supports installing different themes (and has a huge repository of free themes). The content that it manages consists of pages and posts , two arbitrary types of document. Supporting two types of document with a common look and feel and common important data in a side-bar seems to describe the core functionality used by most web sites for small businesses. Does anyone have any other ideas for ways of solving this problem? Note that it should be reasonably easy to use for someone who hasn t had much experience at doing such things, it shouldn t take much sysadmin time to install or cost to run.

EFI includes support for storing persistent 'variables': it implements a key-value store where keys are (UUID, string) and values are arbitrary blobs. (There are also some flags that restrict when the variables can be accessed.) The physical storage medium is normally flash. The Linux efivars module provides access to variable storage, both through a specific interface in sysfs and through the somewhat abstracted pstore interface. An EFI implementation reads the boot order and (if applicable) Secure Boot configuration from specific EFI variables. An operating system can add arbitrary variables, using its own UUID(s). On Linux, the efibootmgr utility is used to read and write the boot order and the pstore interface is used within the kernel to dump kernel log messages or other information when particular events occur. Since efivars is needed to set the boot order on EFI systems, it's a critical part of installation. But it wasn't being loaded on Debian systems until efibootmgr was run. This doesn't work so well when using a rescue system, as the installation kernel and installed kernel may be different (Debian bug 703363). In future (Debian package version 3.2.41-1) this will be loaded automatically on EFI systems. But now comes a new problem: bugs in the implementation of variable storage can prevent a system from booting, and even worse, they may prevent manual recovery without special equipment (the system is 'bricked'). Matthew Garrett recently talked about this and mentioned that some Samsung laptops are bricked if the variable storage becomes more than 50% full. efivars now has some extra checks to avoid triggering this (both in mainline and in stable branches). This isn't the end of the story, unfortunately. Remember that nothing can be overwritten in flash without erasing a relatively large block, and that writes may fail after a few thousand erase cycles (number depending on the type of flash used). Therefore, deleting a variable usually does not free storage immediately, and overwriting a variable usually reduces the amount of free storage. In the case of the Samsung laptops, unused storage is apparently reclaimed at the next reboot. However, while testing the backported efivars changes I found that my EFI-booting system (Asus P8Z68-V LX motherboard) does not reclaim unused storage until it's very nearly full. Since the firmware itself increments a counter variable on every boot, using 36 more bytes of storage, I was able to observe the available space dropping to under 100 bytes, and then after another boot jumping to about 42K (out of 64K). This means that well over 50% of storage was unused, yet efivars would refuse to write any variables because the firmware didn't reclaim it. This would prevent (re)installation of Linux, as the boot configuration could not be written. On the assumption that both behaviours are common, I've settled on a compromise that I hope minimises overall risk:

• Applied pending changes that allow pstore integration to be disabled at run-time, and made this the default. This means only efibootmgr is likely to write to variables.
• Removed the check that variable writes will not fill storage more than 50%.

But it's essential that we find a way to identify and work safely with both behaviours. Not only do I want to eliminate the possibility of bricking or installaton failure, but I want to make EFI-backed pstore safe to enable. Crash logs are often hard to obtain and pstore can help to solve that problem. Updated 2013-09-15: The free space check has been revised (in Linux 3.2.48, 3.5.7.17, 3.8.13.5, 3.9.7 and 3.10) based on advice from Samsung. The kernel leaves at least 5K (rather than 50%) of the EFI variable store free. If it is about to breach this limit, it attempts to write a dummy variable that is larger than the free space, which will either fail or trigger garbage collection. The normal variable can then be written if garbage collection freed up enough space. This check can be disabled using the kernel parameter efi_no_storage_paranoia, but there should be no need to do so.

Two months to the day after the previous 0.3.0 release of RProtoBuf, we are happy to announce a maintenance release 0.3.1 which has now arrived on CRAN. RProtoBuf provides GNU R bindings for the Google Protobuf data encoding library used and released by Google. Once again, Murray lead this release and added a number improvements. The NEWS file entry follows:

Changes in RProtoBuf version 0.3.1 (2013-09-13)

• Added support for setting and getting 64-bit integer types as character strings of decimal integers to work around R's lack of native 64-bit integer types.
• Added better error handling, documentation, and tests to the extensions support (getExtension and setExtension).
• Add support to P for returning extension descriptors.
• Improved error messages to include field names when invalid fields are specified to protocol buffer messages with new, update, etc.
• Improved configure to detect and pass -std=c++0x if it is available to enable long long 64-bit integer support in Rcpp.
• Improved configure in the case when pkg-config is not available or does not know about the google protocol buffer includes.
• Replaced newly deprecated Rcpp macros with a simplified macro functionality to avoid warnings on the latest development version of Rcpp.

CRANberries also provides a diff to the previous release 0.3.0. More information is at the RProtoBuf page which has a draft package vignette, a 'quick' overview vignette and a unit test summary vignette. Questions, comments etc should go to the rprotobuf mailing list off the RProtoBuf page at R-Forge.

I recently mentioned that there wasn't any built-in node.js functionality to perform IP matching against CIDR ranges. This surprised me, given that lots of other functionality is available by default. As a learning experience I've hacked a simple cidr-matching module, and published it as an NPM module. I've written a few other javascript "libraries", but this is the first time I've published a module. Happy times. The NPM documentation was pretty easy to follow:

• Write a package.json file.
• Run "npm publish".
• Wait for time to pass, and Thorin to sit down and sing about gold.

Now I can take a rest, and stop talking about blog-spam.