Search Results: "tar"

16 April 2012

Gunnar Wolf: Biking over Cd. Nezahualc yotl

April 15 2012, my friend Alberto and I took on a difficult mission: Cross the allegedly biggest city in the world and conquer Ciudad Nezahualc yotl, one of its populous and famous districts. Alberto had been to Cd. Neza a couple of times, as he is one of the providers for a physical rehabilitation center built in it, but I had never set foot in it. My friend Al and me started cycling in col. Escand n, in Mexico's Center-West, followed roughly along Metrob s' line 2 until its terminal, and at Cabeza de Ju rez entered Cd. Neza. We went along the main avenue "Adolfo L pez Mateos" until Bordo de Xochiaca, then all straight until V a Tapo, almost all around the airport, got lost a couple of times in the way, then back to the heart of the city. I took only a couple of quick shots, which I have uploaded to this album We saw so much, so huge, so surreal things, but the main goal we set for this was just to enjoy cycling. I can just complain about one thing: We cycled for maybe 10Km following the main open sewage line in Mexico City. I had never breathed so much shit in my life take it as literally as you want. But, all in all, an interesting little trip.

15 April 2012

Gregor Herrmann: RC bugs 2012/15

another weekend, another RCBW report:

Richard Hartmann: Choo choo two

Now that our departure is looming ahead of us, it's time to give a short update. (Turns out it's not so short, after all...) Preparation Visa Mongolian and Russian visas are done; we will apply for the Chinese ones next Monday. Travel All train tickets, hotels, hostels, and tents have been booked and both a horse-back tour through Mongolia and scuba diving in Lake Baikal have been pre-arranged. The Russian railway seems to do maintenance and development on the live website all the time which meant we were forced to book through a local travel agent. We paid a hefty extra charge for that privilege but still, that was less than half of what we would have paid if we had booked through a German agent. With the help of #china-lug on irc.freenode.net, we found accommodation in a traditional Siheyuan in an old Hutong. Hopefully, this will be a interesting as it sounds. Locals By the power of attending conferences and this blog, I now have local contacts in Moscow, Beijing, and Shanghai. Everyone who contacted me was extremely nice and helpful. I have been getting hints and tips galore; I doubt our preparations would be as neat as they are without this invaluable help. Thanks; you know who you are :) Packing list Apparently there's an interest in what we will be bringing; also, this will be a nice reminder for the future. Information and note-keeping Luggage Clothing Personal and computer security Fastening, storage, and tools Health Photography Minimal scuba gear Food Convenience Leisure That's it. Re-reading this list, I wonder how we managed to fit all of this into an 80-liter travel bag and a 15-liter backpack, but we did and with room to spare.

Richard Hartmann: All aboard the Choo Choo train!

All aboard the Choo Choo train! As some of you know, we are planning our trip on the Trans-Siberian (well, Trans-Mongolian to be exact) railway, at the moment. Given the wall of text my post about Svalbard turned out to be, I am trying to split early and often. The plan Our three-week itinerary is pretty much fixed, by now: Visa issues Not wanting to just book any random package with any random travel agent, we are picking and mixing our own, as usual. Given visa requirements, language barriers and things that are plain weird, it's been an interesting ride before we even pack our bags a month or two from now. Being European, visas are something that do not concern us a lot, normally. I am starting to realize how incredibly lucky we are... barring India, we never needed to do much to be allowed to enter any foreign country. Not so on this trip... While Mongolia has rather lax, or let's just say reasonable, terms, China and Russia are different. To get a visa for Russia, you need: For China, you need to provide a night-to-night itinerary of hotel stays. This may be a remnant of Mao's philosophy of restricting free movement of the population within China, I don't know. On the plus side, they don't require an invitation, health insurance or means of industrial espionage. With processing times ranging from one to four weeks per visa, you will wait one to two months until your passports are back with you and properly visa'ed by everyone. Tickets, please Purchasing tickets in advance, but not through a travel agent, is turning out to be rather complicated. As you can not book train tickets with RZD, the Russian rail company, more than 45 days in advance, you end up with nice circular dependencies. We could pay a premium for faster processing of our visas and buy via travel agencies which sell train tickets from their special contingents, but that's kinda boring, innit? Also, rzd.ru offers Russian order forms, only. With the help of Google translate, ##russian on irc.freenode.net, and a lot of guess-work, we were able to hammer out the connections from Moscow to Novosibirsk and from there on to Irkutsk. Yet, it's quite a different experience not to not be able to fall back to English. No matter where we went in the past, English was the lingua franca, at least to some extent. That's not the case in any of our three intended destinations so those phrase books will get used a lot, I suspect... We are still not sure if the 4-people cabins in second class have a door, let alone one that can be locked, but we decided that this is not too much of an issue. Generally speaking, people are nice and there are train attendants so crime should not be an issue. And even if things do disappear, as long as we stay healthy and I still have one of my several copies of all photos I will take, I will be reasonably happy. As to getting from Irkutsk to Ulan Bator... even a Russian travel agent we contacted claimed you can only purchase tickets on site, not online. From Ulan Bator to Beijing, it's a different booking system. And from Beijing to Shanghai, it's yet another one. This is the most fragmented booking process we ever went through. Other than a few moments of stress and frustration, the hunting for information, cross-referencing and having the occasional eureka moment is tons of fun, though; no complaints here. When in Rome... According to Lonely Planet, the Trans-Siberian is more or less one large picnic where everyone shares whatever they have with them with everyone else. I poked a few Russians about what typically German food they like and they came up with Frankfurters, mustard, and beer. Guess that's what we will be stuffing our backpack with once everything else is packed. And snuff; apparently it's common courtesy to offer your snuff to other men you meet in Mongolia and then snuff from their bottle. When in Rome... German snuff tends to be mixed with menthol, I suspect that will raise some eyebrows ;) No idea if there is any non-obvious social grease to bring to China; still working on that. Initially, we wanted to tour Mongolia's main attractions, but we would either have to sit in a car for days on end or go by plane. Driving for days in between a solid week(!) spent on trains is not very appealing and neither is breaking our personal ground travel record by cheating flying. Thus, we decided to take the eco-tourism route: After a two-hour orientation course in Mongolian behaviour and manners, we will travel a few hundred kilometers by public bus, be picked up by a local, driven out into the outback and ride from ger to ger by camel and horse before returning to Ulan Bator by bus. The stays at the gers are immersive, if short; we will live with the families, help them do their daily chores, and visit local places of interest in between. Oddly enough, I am looking to forward to the bus ride even more than to the gers. At the gers, at least one person is able to speak basic English. Not so with the local bus. People on the various trains will be used to strangers, as are the families at the gers. People on the bus are, most likely, not. Communicating with hand, foot, phrasebook, and a smile will, hopefully, be a lasting experience. If time permits, we will try to visit kiva.org borrowers while in Ulan Bator; this is something we never had a chance to do before and it's been on the bucket list for some time, now. As an aside, the local agent in Mongolia asked us if our sleeping bags are rated down to -26 degrees Celsius. I do hope he was joking... On the privileges of living in a first-world country... Travelling by horse means travelling light. Not a problem for us as we pack light by default and still manage to bring everything from Swiss Tool to water bottles, zip ties, medkit, and everything in between. Still, there's one major thing I have been taking for granted all my life: electricity. Being without any source of power for four to five days is... challenging. Even more so as every single gadget I rely on uses a different type of battery. Flashlight: CR123A and AA; GPS: Ni-MH AA rechargeables and Lithium AAA primaries respectively; Laptop, cameras, cell phone, ebook reader, Nintendo DS and MP3 player: proprietary. I am starting to truly understand why NATO has a hard rule of allowing AA-powered devices, only; planning spares is a pain. Obviously, I am focusing on flashlights, GPS, and, above all, cameras. Battery grips with AA adapters to the rescue! Another even more unsettling realization came when I asked if it would be possible to have boodog (vegetarians/vegans: don't click). Mongolians do not usually prepare boodog in spring as the animals which survived the harsh winter need to fatten and breed before being killed for food. All my life, I have never even once considered the remote possibility of not being able to slaughter domesticated animals due to outside constraints. Mongolian nomads need their animals. This is not about wanting to do things one way or another; this is about survival. Quite fascinating, and humbling, to think about. I am incredibly privileged and, as you are reading this, so are you. An exercise for the reader If you have any information regarding: please let me know. As usual, but just a little bit more than usual, comments are appreciated.

Petter Reinholdtsen: Debian Edu interview: Andreas Mundt

Behind Debian Edu and Skolelinux there are a lot of people doing the hard work of setting together all the pieces. This time I present to you Andreas Mundt, who have been part of the technical development team several years. He was also a key contributor in getting GOsa and Kerberos set up in the recently released Debian Edu Squeeze version. Who are you, and how do you spend your days? My name is Andreas Mundt, I grew up in south Germany. After studying Physics I spent several years at university doing research in Quantum Optics. After that I worked some years in an optics company. Finally I decided to turn over a new leaf in my life and started teaching 10 to 19 years old kids at school. I teach math, physics, information technology and science/technology. How did you get in contact with the Skolelinux/Debian Edu project? Already before I switched to teaching, I followed the Debian Edu project because of my interest in education and Debian. Within the qualification/training period for the teaching, I started contributing. What do you see as the advantages of Skolelinux/Debian Edu? The advantages of Debian Edu are the well known name, the out-of-the-box philosophy and of course the great free software of the Debian Project! What do you see as the disadvantages of Skolelinux/Debian Edu? As every coin has two sides, the out-of-the-box philosophy has its downside, too. In my opinion, it is hard to modify and tweak the setup, if you need or want that. Further more, it is not easily possible to upgrade the system to a new release. It takes much too long after a Debian release to prepare the -Edu release, perhaps because the number of developers working on the core of the code is rather small and often busy elsewhere. The Debian LAN project might fill the use case of a more flexible system. Which free software do you use daily? I am only using non-free software if I am forced to and run Debian on all my machines. For documents I prefer LaTeX and PGF/TikZ, then mutt and iceweasel for email respectively web browsing. At school I have Arduino and Fritzing in use for a micro controller project. Which strategy do you believe is the right one to use to get schools to use free software? One of the major problems is the vendor lock-in from top to bottom: Especially in combination with ignorant government employees and politicians, this works out great for the "market-leader". The school administration here in Baden-Wuerttemberg is occupied by that vendor. Documents have to be prepared in non-free, proprietary formats. Even free browsers do not work for the school administration. Publishers of school books provide software only for proprietary platforms. To change this, political work is very important. Parts of the political spectrum have become aware of the problem in the last years. However it takes quite some time and courageous politicians to 'free' the system. There is currently some discussion about "Open Data" and "Free/Open Standards". I am not sure if all the involved parties have a clue about the potential of these ideas, and probably only a fraction takes them seriously. However it might slowly make free software and the philosophy behind it more known and popular.

Vasudev Kamath: Update: Falcon Debian package Status

Recently I wrote about packaging Falcon programming language for Debian. So here is the status of package. Package builds multiple binaries and we have successfully separated modules from core Falcon language component
  1. falconpl (Falcon language itself )
  2. falconpl-dbg (Debug symbols)
  3. libfalconpl-engine1 (Falcon engine library)
  4. libfalconpl-engine1-dbg (Debug symbols for Falcon enging)
Core modules like feathers and modules writtein Falcon itself are part of libfalconpl-engine1. Rest of modules are separated into following packages
  1. falconpl-mongodb: Mongodb connector module
  2. falconpl-dbi : Database Abstraction layer
  3. falconpl-dbi-sqlite3: SQLite3 connector based on DBI module
  4. falconpl-dbi-postgresql: PostgreSQL connector based on DBI module
  5. falconpl-dbi-mysql: MySQL connector based on DBI module
  6. falconpl-dbi-firebird: Firebird DB server connector
  7. falconpl-curl: Curl bindings for Falcon
  8. falconpl-dbus: DBus bindings for Falcon
  9. falconpl-dmtx: Data Matrix reader/writer for Falcon
  10. falconpl-hpdf: Haru based PDF module for Falcon
  11. falconpl-gd2: Graphic manipulation module for Falcon
There are 4 modules which I'm not sure how to build (dependency) or if at all possible to build for Linux they are dynlib, wopi, sdl, conio. I need to consult upstream on these modules status and ability to build. Modules not built We didn't build dbi-oracle and dbi-odbc modules as these are proprietary software dependant. Additionally in a conversation with upstream on IRC it was told dynlib module can't be built for 64bit and X86 version of Linux because lack of assembler, but I've not fully confirmed this. Legal Issues To enable the curl bindings I added curl dependency which in turn has dependency on libssl1.0 library which is part of openssl source package which is causing lintian thrown GPL violation error. Kartik will be looking into this. Patches submitted I've submitted total of 6 patches of which 5 patches are already applied to upstream git repo and 6th one is waiting to be applied. In other words upstream is really helpful to us. Well thats all for this week more updates next week (or may be even before that ;-) )

14 April 2012

NeuroDebian: NeuroDebian nd* tools

NeuroDebian nd* tools One of the goals of NeuroDebian is to provide recent versions of scientific software on stable Debian (and Ubuntu) deployments. That is why we build (whenever possible) every new package not only for the Debian unstable (the entry point of packages into Debian) but also for Debian testing and stable, and Ubuntu releases. To automate such procedure we prepared few rudimentary wrappers around cowbuilder allowing to build packages in isolated environment. Also we provide a backport-dsc script to ease backporting with optional application of per-release patchsets. In this blog post we would like to introduce you to these tools. They will be of use for anyone working on a package intended to be uploaded to NeuroDebian repository or anyone interested to verify if package could be easily backported. With a single command you will be able to build a given Debian source package across distributions. As a result you will verify that there are no outstanding backport-ability issues or compatibility problems with core components (e.g. supported versions of Python) if your source package excercises test suites at build time.
Procedure
  • [1-20 min] If you are not running Debian-based distribution, Install NeuroDebian VM; otherwise just add apt sources for NeuroDebian repository.

  • [<1 min] Install the neurodebian-dev package providing nd* tools:

    sudo apt-get install neurodebian-dev
  • [1-5 min] Adjust default configuration (sudo vim /etc/neurodebian/cmdsettings.sh) used by nd commands to

    • point cowbuilderroot variable to some directory under brain account, e.g. ~brain/debs (should be created by you)
    • remove undesired releases (e.g. deprecated karmic) from allnddist and alldist
    • adjust mirror entries to use the Debian mirror and Ubuntu mirror of your choice or may be even point to your approx apt-caching server
  • [10-60 min] Create the COWs for all releases you left in the configuration file:

    sudo nd_adddistall
Building At this point you should be all set to build packages for all distributions with a single command. E.g.:
sudo nd_build4all blah_1-1.dsc
should take the .dsc file you provide, and build it for main Debian sid and all ND-supported releases of Debian and Ubuntu. nd_build4allnd would build only for the later omitting the vanilla Debian sid. The highlevel summary either builds succeed or failed get reported in summary.log in the same directory, pointing to .build log files for the corresponding architecture/release.
Troubleshooting Failing Build Provide --hookdir cmdline pbuilder argument to point to a hook which would get kicked in by pbuilder upon failure, e.g.:
sudo apt-get install git
git clone https://github.com/neurodebian/neurodebian
sudo nd_build4debianmain *.dsc -- --hookdir $PWD/neurodebian/tools/hooks
If you have any comments (typos, improvements, etc) feel welcome to leave a comment below, or contact us .

Paul Tagliamonte: pyutf (or, as it's known on my machine - .py)

code can be found on my github Forewarning, I can t actually tell if that second UTF (PILE-OF-POO) actually renders, since my typeface is lame. Anyway, introducing an idea that fermented a few days ago with some of the bopeeps ( iros, jugglinmike, tbranyen, greg) - known initially as .py (pronounced snowman-pile-of-poo-dot-py) Here s a code sample:
#!/usr/bin/env pyutf
lis = [ 1, 2, 3, 4, 5, 4, 3, 2, 1 ]
g =   x: x *  
  a   lis:
    print g(a)
Which outputs:
3.14159265359
6.28318530718
9.42477796077
12.5663706144
15.7079632679
12.5663706144
9.42477796077
6.28318530718
3.14159265359
Patches welcome, and bonus points for a py2snowmanpoo / snowmanpoo2py pull request. We ll call it a code compressor. Also, if someone could fix the loading the whole script at startup thing too. Yes, this is a joke. But also, yes, it works.

Axel Beckert: Automatically hardlinking duplicate files under /usr/share/doc with APT

On my everyday netbook (a very reliable first generation ASUS EeePC 701 4G) the disk (4 GB as the product name suggests :-) is nearly always close to full. TL;DWTR? Jump directly to the HowTo. :-) So I came up with a few techniques to save some more disk space. Installing localepurge was one of the earliest. Another one was to implement aptitude filters to do interactively what deborphan does non-interactively. Yet another one is to use du and friends a lot ncdu is definitely my favourite du-like tool in the meanwhile. Using du and friends I often noticed how much disk space /usr/share/doc takes up. But since I value the contents of /usr/share/doc a lot, I condemn how Nokia solved that on the N900: They let APT delete all files and directories under /usr/share/doc (including the copyright files!) via some package named docpurge. I also dislike Ubuntu s solution to truncate the shipped changelog files (you can still get the remainder of the files on the web somewhere) as they re an important source of information for me. So when aptitude showed me that some package suddenly wanted to use up quite some more disk space, I noticed that the new package version included the upstream changelog twice. So I started searching for duplicate files under /usr/share/doc. There are quite some tools to find duplicate files in Debian. hardlink seemed most appropriate for this case. First I just looked for duplicate files per package, which even on that less than four gigabytes installation on my EeePC found nine packages which shipped at least one file twice. As recommended I rather opted for an according Lintian check (see bugs. Niels Thykier kindly implemented such a check in Lintian and its findings are as reported as tags duplicate-changelog-files (Severity: normal, from Lintian 2.5.2 on) and duplicate-files (Severity: minor, experimental, from Lintian 2.5.0 on). Nevertheless, some source packages generate several binary packages and all of them (of course) ship the same, in some cases quite large (Debian) changelog file. So I found myself running hardlink /usr/share/doc now and then to gain some more free disk space. But as I run Sid and package upgrades happen more than daily, I came to the conclusion that I should run this command more or less after each aptitude run, i.e. automatically. Having taken localepurge s APT hook as example, I added the following content as /etc/apt/apt.conf.d/98-hardlink-doc to my system:
// Hardlink identical docs, changelogs, copyrights, examples, etc
DPkg
 
Post-Invoke  "if [ -x /usr/bin/hardlink ]; then /usr/bin/hardlink -t /usr/share/doc; else exit 0; fi"; ;
 ;
So now installing a package which contains duplicate files looks like this:
~ # aptitude install perl-tk
The following NEW packages will be installed:
  perl-tk 
0 packages upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 2,522 kB of archives. After unpacking 6,783 kB will be used.
Get: 1 http://ftp.ch.debian.org/debian/ sid/main perl-tk i386 1:804.029-1.2 [2,522 kB]
Fetched 2,522 kB in 1s (1,287 kB/s)  
Selecting previously unselected package perl-tk.
(Reading database ... 121849 files and directories currently installed.)
Unpacking perl-tk (from .../perl-tk_1%3a804.029-1.2_i386.deb) ...
Processing triggers for man-db ...
Setting up perl-tk (1:804.029-1.2) ...
Mode:     real
Files:    15423
Linked:   3 files
Compared: 14724 files
Saved:    7.29 KiB
Duration: 4.03 seconds
localepurge: Disk space freed in /usr/share/locale: 0 KiB
localepurge: Disk space freed in /usr/share/man: 0 KiB
localepurge: Disk space freed in /usr/share/gnome/help: 0 KiB
localepurge: Disk space freed in /usr/share/omf: 0 KiB
Total disk space freed by localepurge: 0 KiB
Sure, that wasn t the most space saving example, but on some installations I saved around 100 MB of disk space that way and I still haven t found a case where this caused unwanted damage. (Use of this advice on your own risk, though. Pointers to potential problems welcome. :-)

Dominique Dumont: My setup for vdr automatic wake up and shutdown

Hello Like all kids, my kids really enjoy watching cartoons. Unfortunately, most of them are broadcast during the morning, sometimes very early in the morning. Since I don t want my kids to wake up (or wake me up ) too early in the morning to watch cartoon, I record the most interesting programs with vdr (I also edit out commercials before my kids watch them) To limit my carbon (or equivalent radio-active waste) footprint, I do not want to leave the computer on 24/24, so the machine has to power up by itself before a recording time. So I ve setup vdr to automatically setup the wake up time of the computer and shutdown the machine once the recording is finished. Note that this machine is also my main home machine, so it must not shutdown while I m working on it (typing this blog, for instance). This should have been easy and fairly documented. Well, it was not that easy. A lot of doc mentions nvram-wakeup which is more or less obsolete. Other docs were in German. So, here s a summary of the setup so other can benefit from my experience. First, this was done on a Debian/sid machine, this post should be usable on other distros, provided some adaptations. These adaptations are left as an exercise to the reader (which is marketing lingo for I don t have a clue ) First add this line in /etc/default/vdr:
 ENABLE_SHUTDOWN=1
When vdr has nothing to do, it will run some shutdown scripts, including /usr/share/vdr/shutdown-hooks/S90.custom, which is a link to /etc/vdr/shutdown-hooks/S90.custom. The output of this script will be eval ed by vdr, so vdr knows what to do next (e.g. abort shutdown, display a message to user, retry later ). Here are the possible output of a vdr shutdown script (list taken from vdr-wiki): This script will mess with /sys/class/rtc/rtc0/wakealarm so it must be run as root. But vdr will launch this script and does not run as root. Instead of setting a suid bit, let s setup sudo to enable this script to gain root privilege from vdr. Add this line in /etc/sudoers (with visudo):
vdr ALL= NOPASSWD: /etc/vdr/shutdown-hooks/S90.custom *
And here s a commented Perl script that handle the shutdown test:
#!/usr/bin/perl
use IO::File ;
use 5.10.0 ;
use English;
use Path::Class; 
use Sys::Syslog qw(:standard :macros);  # standard functions & macros
openlog('vdr-shutdown', "", "user");
Here s the call to sudo to gain root privilege (if not already run as root):
if ($EUID)  
    system('sudo','/etc/vdr/shutdown-hooks/S90.custom', @ARGV) ;
    syslog(info => "vdr-shutdown: sudo myself returned $@") ;
 
Get the next timer info from the command arguments provided by vdr:
my ($next_timer,$interval) = @ARGV ;
Setup the next alarm in the mother board:
if ( $next_timer)  
  my $dev_fh = IO::File->new("> /sys/class/rtc/rtc0/wakealarm");
  die "cannot open $dev_fh: $!" unless $dev_fh ;
  my $newtime = $next_timer - 300  ;  # Start 5 minutes earlier
  $dev_fh->print( "$newtime\n" ) ;
  $dev_fh->close ;
  syslog(info => "vdr-shutdown: set wakeup to $newtime") ;
 
Now, let s check if someone is logged on the system. Weirdly enough, forking a command from this Perl script does not work when it s run by vdr. Calling the who command always return empty. The trick to find if anyone is logged is to check the content of /dev/pts directory. No files in there means that nobody is logged:
my @pts  = dir('/dev/pts')->children;
syslog(info => "vdr-shutdown:  found pts @pts") ;
if (@pts > 1)  
    syslog(info => "vdr-shutdown: retry later: someone is logged") ;
    say 'ABORT_MESSAGE="someone is logged, shutting down deferred"';
    say 'TRY_AGAIN=30'
 
else  
    syslog(info => "vdr-shutdown: nobody is logged, halting") ;
 
Now some cleanup then exit:
closelog() ;
exit 0;
This script has been working for one month, and not a single programs was missed. My kids are happy. I m not so happy because I now have a lot of commercials to edit out ;-) All the best
Tagged: debian, Perl, vdr

Paul Tagliamonte: gcc 4.7 patchathon

For no reason other then I had some time, I went on a patchathon against gcc-4.7 ftbfss while watching frasier This log is really for my own use :)

Craig Small: PHP floats and locales

I recently had a bug report in jffnms that the SLA checks were failing with bizarre calculations. Things like 300% disk drive utilization and the like. Briefly, JFFNMS is written in PHP and checks values that come out of rrdtool and makes various comparisons like have you used more than 80% of your disk or have there been too many errors. The logs showed strange input variables coming in, all were integers below 10. I don t know of many 1 or 3 kB sized disk drives. What was going on? I ran a rrdtool fetch command on the relevant file and got output of something like 1,780000e+07 which for an 18GB drive seemed ok. Notice the comma, in this locale that s a decimal point hmm. In lib/api.rrdtool.inc.php there is this line around the rrdtool_fetch area:
$value[] = ($line1[$i]=="nan")?0:(float)$line1[$i];
A quick check and I was finding that my 1,7 e+07 was coming back as 1. We had a float conversion problem. Or more specifically, php has a float conversion problem. I built a small check script like the following:
setlocale(LC_NUMERIC,'pl_PL.UTF-8');
$linfo = localeconv();
$pi='3,14';
print "Decimal is \"$linfo[decimal_point]\". Pi is $pi and ".(float)($pi)."\n";
print "Half is ".(1/2)."\n";
Which gave the output of:
Decimal is , . Pi is 3,14 and 3 Half is 0,5
So PHP is saying that decimal point is a comma and it uses it BUT if a string comes in with a comma, its not a decimal point. Really?? Are they serious here? I tried various combinations and could not make it parse correctly. The fix was made easier for me because I know rrdtool fetch only outputs values in scientific notation. That means if there is a string with a comma, then it must be a decimal point as it could never be used for a thousands mark. By using str_replace to replace any comma with a period the code worked again and didn t even need the locale to be set correctly, or that the locale for the shell where rrdtool is run is the same as the locale in php.

Wouter Verhelst: LOAD: tutorial on Debian Packaging

Two weeks ago, I was at LOAD, where I did a tutorial on Debian Packaging. Unfortunately, I wasn't doing very well physically, and as a result my preparation wasn't what I had hoped for. It ended up being not so much a tutorial as a walking the audience through packaging a piece of software in casu NBD with debhelper in dh mode. Not very difficult, but since there's nothing for people to fall back on afterwards, it may help if they have a writedown of what I said during the session; and I promised to put it on my blog. So, here goes. However, note that the canonical written tutorial on Debian Packaging is here. With apologies to readers of Planet Debian, most of whom for which all this is probably old hat. Unlike RPM packaging, in Debian, the data that tells the debian packaging system how to package a piece of software is functionally split among a number of files. All these files go in a toplevel directory in the source package, (unsurprisingly) called 'debian'. There are four files that are required for every Debian package: the control, changelog, copyright, and rules files. Without these files, dpkg-dev will fail to produce a package any package. The first, the control file, contains metadata on packages: the name of the package, its description, the dependencies, etc. Basically, almost all the information you see when you run 'apt-cache show <package>' (with one exception) is in the control file. If you have a source package that builds multiple binary packages, then you should have multiple 'Package' stanzas in the control file. In my opinion, creating a control file is easiest done by copying a control file from another (similar?) package, and modifying it to suit the software you're dealing with. The one exception, the one bit of metadata that is not found in the control file is the version number: this data is contained in the changelog file. This file looks like a free-form file for the most part, but it really is a machine-parsable format; as such, it's best to edit it with specialized tools, such as the debian-changelog-mode in the emacs editor, or the debchange script, also available as dch, from the devscripts package. In the changelog, you should document any changes you make to the package, making sure the version number (top line), distribution (unstable, experimental, stable, ...; top line), urgency (top line; used mainly when testing needs to be updated urgently for things like security updates), author (bottom line) and date and time (also bottom line) are correct. The copyright file, unsurprisingly, should contain the copyrights statements of the original software, and the license (or a reference to a copy of the same license text under /usr/share/common-licenses). This file is still free-format in most packages today, although a machine-readable format for this file has recently been defined. Of the four required files, this is probably the most boring one, but hey, we can't like everything we do. The last file, the rules file, is where all the action happens. This file is defined as a Makefile, of which the targets are called for various parts of the build system. The rules file has a number of required targets (such as build, clean, and others), but if you're using debhelper in dh mode, you don't need to worry about those; instead, you can use the following simple rules file:
#!/usr/bin/make -f
%:
	dh $@
With the first line, the shebang, we make clear that this is a makefile, and that it should be called by make. The next is a generic target (the % is a wildcard for make), with just one command: 'call dh with the name of the target being called'. Since dh implements all required rules targets, that immediately gets you a working package. Go ahead, try! Run 'dpkg-buildpackage -rfakeroot', and see what happens. Did that work? Maybe, maybe not. First, you'll have seen loads of warnings about something involving a "compatibility level". This is from debhelper; this compatibility level, or 'API level' if you wish, allows debhelper to move forward and make incompatible changes without breaking all the existing packages out there; whenever the compat level is bumped, some new functionality will only be made available if you also raise the compat level in your source package (and then you'll know you may have some changes to do in your package). Since the original debhelper, over a decade ago, did not have a compat level, the absense of a compat level signals compat level 1, which is far outdated now, and about to be unsupported. Hence the warning. The fix is simple: create a file debian/compat containing a single number: the compatibility level you're working with. It's best to use the most recent level which debhelper supports when you create your package, which as of this writing is level 9. Ignoring the compatibility level, if you're building a software package which uses a well-established build system, and you only build one binary package out of that, chances are pretty high that everything worked as expected. If not, you'll have more work. Building multiple packages requires that you tell debhelper somehow which file goes in which package. You can do this with a file debian/package.install, where package is the name of the binary package you're building. The install file is read by dh_install, one of many tools in the debhelper suite. You see, in the old days (before debhelper 7), debhelper was just a suite of tools, which required that you wrote a debian/rules file containing all the individual tools to be called in the correct order. This mode is still supported; in fact, even if you do use dh, you do, since all that tool really does is call the right tools in the right order; the real work is done by the invidual tools. They all have their own man page; to understand how dh_install chooses which file to put in which package, go read man dh_install. Go ahead, do it now; I'll wait. Back? Good. You've now learned that dh_install can install files either from the source directory (useful for packages containing only scripts or so), or from a directory debian/tmp. If you use an autotools-based software package, this is what dh will do for you. When you called dpkg-builpackage above, you may also have noticed that the output contained many lines starting with dh_, one of which said dh_install. As you may have guessed, dh echoes every debhelper command just before it will execute them. This allows you to look at the output, and see what's happening. For more detail, set the environment variable DH_VERBOSE to a non-zero value. I'm sure you'll have seen one or two debhelper tools that could make your package better. Go ahead, go and read their man pages to see what they do, and how they do it. Most of these will require you to create a file debian/package.toolname to specify details. In some cases, you may need to specify command-line arguments to the tool to get it to do what you want. In yet other cases, the tool won't support what you need it to do, and you'll have to do something manual. What to do now? The dh command line doesn't support adding extra commands easily. Does this mean you'd have to revert to old-style long, non-dh debian/rules files? Luckily, no. You can create an override target. If dh detects that you have such a target for a particular tool, it will call that target instead of the tool. This target's rules can then call the tool in question (or not), and add any command line arguments, or extra commands, as needed. An override target is a normal Makefile target with a name of the form override_dh_something, where dh_something is the name of the tool you wish to override. At this point, I'd reached almost the end of my two allotted hours, and a member of the audience asked how to make dpkg deal with configuration files. The answer is, you don't need to do anything! Not if you use debhelper, anyway; all you need to do is install the file in /etc. Since debian policy specifies that all files in /etc need to be configuration files and that no conffiles may be placed anywhere but in /etc, debhelper will automatically mark any file installed in /etc as a conffile, which will cause dpkg to ask the user what to do with changes to such files. Note that in Debian parlance, a conffile is not the same thing as a configuration file: a conffile is a configuration file that is part of the binary package (i.e., if you call 'dpkg --contents' on the .deb file, you'll see it in the output), whereas a configuration file is a file (any file, including files generated during or after installation of the .deb file on a system). Occasionally, this is also why some differences to configuration files are managed through debconf, while others aren't: the ones managed through debconf are non-conffile configuration files managed through ucf, whereas the others are conffiles. So in the simplest of cases, if you want to install a configuration file and you want to make sure it's protected against accidental overwrites by package upgrades, all you need to do is make sure it's installed to /etc; debhelper will do the rest. And that concludes this introduction. Please note it's only an introduction, not a full-blown tutorial; while this will allow you to get started, you may have to learn a bit more if you wish to eventually upload a package to Debian.

Craig Sanders: why is bzr so slow?

from the whinge of the day dept: I started a bzr branch of calibre about 2.5 hours ago because I wanted to see how difficult it would be to understand the code and make a few changes. The calibre Get Involved page warns that it can take about an hour .which is excessive to begin with and, worse, a huge understatement.
$ date ; ps u -Cbzr
Sat Apr 14 15:10:00 EST 2012
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
cas      14279  3.0  1.3 286836 215976 pts/3   S+   12:39   4:36 /usr/bin/python /usr/bin/bzr branch lp:calibre
It's still far from finished.
cas@ganesh:/usr/local/src/calibre$ bzr branch lp:calibre
You have not informed bzr of your Launchpad ID, and you must do this to
write to Launchpad or access private data.  See "bzr help launchpad-login".
1395260kB    65kB/s - Fetching revisions:Inserting stream:Estimate 178815/203940
a git clone of a much larger project would have been finished in mere minutes. WTF is bzr so horrendously slow? It's not my connection, I'm on an otherwise idle ADSL2 connection, that syncs about 14Mbps. i.e. capable of downloads of up 1.4 megabytes per second, not the 50-70kB/s that bzr is dawdling along at. It's not my CPU, an AMD 1090T hex-core overclocked to 3.7GHz (and almost completely idle) and since I have 16GB RAM, it's not lack of RAM either. Is there anything good about bzr that makes people actually want to use it? or is it just the association with ununtu and launchpad? maybe the purpose is to actively discourage casual involvement .you have to make a massive life commitment and run the gauntlet of tediously long waits before you can even look at the code. No wonder github is so popular. why is bzr so slow? is a post from: Errata

13 April 2012

David Welton: Mr. Blank, we're outside the building, and we want eBooks!

Steve Blank is known for his teachings on the Silicon Valley type of entrepreneurship, with his ideas forming the basis for the "lean startup movement" amongst other things. He writes frequently on entrepreneurship, and with a great deal of credibility, having been involved in various startups in a number of roles. He has, without a doubt, walked the walk in terms of startups, and now seems to be spending his time helping other people learn how to walk the same path. That's a noble thing to be doing when, with the money he's made, he could probably be off doing pretty much whatever he wants. If you've heard of Steve Blank, you've probably also heard his famous phrase: "get out of the building", an admonition to startup founders to get out and talk with their customers to validate their ideas, rather than huddling in their offices building something that may or may not have a market. With that in mind, when I saw he had a new book out, The Startup Owner's Manual, I thought "great, that's one I'll get without hesitating!". Unfortunately, though, an eBook wont' be out until "2nd half of 2012"! Ouch. To me, his ignoring eBooks is indicative of a need to get a bit further outside the building, though. "I want an eBook" was probably the biggest request on his blog post announcing the new book, along side messages of thanks for writing the book. After reading, on Blank's blog about the availability of the book from BookDepository Ltd, who offer free worldwide shipping, I went ahead and ordered it even if I would have prefered the eBook. Since they're in the UK, and I'm in Italy, I figure it can't take that long, right? Wrong. I ordered on March 15th, and as of April 13th, it still isn't here. Compare and contrast with the other books I'm currently reading which I was able to order and start looking at in just a few minutes on my Kindle. Granted, Steve Blank surely isn't doing this for the money, and from that point of view has little real need to listen to his customers - it's not wrong to say he's doing the world a favor by writing the book in the first place. If he thinks a paper version is far superior, that's his perogative. However, I think he's doing a lot of his readers a disservice by not making the eBook available sooner. I know I would have liked to start reading what he had to say last month, rather than waiting for a paper book to make its way (by mule train?) down here to Italy. The crux of the matter is that while he may well be right in thinking a paper book is "better", for some people, an eBook is the only option, and for them, an "inferior" eBook is a heck of a lot better than no book at all. Also, on a more constructive note, with eBooks, you can get pretty creative. For instance, if you have a tabular worksheet, you can simply hyperlink to it in, say, Google Docs, so that those with more advanced devices like iPads can open up the link and start working with a real, live spreadsheet immediately, rather than a chart in a printed book. Granted, that means 'giving away' the worksheet, but presumably it's not that valuable on its own, and makes for great advertising if it gets a lot of attention. Finally, since I actually run a business that does eBook conversions , on the blog post announcing the book, I offered to donate our services, so he'd get his book done for free, so you can't accuse me of just complaining! Mr. Blank, get out of that building and make an eBook available, please!

Peter Eisentraut: Enabling core files for PostgreSQL on Debian

The other day, I was a bit puzzled over a seemingly simple task: Enable core files to be generated from a PostgreSQL instance running on Debian. That instance has unfortunately been segfaulting on occasion, but never left a core file.

Now in principle it is clear that
ulimit -c unlimited
is the incantation to get this done. But where do you put this? You could hack it into the init script, but that seemed a bit ugly, and I wanted a sustainable solution.

A useful thing in the meantime is to check the current settings. That information is available in /proc/$PID/limits with the PID of the postmaster process (or any child process, really), and it looked like this to begin with:
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time unlimited unlimited ms
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
...
Use sudo grep core /proc/$(sudo cat /var/run/postgresql/8.4-main.pid)/limits if you want it automated.

So it's good to know that we only need to set the soft limit.

One way to configure this properly would appear to be in /etc/security/limits.conf. There you can add a line like
*               soft    core            unlimited
to enable core dumps globally. I'm not actually sure whether that would work if the service is started during the boot without PAM. In any case, I didn't want to enable core files globally; who knows what that would lead to.

One could replace the * by a user name, such as postgres, and then enable pam_limits.so in /etc/pam.d/su. But the postgresql init script in Debian is nested about four levels deep, so it wasn't clear whether it called su at all.

Now as it turns out, the init script ends up changing the user using this Perl code:
$) = $groups;
$( = $gid;
$> = $< = $uid;
(see change_ugid in /usr/share/postgresql-common/PgCommon.pm), so the whole PAM line of thought wasn't going to work anyway. (Other packages such as pgbouncer and slony1 do got through su, so that would be a solution for those.)

The way to solve this is the pg_ctl -c option, which sets the soft limit for core files to unlimited. And the way to pass this option through the init script maze is the file /etc/postgresql/8.4/main/pg_ctl.conf, which should contain a line like this:
pg_ctl_options = '-c'
Then restart postgresql, and check /proc/$PID/limits again:
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time unlimited unlimited ms
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size unlimited unlimited bytes
OK.

Another thing that's recommendable in this context is to change the core file names to have a unique element, so that if multiple backends crash before you can take a look, they don't overwrite each other's core files. The core(7) man page explains the configuration options; I went with this sysctl setting:
kernel.core_pattern = core.%e.%p
which includes process name and PID. The PID file still ends up in the data directory of the PostgreSQL instance, which could also be changed, but I didn't find it necessary.

Stick the above line in /etc/sysctl.d/local.conf and reload with
service procps force-reload
I actually use a setting like that on all machines now; it's just nicer.

OK, and now I'll wait for the next core file. Or not.

Martin F. Krafft: Mouse on Mars

The atmosphere in Munich s Backstage Werk just before the opening act to the Mouse on Mars was very chilled. People sat on the stairs or scattered themselves over the dance floor while low-fi ambient tunes came from the speakers. It wasn t loud, you had to try hard to hear the people mumble. I have no idea who the opening act was, and their first tune was very nice and groovy. Then ensued a noise explosion, one could only pity the electronic equipment that was being asked to perform in ways that may be described as everything else than you expect , and of course, the base beat shook the building; I am quite sure they didn t use treble at all, but I may also simply have been unable to hear it. Plus, it seemed to us that the musicians catered for what may be a widespread decrease of attention span: it was noticable how they jumped from one thing to the next, not leaving them (or their listeners) any time to get in the groove. My brother and I went outside for a bit and talked about today s music and its simplicity. We postulated the repetitiveness as the basis of a mass movement, considered scene clubs that played heavy techno to an audience that is so entirely different to who historically frequented such musical performances, and in general tried to avoid assuming a position between simplifying society and accepting that individual freedom is as eclectic as can be. When MoM opened, they continued pretty much in line with their openers and half way through the first tune, I started to wonder how long I would last, or when it would be reasonable to step outside again. I had been a little afraid this would happen, having bought and listened to their latest album Parastrophics in preparation of the concert and not being able to get into it. However, what then followed blew us away. Still heavy, still all over the place, but now they were developing sound scenes, ripping them apart, having fun playing with and teasing the audience, while putting on a groove that inevitably made your muscles twitch with the beat. David Bowie called MoM the next big thing and I have to give it to them: MoM have always had a certain aura of that s what your music is like? we can improve on that! to them, and yesterday, they continued along those lines with astounding consistency, and it felt fresh. It also felt real. They weren t just pushing buttons and computers making music, they were making music and the computers were their instruments. Between the two founding members of MoM sat Dodo Nkishi, drummer and microphone artist, and if you don t believe, fast, big breakbeat can be performed live, well, you re wrong. Most everyone in the room was dancing. And while I was more swaying in awe, watching and wondering how the heck they are doing what they are doing, I couldn t contain the bouncing any longer. They came back for an encore and there was no more stopping the crowd, Thomas or me. Three tracks later, they waved goodbye and left, but a bunch of us simply continued to dance. Thomas questioned who would last longer and I started yelling loudly for another encore. The lights turned on, I considered it a slap in the face, but I did not stop yelling. Others tuned in. And then the lights went off and the band came back. Following their 2.5 hour show, gosh was I exhausted. It was a magnificent show. If you aren t afraid of big beat electronica and you take pleasure in nonstandard art, I heartily recommend you ensure that MoM aren t soon playing near you without you there. PS: MoM will play at the (D sseldorf Open-Source Festival)[http://www.open-source-festival.de/en/] on 30 June 2012! PPS: Now I listen to Parastrophics and I am really enjoying it. NP: Mouse on Mars: Parastrophics

12 April 2012

Andrew Cater: Raspberry Pi credit card computer - first two thousand shipping

RS and Farnell have started contacting the first wave of customers: some of the first Raspberry Pi boards will be shipping tomorrow and over the weekend.

Similarly, the two distribution companies are ramping up for constant rolling production and mass volume shipping.

Raspberry Pi post

[Courtesy www.raspberrypi.org - Raspberry Pi is a trademark of the Raspberry Pi foundation which is a registered charity in the UK]

Thomas Koch: No more self-hosting

A few years ago I started to run my own server. This was an interesting challenge back then but today I only consider it a waste of time and money. Therefore I'm searching for (a) service provider(s) to substitute my self hosted applications:

However I'm a bit insane about privacy, security and reliability. I simply want to be able to TRUST. This rules out IMHO: Google, Github, Yahoo. On the other hand I'd probably be prepared to pay an insane monthly fee for a provider that just keeps me happy.

I'd love to here your recommendations! I'm especially interested in providers that are organized as cooperatives and care about environmental issues.

Do I need to mention free software? An extra plus would be a healthy distaste for PHP.

some cooperative-style provider initiatives that I still need to evaluate:

Paul Tagliamonte: GeoParse update

I ve thrown a bit of work at the wall, and some of it stuck. I ve been working (slowly) on GeoParse, which is a (hopefully pythonic) library to clean, pamper, separate and really dig into fairly low-quality GPS data (like, say, from Google Latitude) Just the basics are in place now, but work s slowly progressing. Hopefully, I should have something workable in the short-term future.

Next.

Previous.