Search Results: "tar"

3 April 2013

Julien Danjou: Hy, Lisp in Python

I've meant to look at Hy since Paul Tagliamonte started to talk to me about it, but never took a chance until now. Yesterday, Paul indicated it was a good time for me to start looking at it, so I spent a few hours playing. But what's Hy? Python is very nice: it has a great community and a wide range of useful libraries. But let's face it, it misses a great language. Hy is an implementation of a Lisp on top of Python. Technically, Hy is built directly with a custom made parser (for now) which then translates expressions using the Python AST module to generate code, which is then run by Python. Therefore, it shares the same properties as Python, and is a Lisp-1 (i.e. with a single namespace for symbols and functions). If you're interested to listen Paul talking about Hy during last PyCon US, I recommend watching his lightning talk. As the name implies, it's only a few minutes long. <iframe allowfullscreen="allowfullscreen" class="shadow" frameborder="0" height="374" src="http://www.youtube.com/embed/1vui-LupKJI?wmode=transparent&amp;autohide=1&amp;egm=0&amp;hd=1&amp;iv_load_policy=3&amp;modestbranding=1&amp;rel=0&amp;showinfo=0&amp;showsearch=0#t=16m14s" width="500"></iframe> Does it work? I've been cloning the code and played around a bit with Hy. And to my greatest surprise and pleasure, it works quite well. You can imagine writing Python from there easily. Part of the syntax smells like Clojure's, which looks like a good thing since they're playing in the same area. You can try a Hy REPL in your Web browser if you want. Here's what some code look like:
(import requests)

(setv req (requests.get "http://hy.pault.ag"))
(if (= req.status_code 200)
(for (kv (.iteritems req.headers))
(print kv))
(throw (Exception "Wrong status code")))

This code would ouput:
('date', 'Wed, 03 Apr 2013 12:09:23 GMT')
('connection', 'keep-alive')
('content-encoding', 'gzip')
('transfer-encoding', 'chunked')
('content-type', 'text/html; charset=utf-8')
('server', 'nginx/1.2.6')
As you can see, it's really simple to write Lispy code that really uses Python idioms. There's obviously still a lots of missing features in Hy. The language if far from complete and many parts are moving, but it's really promising, and Paul's doing a great job implementing every idea. I actually started to hack a bit on Hy, and will try to continue to do so, since I'm really eager to learn a bit more about both Lisp and Python internals in the process. I've already send a few patches on small bugs I've encountered, and proposed a few ideas. It's really exciting to be able to influence early a language design that I'll love to use! Being a recent fan of Common Lisp, I tend to grab the good stuff from it to add them into Hy.

Joey Hess: upstream git repositories

Daniel Pocock posted The multiple repository conundrum in Linux packaging. While a generally good and useful post, which upstream developers will find helpful to understand how Debian packages their software, it contains this statement:
If it is the first download, the maintainer creates a new git repository. If it has been packaged before, he clones the repository. The important point here is that this is not the upstream repository, it is an independent repository for Debian packaging.
The only thing important about that point is that it highlights an unnecessary disconnect between the Debian developer and upstream development. One which upstream will surely find annoying and should certainly not be bothered with. There is absolutely no technical reason to not use the upstream git repository as the basis for the git repository used in Debian packaging. I would never package software maintained in a git repository upstream and not do so. The details are as follows: So using the upstream git repository is valuable. And there is absolutely no technical value, and plenty of potential friction in maintaining a history-disconnected git repository for Debian packaging.

Daniel Pocock: The multiple repository conundrum in Linux packaging

I'm involved with a number of free software projects as both a developer and as the maintainer of packages for various distributions such as Debian (which also feeds packages to Ubuntu) and OpenCSW. I regularly come across the following situations:
  • Developers of great software who would like to see it packaged, distributed and promoted conveniently through platforms like Debian
  • Users of Linux distributions who are keen to use free software if it is presented in a convenient and accessible manner they are familiar with.
Sadly, despite everybody having the best intentions, there is sometimes a chasm separating these two groups of people. Upstream developers are often busy developing new features and don't have time to work on the intricacies of packaging. I hope that by sharing a few of my own experiences I can help more developers get their software packaged more easily. Fortunately, a number of great tools like git-buildpackage have emerged for streamlining the packaging process, but this has also created more confusion for developers who have their own git repositories and don't quite understand how the Debian git repository and patching process relates to their own repository. The autotools world Here, I focus on autotools based software, because this type of software has it's own peculiar issues when packaging. In particular, these issues appear when using a version control system to track upstream releases. Some of the concepts can be applied to the study of regular Makefile or cmake projects as well. Here is a diagram giving an overview: Let's work through each of the steps in the diagram: The upstream release
  1. The developer/release manager updates the version number in
    configure.ac
    (sometimes called configure.in) and tags the code. (Usually this tag is on a dedicated release branch.)
  2. The developer checks out a copy of the code from the tag into a fresh working directory
  3. The developer runs the autoreconf/automake tools, usually from a bootstrap script. These tools create a number of new files that don't exist in the project repository. Finally, the developer runs
    make dist
    , which puts all the files,
    including the generated files, into a distribution tarball. It is worth emphasizing this point: the tarball is not just an archive of the files from the repository/tag, it also contains a number of
    files generated by autotools.
  4. The developer uploads the tarball to a web site such as the Sourceforge download page. Usually a release announcement is made now containing checksums for the tarball.
At this point, the upstream developer's work is done and packaging teams from various projects such as Debian will take over. Sometimes, the upstream developer is also building the packages and continues
onto the next steps himself: Packaging
  1. The package is downloaded by the package maintainer
  2. If it is the first download, the maintainer creates a new git repository. If it has been packaged before, he clones the repository. The important point here is that this is not the upstream repository, it is an independent repository for Debian packaging. The maintainer uses the git-import-orig tool to import the upstream tarball into the packaging repository. The git-import-orig tool captures an exact snapshot of the upstream release tarball contents in a branch called upstream. One point where the Debian repository differs fundamentally from the upstream repository is that all files from the tarball will be tracked in the Debian git repository, even those automatically generated files that were created by autotools and don't exist in the upstream repository.
  3. The maintainer creates or updates the various artifacts for packaging. These files are kept on the master branch, and the tarball contents from the upstream branch are merged into master to create packages.
  4. When the maintainer feels the code is ready, he will check out a clean copy of the repository to build the package from.
  5. The maintainer executes a tool such as git-buildpackage or regular dpkg-buildpackage, which creates the *.deb files.
  6. The files are checked with a tool like lintian and some manual testing/installation. If all is OK, a tag is made in the packaging repository, with a suffix appended to the upstream version number to indicate which iteration of the package it applies to.
AttachmentSize
release-packaging-workflow.dia3.97 KB

Peter Eisentraut: Installing multiple PostgreSQL versions on Homebrew

I was going to post this yesterday, but some might have thought that it was a joke. April 1st is always an annoying day to communicate real information. If you have been fond of the way Debian and Ubuntu manage multiple PostgreSQL versions in parallel, you can now have the same on OS X with Homebrew:
brew tap petere/postgresql
brew install postgresql-9.2
# etc.
brew install --HEAD postgresql-common
postgresql-common is the same code as in Debian, only mangled a little. Now you have all the client programs symlinked through pg_wrapper, and you can use the server management tools such as:
pg_createcluster 9.2 main
pg_ctlcluster 9.2 main start
pg_lsclusters
Let me know if you find this useful. Links:

Paul Tagliamonte: Here s my talk from PyCon 2013 (I tried to queue it up to...

<iframe allowfullscreen="allowfullscreen" frameborder="0" height="299" src="http://www.youtube.com/embed/1vui-LupKJI?wmode=transparent&amp;autohide=1&amp;egm=0&amp;hd=1&amp;iv_load_policy=3&amp;modestbranding=1&amp;rel=0&amp;showinfo=0&amp;showsearch=0#t=16m14s" width="400"></iframe>

Here s my talk from PyCon 2013 (I tried to queue it up to where I start (@ 16:14)

A bit more giggley then I intended, but apparently it s endearing.

If anyone s actually interested in Hy, check out #hy on irc.freenode.net, star the code @ git.io/hy - it s actually pretty feature complete. If you want to play with it, check out the magic repl I ve been pondering taking it a bit more serious, so there is even a bit of documentation thanks to Chris Webber (hey, thanks!). Rock on!

2 April 2013

Jon Dowland: GSettings

With GNOME 3, if you want to override a setting with a new site default, assuming your app has been updated to use the new gsettings API and doesn't still use the old gconf one, the best guide on what to do is still Ross Burton's blog post from two years ago, although you can find most of what you need to know buried deep in the GSettings C API specification, which is not the first place I would have looked (lesson learned!) Note: In this context I'm in the role of systems administrator, rather than distributor/vendor or software author. Whereas you are generally advised to use gsettings itself and not poke at the backend (dconf for Linux), the documentation seems to point sysadmins towards dconf hackery rather than gsettings. However gsettings worked for me, and I got nowhere with dconf direct. Some caveats worth knowing about: Today I had to create 40 guest accounts on our 70-seat Linux cluster, including randomly generate passwords, set up persistent storage on our NFS server, etc.; quickly. As a nice-to-have I also wanted to add some application icons to the GNOME 3 "Favourite Apps" menu for the guests. Would you like to guess which of those jobs was the easier of the two?

Hideki Yamane: re-compress debian.tar. gz.bz2

You can use bzip2 to compress debian.tar source file in debian/source/options. But usually xz is more efficient than bzip2 (in my experience, and I love xz :) So, how about re-compressing those files with xz?

This is the result.

2.1G tar
503M current
414M xz_6e
410M xz_9e

non-compressed file is 2.1GB and we've compressed to 0.5GB. If we would use xz, it becomes 0.4GB.

debian.tar.bz2 files are 6.3M, if we would use gz, it becomes 40M, it means that package maintainers' choice is right.

6.3M bz2
40M gz
5.8M xz

Russ Allbery: Review: An Election

Review: An Election, by John Scalzi
Publisher: Subterranean
Copyright: 2010
Printing: 2012
ISBN: 1-59606-496-X
Format: Kindle
Pages: 24
This is another story from the Subterranean Scalzi Super Bundle that probably doesn't justify a full review, particularly since this time it's a short story and not even a novelette, but as with the others it's been independently published on the Kindle (and is still available that way for 99 cents). So by my obsessive internal consistency that matters only to me, it gets a full review anyway. I was mildly surprised to find that this was actually my second reading, and now vaguely remember that Subterranean had actually sponsored the story's appearance on Scalzi's blog as an advertising experiment. So I originally read the story from Scalzi's blog, and if you don't really want to pay money for it, you still can as well. Along with a couple of ads that appear to have bit-rotted. The chief feature of An Election is the background: a world in which aliens and humans are intermingled in a political district, going about their lives against a background of rather extreme multispecies diversity. The protagonist, David, decides to run for the upcoming city council special election, despite living in a district that's minority-human and that hasn't seen a human councilor in forty-four years. (David also happens to be a gay man, which goes entirely unremarked in the story and is, in fact, entirely unremarkable given the excitingly diverse characteristics of his alien neighbors. I thought that was a nice touch.) He of course appears to stand no chance, particularly when one of the city bosses decides to run as well, but ends up with a campaign advisor almost by accident and determinedly attempts to connect with his neighbors and earn their votes. This felt quite typical to me of Scalzi's humor writing. The protagonist is a good-hearted person who gets in rather over his head but is still determined to do the right thing, while not taking anything too seriously. The background humor is never particularly threatening and tends towards slapstick and banter. I found it similar in tone to most of Agent to the Stars, although of course much shorter. As usual, Scalzi's grasp of conversational banter is probably the strongest part of the story; the dynamic between David, his husband James, and his campaign advisor brought several chuckles for me. This is slight but enjoyable. I wouldn't go out of my way to read it, but it's a good story to toss into a bundle of other work, and I remember it holding my interest when reading it on-line. (Which is rare; usually I fail to finish any fiction I try to read in a browser unless it's extremely short.) Rating: 6 out of 10

1 April 2013

Andrew Pollock: [life/repatexpat] On Queensland electricity retailers

So one of the first things I need to do when I get my apartment today is get the electricity on. I've actually already ordered ADSL, which shows you what I think is more important, but without electricity, there is no Internet... It would appear that in 2007, Queensland opened up its electricity and gas markets to "full retail competition". According to Energex, they do the generation, distribution, and the retailer handles connections, disconnections, billing and green energy. I still need to deal directly with Energex for power outages, reporting faulty street lights, requesting a tree that's near power lines be trimmed, and so on. So if the retailers have no say in what price they're paying for the electricity that's being generated by Energex, I'm struggling to see what their point is. I guess they get to differentiate on customer service, but really, that's it? It was also really hard to find a canonical list of retailers to choose from. I would have thought that'd be linked off Energex's "Choosing your electricity retailer" page, but no, it's buried in the FAQ. Now it comes down to a case of doing a comparison between 11 retailers and trying to choose one. Or just going with the first one on the list. It's just electricity, people. It's a utility. I do not want to expend as much time on choosing an electricity retailer as I would my ISP (interestingly, it looks like Dodo has gotten in on the electricity retailing act). But one could say that an upside of having jet lag and being awake since 3am, is that one has time to comparison shop the 11 electricity retailers, except I won't. I'll just write this blog post instead. I have heard of one horror story, where a homeowner returned to living in his house, and had a nightmare time with one retailer because the previous occupants had an outstanding debt with that retailer, and he was trying to get a connection going with a different retailer, and lots of hilarity ensued. Except it wasn't hilarious. So I'll have to keep an eye on that. My current thought is to go with AGL, because What's pretty crazy is that at no point from quickly skimming AGL's landing page for Queensland pricing is there any indication of kWh or an actual price for anything. There's lots of noise about discounts, and flexibility, and other nonsense, but it seems like the actual cost of energy is so buried, it's incredible. It seems to be all about locking in on a contract, which is somewhat amazing. This is just electricity, but they seem to have turned this into a mobile phone plan type of situation. Amazing. After more digging, it looks like I can pay 5.5 cents per kWh for 100% green energy. Maybe.

Lisandro Dami&aacute;n Nicanor P&eacute;rez Meyer: On the road to Qt 5: Qt 4.8.4 and qtchooser uploaded to experimental

I have just uploaded Qt 4.8.4 to experimental. Apart from being the newest upstream release of the 4 series, it adds the basic support for coexisting with Qt 5. Because of this support it will need to go trough the NEW queue though, so we will need to be patient.

Developers will be interested in qtchooser, the tool that allows to switch between Qt 4 and 5 development apps. It has also been uploaded to experimental.

Of course, all this has been possible thanks to the efforts of the wonderful Qt/KDE team =)

Update: our efficacious FTP masters have already made Qt pass the NEW queue. Thanks a lot!

Andrew Pollock: [life/repatexpat] Day #1 of repatriation

Well, we're back in Brisbane, for good. Now starts the long and involved process of bootstrapping a new life for myself. Fortunately it's slightly more familiar territory than starting up in the US was. The tenants in my Hawthorne apartment have agreed to terminate their lease early, and I should get the keys tomorrow. The lease on my Woolloongabba apartment ends on Friday I think, and all of our stuff is scheduled to be delivered there next Monday. My primary goal is to get my apartment habitable for Zoe and I as soon as possible. That means beds and a fridge, and Zoe's going to need some way of being kept entertained while I'm running around like a chook with its head cut off, which is most likely going to mean a TV and a DVD player. This week is going to be spent doing a lot of running around. I need to get a car, get a Queensland driver's licence, get the electricity on, get the ADSL on, buy appliances, furnish Zoe's room, furnish my room, heck, furnish the whole apartment, really. It's going to be very full on busy. I don't want to race out and buy anything today, even though the shops seem to be open, because I want to measure up my apartment first. The day's sole purchase was an electric screwdriver from Bunnings. There will be much Ikea assembly in my future.

Russ Allbery: Review: Echo

Review: Echo, by Jack McDevitt
Series: Alex Benedict #5
Publisher: Ace
Copyright: November 2010
Printing: November 2011
ISBN: 1-937007-00-6
Format: Mass market
Pages: 367
This is the fifth book in the ongoing Alex Benedict series, but reading the previous volumes is generally optional. While there are references to earlier events (and indeed I resorted to Wikipedia at one point to refresh my memory of the plot of the the first book in the series), they aren't critical to the plot. They mostly are background for the reactions of other characters to Benedict and his firm. Echo opens with the now-familiar prologue, giving us a fleeting and limited glimpse into the mystery that Benedict and Chase Kolpath will be pursuing for the rest of the book. This time, it features the explorer Somerest Tuttle, who has spent most of his life looking for intelligent alien life. (In the far future setting of these books, humanity has spread through a large region of the galaxy but has only encountered one other alien race.) In the twilight of his career and life, a friend who works as a tour guide for astronomical tours comes to him with a story about something she saw. What, we don't know, and won't for most of the book. But, whatever it is, Tuttle never goes public with it, which is directly contrary to what everyone would expect him to do with any news about alien life. Benedict and Kolpath get involved some years later, after Tuttle's death, when a stone slab with unknown markings that was part of his estate shows up on the future equivalent of Craigslist, free to the first person who will haul it away. Benedict finds that interesting enough to go pick up, but then is really hooked when someone else goes to considerable lengths to try to make the slab disappear before anyone can get a good look at it. This, slowly, leads into poking around Tuttle's life and connections, the star tour industry, and the possible origins of the slab. If you've read any of the previous books in this series, Echo is more of the same, and I mean that in a good way. I think McDevitt is at his best when writing this sort of cross between scientific puzzle, mystery, and non-military adventure, full of people who care deeply about knowing and understanding things and don't like to be thwarted in their efforts. There is McDevitt's trademark halting and red-herring-ridden investigation that succeeds in being engrossing and compelling, plenty of opportunities to guess what's really going on (a bit easier this book than in some others), and a bit of action that I thought was better-handled than the somewhat strained action scenes of The Devil's Eye. The book moves right along, despite multiple reversals in the progress of the investigation, and the final revelation does indeed justify the strange behavior of the various people Benedict encounters. As with McDevitt in general and this series in particular, there is one caveat: the astronomical details are often interesting, but the world building in general is not terribly believable. Despite supposedly taking place millennia into the future, the society is a straight transplant (projection would imply more change than exists) of middle-class American culture, with bonus air cars, space ships, and slightly better computers. Similarly, alien ecologies suffer from severe Star Trek problems: all the planets feel like copies of Earth with a few minor tweaks. This just goes with the territory, and if you're up to the fifth book of the series, you've probably gotten used to it, but the unrealistic lack of divergence is actually used as a plot point here. That strained my suspension of disbelief a bit. That said, this sort of extremely familiar social context does give the books an oddly calm, comfortable feeling. It's tempting to describe McDevitt as cozy science fiction. There's a bit of danger, a bit of suspense, but one is fairly certain that everything will work out in the end and the reader will leave the book knowing the solutions to most of the mysteries raised without having to work very hard at understanding the society or the puzzle. If that's what you're in the mood for (and I was when I read Echo), McDevitt delivers reliably. If you like the Alex Benedict series in general, recommended, as this is an excellent example of the type. Rating: 8 out of 10

Francois Marier: Poor man's RAID1 between an SSD and a hard drive

After moving from a hard drive to an SSD on my work laptop, I decided to keep the hard drive spinning and use it as a backup for the SSD. With the following setup, I can pull the SSD out of my laptop and it should still boot up normally with all of my data on the hard drive.

Manually setting up an encrypted root partition Before setting up the synchronization between the two drives, I had to replicate the partition setup. I used fdisk, cfdisk and gparted to create the following partitions:
   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        2048      499711      248832   83  Linux
/dev/sdb2          501760   500117503   249807872    5  Extended
/dev/sdb5          503808   500117503   249806848   83  Linux
and then loosely followed these instructions to create an encrypted root partition on /dev/sdb5:
$ cryptsetup luksFormat /dev/sdb5
$ cryptsetup luksOpen /dev/sdb5 sdb5_crypt
$ pvcreate /dev/mapper/sdb5_crypt
$ vgcreate akranes2 /dev/mapper/sdb5_crypt
$ vgchange -a y akranes2
$ lvcreate -L247329718272B -nroot akranes2
$ lvcreate -L8468299776B -nswap_1 akranes2
$ mkfs.ext4 /dev/akranes2/root
Finally, I added the new encrypted partition to the list of drives to bring up at boot time by looking up its UUID:
$ cryptsetup luksUUID sdb5_crypt
and creating a new entry for it in /etc/crypttab.

Copying the boot partition Setting up the boot partition was much easier because it's not encrypted. All that was needed was to format it and then copy the files over:
$ mkfs.ext2 /dev/sdb1
$ mount /dev/sdb1 /mnt/boot
$ cp -a /boot/* /mnt/boot/
The only other thing to remember is to install grub on the boot loader of that drive. On modern Debian systems, that's usually just a matter of running dpkg-reconfigure grub-pc and adding the second drive (/dev/sdb in my case) to the list of drives to install grub on.

Sync scripts To keep the contents of the SSD and the hard drive in sync, I set up a regular rsync of the root and boot partitions using the following mount points (as defined in /etc/fstab):
/dev/mapper/akranes-root /           ext4    noatime,discard,errors=remount-ro 0       1
/dev/mapper/akranes2-root /mnt/root  ext4    noatime,errors=remount-ro         0       2
UUID=0b9109d0-... /boot              ext2    defaults                          0       2
UUID=6e6f05fb-... /mnt/boot          ext2    defaults                          0       2
I use this script (/usr/local/sbin/ssd_boot_backup) for syncing the boot partition:
#!/bin/sh
if [ ! -e /mnt/boot/hdd.mounted ] ; then
    echo "The rotating hard drive is not mounted in /mnt/boot."
    exit 1
fi
if [ ! -e /boot/ssd.mounted ] ; then
    echo "The ssd is not the boot partition"
    exit 1
fi
nice ionice -c3 rsync -aHx --delete --exclude=/ssd.mounted --exclude=/lost+found/* /boot/* /mnt/boot/
and a similar one (/usr/local/sbin/ssd_root_backup) for the root partition:
#!/bin/sh
if [ ! -e /mnt/root/hdd.mounted ] ; then
    echo "The rotating hard drive is not mounted in /mnt/root."
    exit 1
fi
if [ ! -e /ssd.mounted ] ; then
    echo "The ssd is not the root partition"
    exit 1
fi
nice ionice -c3 rsync -aHx --delete --exclude=/dev/* --exclude=/proc/* --exclude=/sys/* --exclude=/tmp/* --exclude=/boot/* --exclude=/mnt/* --exclude=/lost+found/* --exclude=/media/* --exclude=/var/tmp/* --exclude=/ssd.mounted --exclude=/var/lib/lightdm/.gvfs --exclude=/home/francois/.gvfs /* /mnt/root/
To ensure that each drive is properly mounted before the scripts run, I created empty ssd.mounted files in the root directory of each of the partitions on the SSD, and empty hdd.mounted files in the root directory of the hard drive partitions.

Cron jobs The sync scripts are run every couple of hours through this crontab:
10 */4 * * *                root    /usr/local/sbin/ssd_boot_backup
20 0,4,8,12,16,20 * * *     root    /usr/local/sbin/ssd_root_backup
20 2,6,10,14,18,22 * * *    root    /usr/bin/on_ac_power && /usr/local/sbin/ssd_root_backup
which includes a reduced frequency while running on battery to avoid spinning the hard drive up too much.

31 March 2013

Enrico Zini: A proposal to solve gender imbalance in Debian

A proposal to solve gender imbalance in Debian We've done all we can so far: Debian Women, the Diversity Statement, the anti-harassment contact, gender neutral language, lots of education all round, but we still suffer from a strong gender imbalance in Debian. I think that the reason is that the majority group of cisgender men in the project, although they don't actively work /against/ the rest, still have /no incentive/ to be inclusive, and generally do not understand what bearing a female name online is like. I think it is about time we addressed that, and after a lot of thinking and discussing with many other concerned debianers, I think I have just the right proposal, which is twofold. The first part is this: since the goal is to have an equal gender perception in Debian, we can just decide to only approve one obviously-male-named DD for every obviously-female-named one. That's right: no new obviously-male-named DD unless an obviously-female-named DD has just been approved. It may sound like affirmative action gone wild, but please stop a moment to think about it: this would create precisely the right incentive for the currently dominant group of developers to be inclusive! People shouldn't just assume they can get a Debian account regardless of what happens around them. We already ask NM candidates to fix RC bugs and, well, gender imbalance should be treated as an RC bug, and everyone should feel compelled to join the effort to fix it! Now, of course since there currently are many male names in NM but not a single female name, it would not be reasonable to just stop the flow of new developers into the project: that would just have the effect to make us starve on personpower. So here is the second part of the proposal: the one-female-name-one-male-name policy will not be enforced for, say, a year. But during this year, everyone currently in NM or joining NM will be asked to adopt a female identity. Crazy? No, genius! It's about time people understood what it means to get advances in private every time they make a public contribution! What better way than just trying it out for themselves? On top of that, as more and more female names appear in Debian changelogs, fake or not, people will finally start to understand that it does not matter what name is used to sign the contribution, but the contribution itself. I don't know if this will give us a community where people realise female or male contributions are equally valuable, which is what I hope, or a community where people will think that everyone is a cisgender man even if they have female names. In the end, really, it does not matter. Either way, we finally get to have a community where everyone is /guaranteed/ to be treated the same. But gender imbalance isn't the only imbalance we have in Debian. People accrue a reputation over time, good or bad, and this reputation tends to stick on you for years, regardless of how you may change, for better or for worse. When we evaluate the merit a contribution, we should not be biased by the reputation of the contributor! How can new contributors be taken seriously otherwise? I believe we are loosing lots of fresh, good ideas this way. And how much damage could be wrecked on the project by a well-respected contributor, like a Debian Account Manager, who is having a funny day? I think we can address this just as I propose to address gender imbalance: let's swap identities from time to time, like it usually happens with nametags at the end of Debconfs. Let's see gregoa upload a patched versions of python3.2, and enrico upload a new upstream version of eglibc! See if we won't finally have some peer review at last! Reputation and real identities have many merits, but we have come to rely too much on them, and it is hurting us. It is time we did something about it, before it is too late!

30 March 2013

Joey Hess: Goodreads vs LibraryThing vs Free software

Four years ago I started using Goodreads to maintain the list of books I've read (which had lived in a flat text file for a decade+ before that). Now it's been aquired by Amazon. I doubt it will survive in its current form for more than 2 years. Anyway, while Goodreads has been a quite good way to find what my friends are reading, I've been increasingly annoyed by the quality of its recommendations, and its paucity of other features I need. It really doesn't seem to help me keep up with new and interesting fiction at all, unless my friends happen to read it. So I looked at LibraryThing. Actually, I seem to have looked at it several times before, since it had accounts named "joey", "joeyh", and "joeyhess" that were all mine. Which is what happens to me on sites that lack Openid or Browserid. Digging a little deeper this time, I am finding its recommendations much better than Goodreads' -- although it seems to sometimes recommend books I've already read. And it has some nice features like tracking series, so you can easily tell when you've read all the books in a series or not. The analytics overall seem quite impressive. The UI is cluttered and it seems to take 5 clicks to add and rate a single book. It supports half stars. Overall I get the feeling this was designed for a set of needs that doesn't quite match mine. For example, it seems it doesn't have a single database entry per book; instead each time I add a book, it seems to pull in data from primary sources (library of congress, Amazon cough) and treat this as a separate (but related) entry somehow. Weird. Perhaps this makes sense to say, librarians. I'm willing to adjust how I think about things if there's an underlying reason that can be grasped. There's a quite interesting thread on LibraryThing where the founder says:
Don't say we should open-source the code. That would be a nightmare! And I have limited confidence in APIs. LibraryThing has the book geeks, but not so much the computers geeks.
I assume that the nightmare is that there would be dozens of clones of the site, all balkanized, with no data transfer, no federation between them. Except, that's the current situation, as every Goodreads user who is now trying to use LibraryThing is discovering. Before I ever started using Goodreads, I made sure it met my minimum criteria for putting my data into a proprietary silo: That I could get the data back out. I can, and have. LibraryThing can import it. But the import process loses data! And it's majorly clunky. If I want to continue using Goodreads due to its better UI, and get the data into LibraryThing, for its better analytics, I have to do periodic dumps and loads of CSV files with manual fixups. This is why we have standards. This is why we're building federated social networks like status.net and the upcoming pump.io that can pass structured data between nodes transparently. It doesn't have to be a nightmare. It doesn't have to rely on proprietary APIs. We have the computer geeks. Thing is, sites like GoodReads and LibraryThing need domain-specific knowledge, and communities to curate data, and stuff like that. Things that work well in a smallish company. (LibraryThing even has a business model that makes sense, yearly payments to store more books in it.) With free software, it's much more appealing to sink the time we have into the most general-purpose solution we can. Why build a LibraryThing when we could build something that tracks not only books but movies and music? Why build that when we could build a generic federated network for structured social data? And that's great, as infrastructure, but if that infrastructure is only used to build a succession of proprietary data silos, what was the point? So, could some computer & book geeks please build a free software alternative to these things, focused on books, that federates using any of the fine APIs we have available? Bear in mind that there is already a nice start at a comprehensive collection of book data in the Open Library. I'd happily contribute to a crowd funded project doing this.

Evgeni Golov: Opera, standards and why I should have stayed in my cave

So you probably heard that I have that little new project of mine: QiFi the pure JavaScript WiFi QR Code Generator. It s been running pretty well and people even seem to like it. One of its (unannounced) features is a pretty clean stylesheet that is used for printing. When you print the result will be just the SSID and the QR code, so you can put that piece of paper everywhere you like. That works (I tested!) fine on Iceweasel/Firefox 10.0.12 and Chromium 25.0. Today I tried to do the same in Opera 12.14 and it failed terribly: the SSID was there, the QR code not. And here my journey begins First I suspected the CSS I used was fishy, so I kicked all the CSS involved and retried: still no QR code in the print-out. So maybe it s the QR code library I use that produces a weird canvas? Nope, the examples on http://diveintohtml5.info/canvas.html and http://devfiles.myopera.com/articles/649/example5.html don t print either. Uhm, let s Google for opera canvas print And oh boy I should not have done that. It seems it s a bug in Opera. And the proposed solution is to use canvas.toDataURL() to render the canvas as an image and load the image instead of the canvas. I almost went that way. But I felt that urge need to read the docs before. So I opened http://www.w3.org/html/wg/drafts/html/master/embedded-content-0.html#dom-canvas-todataurl and https://developer.mozilla.org/en-US/docs/DOM/HTMLCanvasElement and started puking:
When trying to use types other than image/png , authors can check if the image was really returned in the requested format by checking to see if the returned string starts with one of the exact strings data:image/png, or data:image/png; . If it does, the image is PNG, and thus the requested type was not supported. (The one exception to this is if the canvas has either no height or no width, in which case the result might simply be data:, .)
If the type requested is not image/png, and the returned value starts with data:image/png, then the requested type is not supported.
Really? I have to check the returned STRING to know if there was an error? Go home HTML5, you re drunk! Okay, okay. No canvas rendered to images then. Let s just render the QR code as a <table> instead of a <canvas> when the browser looks like Opera. There is nothing one could do wrong with tables, right? But let s test with the basic example first: Yes, this is 2013. Yes, this is Opera 12.14. Yes, the rendering of a fucking HTML table is wrong. Needles to say, Iceweasel and Chromium render the example just fine. I bet even a recent Internet Explorer would That said, there is no bugfixworkaround for Opera I want to implement. If you use Opera, I feel sorry for you. But that s all. Update: before someone cries ZOMG! BUG PLZ!!! , I filled this as DSK-383716 at Opera.

Steve Kemp: Time passes, Thorin sits down and starts singing about gold.

This weekend I have mostly been reading Longitude: The True Story of a Lone Genius Who Solved the Greatest Scientific Problem of His Time . In modern times we divide the earth up into rings of lines, latitude and longitude, as wikipedia will explain. Finding your latitude is easy, finding your longitude is a difficult process, and it was vitaly important when people started to sail large distances, the book contained lots of stories of sailors being suddenly suprised by the appearance of land - because they'd misjudged their position. Having four ships, containing garlic, pepper, and other goods of value exceeding the total wealth of the UK, sink all at once was a major blow. Not to mention the large number of sailors who lost their lives. There were several solutions proposed, involving steady hands and telescopes, etc, but the book mostly discusses John Harrison and his use of watches/clocks.
John Harrison was featured in Only Fools & Horses, as the designer of the watch that made Delboy & Rodney millionaires. ->Time on our hands
The idea of using a clock is that you take one with you, set to the time of your departure location. Using that clock you can compare the time to the local-time, by viewing the sun, etc. Calculating the difference between the two times allows you to see how far away, in degrees, from your port, and thus how far you've traveled. Until harrison came along clocks weren't accurate enough to keep time. His clocks would lose a second a month, until then clocks might lose 15 minutes a day. (With more variations depending on temperture, location, and pressure. Clearly things like pendulum clocks weren't suitable for rocking ships either.) All in all this book was a great read, there were mentions of Galilao, Newton, and similar folk who we've all heard of. There was angst, drama, deceit, and some stunning craftmanship. Harrison was a woodworker, and he made his clocks out of wood (+brass where necessary). Choosing fast/slow-grown wood depending on purpose, and using wood that secreted oils naturally allowed him to avoid lubrication - which improved accuracy, as lubricants tend to thin/thicken when temperature/pressure change. A lovely read, thank you very much. In other news I received several patches for my templer static-site generator, and this has resulted in much improvement. I've also started using Test::Exception now, and slowly updating all my perl code to use this.

Paul Tagliamonte: Debian Stable (Or: When to switch)

I ve noticed something funky when looking over the Release Critical page on the BTS Look! There s a cross-over between the Stable bug count and the Testing bug count around 6/2012. Since then, Stable has actually been more buggy the testing. What should we (as a project) do? Why is stable so unloved during the freeze? Do we have too long a freeze? Hilariously, this makes me think I should be suggesting users to switch from stable to testing when the crossover happens during the release cycle. Has there been previous discussion of this?

29 March 2013

Olivier Berger: Managing Python code with UTF-8 (french chars) in org-mode + babel + minted for LaTeX export

The goal of this article is to illustrate how to manage Python code which includes comments in UTF-8 characters inside a latin-1 source org-mode for LaTeX export. Note that I ve pasted in wordpress the HTML generated by org-mode, so I hope it isn t broken too much. My typical use case is a french lecture on Python where the text is written in french, as well as some of the code comments and examples We ll use org-mode s babel module to include and manage the Python
examples. The goal is to write the source of the Python programs
directly in the same org source as the class book s text, and to extract them into a subdir (with the tangle feature), so that they can be shipped to the students to experiment with. The minted LaTeX environment is used, for babel, to make the Python syntax highlighting.

1.1 Issues
  • The source org-mode is in latin-1 so that it compiles with pdflatex
  • The examples source-code will be in UTF-8 so that Python 2.7 executes them well on a modern Linux desktop
  • Minted relies on pygmentize wich doesn t seem to handle UTF-8 so well

1.2 Solution
We ll show how to patch minted s use of pygmentize to take advantage
of its conversion capacity to convert the Python sources from UTF-8 to
latin-1 at document rendering time (LaTeX compilation). It s a hack, but works. The current document s source shows how this works

1.2.1 Customization
The org document should contain the following headers :
#+LANGUAGE:  fr
#+LaTeX_HEADER: \usepackage[latin1] inputenc 
#+LaTeX_HEADER: \usepackage[french] babel 
#+LaTeX_HEADER: \usepackage color \usepackage minted 
and the footers :
# Local Variables:
# coding: latin-1
# org-src-preserve-indentation: true
# tab-width: 4
# End:
Also note that pdflatex should use the -shell-escape option necessary for minted Now, the document should also include the following patch for minted :
#+LATEX_HEADER: \makeatletter
#+LATEX_HEADER: \renewcommand\minted@pygmentize[2][\jobname.pyg] 
#+LATEX_HEADER:   \def\minted@cmd pygmentize -l #2 -f latex -F tokenmerge
#+LATEX_HEADER:     \minted@opt gobble  \minted@opt texcl  \minted@opt mathescape 
#+LATEX_HEADER:     \minted@opt startinline  \minted@opt funcnamehighlighting 
#+LATEX_HEADER:     \minted@opt linenos  -P "verboptions=\minted@opt extra "
#+LATEX_HEADER:     -O encoding=UTF-8,outencoding=iso-8859-1 -o \jobname.out.pyg #1 
#+LATEX_HEADER:   \immediate\write18 \minted@cmd 
#+LATEX_HEADER:   % For debugging, uncomment:
#+LATEX_HEADER:   %\immediate\typeout \minted@cmd 
#+LATEX_HEADER:   \ifthenelse \equal \minted@opt@bgcolor 
#+LATEX_HEADER:     
#+LATEX_HEADER:     \begin minted@colorbg \minted@opt@bgcolor 
#+LATEX_HEADER:   \input \jobname.out.pyg 
#+LATEX_HEADER:   \ifthenelse \equal \minted@opt@bgcolor 
#+LATEX_HEADER:     
#+LATEX_HEADER:     \end minted@colorbg 
#+LATEX_HEADER:   \DeleteFile \jobname.out.pyg 
#+LATEX_HEADER: \makeatother
The important change is the addition of the -O encoding=UTF-8,outencoding=iso-8859-1 option of pygmentize, that will convert the UTF-8 source code to latin-1. Hopefully most of the characters will convert fine, for a document written in french.

2 Examples

2.1 Basic Python
This is a generic verbatim example, which doesn t use babel / minted.
$ python
Python 2.7.3 (default, Jan  2 2013, 16:53:07) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> print "Hello, world!"
Hello, world!
>>>

2.2 Python source code for french speakers
Here, we first include the source code, so that it can be exported to
an examples/ subdir, with the following (note that the #+ BEGIN_src / #+ END_src should be changed to get rid of the space
character between + and BEGIN / END, and that the :shebang and following text should be on the BEGIN_SRC line I seem to have not quoted
these properly in this document):
#+name: helloworld
#+ BEGIN_src python :tangle examples/helloworld.py :noweb yes \\
:shebang #!/usr/bin/python :padline no :exports none
# -*- coding: utf-8 -*-
# Ceci est un exemple d'affichage d'une cha ne accentu e
name = raw_input("Quel est v tre nom ? ")
print "J'esp re que  a va bien aujourd'hui", name
#+ END_src
Editing it with C-c will open an UTF-8 buffer, so hopefully, this will be consistent with the utf-8 coding system declared on the second line. The regeneration of the examples/helloworld.py file is made with C-c C-v t (babel tangling ). Note that we don t add the #!/usr/bin/python first line in the source, as we want it to be added by the tangling process (which will also make the script executable). Once this is done, the script may be run :
$ python examples/helloworld.py 
Quel est v tre nom ? Fran ois
J'esp re que  a va bien aujourd'hui, cher Fran ois
Now, to add the colorized rendering in the lecture book, we do (here again, remove the spaces):
# +BEGIN_latex
\inputminted python examples/helloworld.py 
# +END_latex
And here it is, rendered by minted:
# -*- coding: utf-8 -*-
# Ceci est un exemple d'affichage d'une cha ne accentu e
name = raw_input("Quel est v tre nom ? ")
print "J'esp re que  a va bien aujourd'hui, cher", name
Here are links to : the source, the PDF result, and the generated Python program.

Hideki Yamane: build failure with certain cowbuilder

I've got a build failure with freewnn via cowbuilder. However, same source can be built with pbuilder. And strangely, cowbuilder in another host can build it.

  • host + pbuilder = fine
  • host + cowbuilder = fail
  • host(chroot) + cowbuilder = fine
  • host(old kernel) + cowbuilder = fine
  • laptop + pbuilder = fine
  • laptop + cowbuilder = fine
  • virtualbox + cowbuilder = fine
... it's strange. What is the root cause of this?

Next.

Previous.