Search Results: "rul"

7 March 2012

Michal Čihař: Contest for phpMyAdmin books

Packt Publishing has offered phpMyAdmin to sponsor some coding contest with books. The contest is already running since Monday and we have first winner, but still, your chances are pretty high. So in case you are interested in having Mastering phpMyAdmin 3.4 for Effective MySQL Management, go ahead and check out the contest rules.

Filed under: English Phpmyadmin 0 comments Flattr this!

6 March 2012

Ian Wienand: Investigating the Python bound method

You work with Python for a while and you'll become familiar with printing a method and getting
<bound method Foo.function of <__main__.Foo instance at 0xb736960c>>
I think there is room for one more explanation on the internet, since I've never seen it diagrammed out (maybe for good reason!). An illustration of Python bound methods In the above diagram on the left, we have the fairly simple conceptual model of a class with a function. One naturally tends to think of the function as a part of the class and your instance calls into that function. This is conceptually correct, but a little abstracted from what's actually happening. The right attempts to illustrate the underlying process in some more depth. The first step, on the top right, is building something like the following:
class Foo():
      def function(self):
      	  print "hi!"
As this illustrates, the above code results in two things happening; firstly a function object for function is created and secondly the __dict__ attribute of the class is given a key function that points to this function object. Now the thing about this function object is that it implements the descriptor protocol. In short, if an object implements a __get__ function; then when that object is accessed as an attribute of an object the __get__ function is called. You can read up on the descriptor protocol, but the important part to remember is that it passes in the context from which it is called; that is the object that is calling the function. So, for example, when we then do the following:
f = Foo()
f.function()
what happens is that we get the attribute function of f and then call it. f above doesn't actually know anything about function as such what it does know is its class inheritance and so Python goes searching the parent's class __dict__ to try and find the function attribute. It finds this, and as per the descriptor protocol when the attribute is accessed it calls the __get__ function of the underlying function object. What happens now is that the function's __get__ method returns essentially a wrapper object that stores the information to bind the function to the object. This wrapper object is of type types.MethodType and you can see it stores some important attributes in the object im_func which is the function to call, and im_self which is the object who called it. Passing the object through to im_self is how function gets it's first self argument (the calling object). So when you print the value of f.function() you see it report itself as a bound method. So hopefully this illustrates that a bound method is a just a special object that knows how to call an underlying function with context about the object that's calling it. To try and make this a little more concrete, consider the following program:
import types
class Foo():
    def function(self):
        print "hi!"
f = Foo()
# this is a function object
print Foo.__dict__['function']
# this is a method as returned by
#   Foo.__dict__['function'].__get__()
print f.function
# we can check that this is an instance of MethodType
print type(f.function) == types.MethodType
# the im_func field of the MethodType is the underlying function
print f.function.im_func
print Foo.__dict__['function']
# these are the same object
print f.function.im_self
print f
Running this gives output something like
$ python ./foo.py
<function function at 0xb73540d4>
<bound method Foo.function of <__main__.Foo instance at 0xb736960c>>
True
<function function at 0xb73540d4>
<function function at 0xb73540d4>
<__main__.Foo instance at 0xb72c060c>
<__main__.Foo instance at 0xb72c060c>
To pull it apart; we can see that Foo.__dict__['function'] is a function object, but then f.function is a bound method. The bound method's im_func is the underlying function object, and the im_self is the object f: thus im_func(im_self) is calling function with the correct object as the first argument self. So the main point is to kind of shift thinking about a function as some particular intrinsic part of a class, but rather as a separate object abstracted from the class that gets bound into an instance as required. The class is in some ways a template and namespacing tool to allow you to find the right function objects; it doesn't actually implement the functions as such. There is plenty more information if you search for "descriptor protocol" and Python binding rules and lots of advanced tricks you can play. But hopefully this is a useful introduction to get an initial handle on what's going on!

5 March 2012

Benjamin Mako Hill: Unhappy Birthday Hall of Shame

/copyrighteous/images/no_happy_bday.png I roll my eyes a little when I think that Unhappy Birthday is the document I have written that has been read by the most people. The page -- basically a website encouraging people to rat on their friends for copyright violation for singing Happy Birthday in public -- has received millions of page views and has generated tons of its own media (including a rather memorable interview of CBC's WireTap). At the bottom of the page I am listed, by name and email, as the "copyrighteous spokesman" for the initiative. And since the page has been online, I have received hate mail about it. Constantly. Since the email only goes to me, I thought it might be fun to share some of these publicly. All these messages are quoted verbatim but I have not included the senders' names. Be warned: the language is often salty. This email is years old now but it is probably still my favorite:
Atrocity and strife run rampant in this world. Babies are abandoned in dumpsters. Teachers molest students. Impoverished Indonesians make sneakers for pennies while the spoiled jackhole in the 30-second commercial makes millions for sinking a three-pointer and smirking at the camera. Forms of religion are interpreted as to compel people to strap explosives to their chest and board buses full of innocents. Boss Tweeds embezzle and get severance pay while John Q. Workingman gets put out on the street when the corporation goes belly up. Out of all these indignities and countless others I haven't the time to mention, why do you make it your personal crusade to assist in the flagrant persecution of family restaurants for partaking in the time-honored tradition of singing "Happy Birthday"? God forbid these foul brigands bend copyright law in order to bring a smile to somebody's face. Food for thought...without the accompanying song.
Many others strike a defiant, if less poetic, tone:
Good luck! There are millions of us who refuse to accept the ridiculous "copyright" on Happy Birthday. If Time Warner were an ethical company rather than a greedy megacorp they would do something truly special and release it into the public domain. There are some things in this world more important than money.
Quite a few people notice that my last name is Hill and suspect that I must be related to the Hill sisters who originally penned the song. I'm not, to my knowledge, although since Time Warner bought the rights, it's not clear it would matter:
I am writing to just let you know how disappointed I am that a large corporation and others (like the HILL family) are making $2 million plus for a song that was created over 100 years ago with noone knowing who created the lyrics! None of us at our place of employment could believe this and we certainly won't encourage people to send money to ASCAP. It is a shame that ASCAP license fees aren't used to pay more to up-and-coming artists who I'm sure need this money alot more than Time Warner. We all plan to sing Happy Birthday MORE now in public places and if anyone asks if it is copyrighted we will say "of course not". Maybe this way the song will not die out completely as more and more other "birthday" songs are being sung. It would also be nice if your website cited whose opinion is writing the piece and your obvious conflict of interest.
Or another:
Is it a coincidence that your last name is the same as the last name of the authors of the song "Happy Birthday?" You seem to have a personal monetary motive for your work with the "grassroots project" you call Unhappy Birthday, and if you do not, your concern is misplaced all the same. Whom do you imagine your campaign serves? And do you realize whom it harms? I do not question the illegality of performing the copyrighted song publicly. And you are correct that most of the public is not even aware that the song is under copyright. I think the harm done to Time Warner and its associates by such public performances is far outweighed by the joy created when the much-loved happy tune is shared. I urge you to ask yourself why you think the immortal Hill sisters wrote the song in the first place. It was not to put more money Time Warner's pocket. It was, I would argue, for the sake of the song itself and the happiness it brings when performed (publicly or otherwise). Please consider siding with the children and the artists; let the lawsuits alone.
Some people suspect the site may be satire, but include insults and and attacks just in case it isn't:
I'm trying to figure out if your Unhappy Birthday site is meant to be in jest. If so Rofl, and congrats on a hilarious site. If you're actually serious, then fuck you Nazi cunts and your corporate butt buddies. Thank you for your time.
Or these two alternatives (each were separate emails):
If this is a joke then it's rather funny. However if this website is serious then you're a fucking idiot. Get a life!!!! if it is a form of protest, then THANK YOU! if it is not, then screw you all!
One memorable piece of mail was from someone who knew of me from my activities in the free software and free culture communities and had a hard time reconciling my work there with the high protectionist website:
I was quiet surprised to see your name and email address at the bottom of the home page of the site Unhappy Birthday. The site claims that you are their spokesman. Is this correct? I do not understand... You have all this Open Source/ Free Software background and then this site that defends one of the most controversial copyright issues??? Do you really mean this? Do you want to help Time Warner?
I've also received probably half a dozen mails that offer some sort of support! For example, this person liked the website -- and even wanted to buy one of our t-shirts -- but objected to our logo:
I was going to buy one of your products from your Unhappy Birthday Shop at CafePress but there's a problem. I hate emblems that uses human skulls in them. Being a member of ASCAP I really do support your cause but I can't buy a product that I would never wear.
And many people are simply confused asking something like this one:
So I saw the unhappy birthday site and I'm just a little confused. Is this a joke or a serious thing?
I usually reply and explain that I have tried to ensure that the site describes the legal situation around Happy Birthday honestly and correctly. That said, the vast majority of messages I receive are unequivocal. Like this email that I received last week addressed to "you anti-free speech fascists":
         __
        /  \
            
            
            
 __  __      __
/  \/  \    /  \
                \
                  
                 /
                 \
                  /
\                /
                 
                 
Half an hour later, the author followed up with a English version of the same message, set to the tune of happy birthday. You might think that getting insulted and flipped off by confused people on the Internet might get me down. It doesn't! I made Unhappy Birthday because I thought that the fact that something as important to our culture as Happy Birthday could be owned was outrageous. Every piece of hate mail means that somebody else -- almost always somebody who isn't a "copyfighter" or a free culture geek -- is now upset about the current state of copyright too. Sure, Unhappy Birthday makes me a tiny bit sad about people's ability to recognize satire. But it makes me really happy about people's ability to get very annoyed at what they think is the outrageous control of our culture through copyright. When more people are as mad as the the people I've quoted above, we will be able to change copyright into something less outrageous to all of us.

4 March 2012

Jamie McClelland: Managing KVM instances

At May First/People Link we have been using KVM for several years now and recently I have been running KVM instances on my local laptop. I'm pleased to see all the work that has gone into libvirt, which seems like a robust and full-featured suite of tools for managing many virtualization technologies, including KVM. However, we don't use it at May First/People Link for a number of reasons. The most pressing is that it runs all virtual guests as the same user, but also because it offers far more features than we need (such as graphical access to virtual server, which we don't need since none of our guest servers run X). On May First/People Link hosts, we are using a relatively simple set of bash scripts (accessible via git at git://lair.fifthhorseman.net/~dkg/kvm-manager). These scripts re-use many tools we are already familiar with to build and launch kvm guests. Each guests runs as a dedicated non-provileged user, with a console available using screen, and the kvm process is managed using runit. Since our admins are familiar with these tools already, the learning curve involved is much less steep. Despite the relative simplicity of kvm-manager, it was still more complicated and involved than I wanted on my laptop. Additionally, I wanted to fully understand every piece of the puzzle and separating out user privileges wasn't important to me. So - I wrote the following bash script to launch virtual servers. It assumes you are using logical volume manager. Some editing required if you want to re-use it.
#!/bin/bash
# manage a virtual server
# add /etc/udev/rules.d/92-kvm.rules with the following line (change "jamie" 
# to the group you are running as and vg_animal0 to the name of your volume 
# group):
# ACTION=="change", SUBSYSTEM=="block", ATTR dm/name =="vg_animal0-*_root", GROUP="jamie"
#
# install a new server by:
# * create logical volume with the name: servername_root
# * $0 servername start /usr/local/share/ISOs/default.iso
bridge=virbr0
user=jamie
base=/home/jamie
server="$1"
vg=vg_animal0
# modify server memory here
mem=768
command="$2"
[ -z "$command" ] && command=start
die ()  
  printf "$1\n"
  exit 1
 
cdarg=
if [ -n "$3" ]; then
  [ ! -f "$3" ] && die "Third argument should be path to cd iso. Can't find that path."
  cdarg="-cdrom $3"
fi
function start()  
  lvname="$ vg -$ server _root"
  # trigger udev to ensure we have proper ownership of the block device
  sudo udevadm trigger --subsystem-match=block --attr-match=dm/name="$lvname"
  lv="/dev/mapper/$lvname"
  [ ! -e "$lv" ] && die "Can't find $lv"
  sudo modprobe -v tun   die "Failed to modprobe tun module"
  # create network device if it doesn't already exist
  ip tuntap   grep "$tap"   sudo ip tuntap add dev "$tap" mode tap user "$user"   die "Failed to create device $tap"
  # bring up device if it's not already up
  ip link   grep " $tap "   sudo ip link set "$tap" up   die "Failed to set $tap to up"
  # add the device to the bridge so it get use the upstream network connections
  /sbin/brctl show   grep "$tap"   sudo brctl addif "$bridge" "$tap"   die "Failed to add tap to bridge"
  # launch kvm in a screen session
  screen -S "$server" kvm -drive "file=$lv,if=virtio,id=hda,format=raw" -m "$mem" -device "virtio-net-pci,vlan=1,id=net0,mac=$mac,bus=pci.0" -net "tap,ifname=$tap,script=no,downscript=no,vlan=1,name=hostnet0" $cdarg -nographic   die "Failed to start kvm" 
 
function cleanup()  
  read -p "Please shutdown the host first then hit any key to continue..."
  sudo brctl delif "$bridge" "$tap"
  sudo ip link set "$tap" down
  sudo ip tuntap del mode tap dev "$tap"
 
# generate reproducible mac address
mac="$(printf "02:%s" "$(printf "%s\0%s" "$(hostname)" "$ server "   sha256sum   sed 's/\(..\)/\1:/g'   cut -f1-5 -d:)" )"
tap="$ server 0"
case "$command" in
  start)
    start
    ;;
  cleanup)
    cleanup
    ;;
  *)
    die "Please pass start or cleanup as first argument"
esac

Gregor Herrmann: RC bugs 2012/09

as usual, here's my RCBW report. like in the last weeks, a mixture of NMUs and mails to the BTS, grabbing and writing patches, ...

2 March 2012

Sylvestre Ledru: Report from the BSP in Paris

From February 17 to 19th 2012, between 10 to 30 hackers met at IRILL's offices to squash as many bugs as possible for the Wheezy release. Besides about 42 pizzas, more than one hundred bugs (104 to be precise) have been fixed. Among them, 18 of them were fixed by removing old/unmaintained/deprecated packages from the Debian archive. In a great atmosphere, one of the most important achievements of this BSP was that Yada, a deprecated packaging helper, finally passed away thanks to the efforts of Cyril Brulebois. Thanks for all the contributors for making of this week end a success.

1 March 2012

Mike Hommey: Introducing faulty.lib

TL;DR link: faulty.lib is on github. Android applications are essentially zip archives, with the APK extension instead of ZIP. While most of Android is java, and java classes are usually loaded from a ZIP archive (usually with the JAR extension), Android applications using native code need to have native libraries on the file system. These native libraries are found under /data/data/$appid/lib, where $appid is the package name, as defined in the AndroidManifest.xml file. So, when Android installs an application, it puts that APK file under /data/app. Then, if the APK contains native libraries under a lib/$ABI subdirectory (where $ABI is armeabi, armeabi-v7a or x86), it also decompresses the files and places them under /data/data/$appid/lib. This means native libraries are actually stored twice on internal flash: once compressed and once decompressed. This is why Chrome for Android takes almost 50MB of internal flash space after installation. Firefox for Android used to have that problem, and we decided we should stop doing that. Michael Wu thus implemented a custom dynamic linker, which would load most of Firefox libraries directly off the APK. This involves decompressing the zipped data somewhere in memory, and doing ld.so s job to make the library usable (please note that on Android, ld.so is actually named linker). There were initially circumstances under which we would decompress into a file and reuse it the next time Firefox starts, but we subsequently removed that possibility (except for debugging purpose) because it ended up being slower than decompressing each time (thanks to internal flash being so slow). Anyways, in order to do ld.so s job, our custom linker was directly derived from Android s system linker, with many tweaks. This custom linker has done its job quite well for some time, now, but has been recently replaced, see further below. Considering Firefox can t do anything useful involving Gecko until its libraries are loaded, in practice, this means Firefox can t display a web page faster than completely decompressing the libraries. Or can it? Just don t sit down cause I ve moved your chair We know that a lot of code and data is not used during Firefox startup. Based on that knowledge, we started working on only loading the necessary bits. The core of the idea is, when a library is requested to be loaded, to reserve anonymous memory for its decompressed size, and that s all. That memory is protected such that any access to it triggers a segmentation fault. When a segmentation fault happens, the required bits are decompressed, and execution is resumed where it was before the segmentation fault. The original prototype was decompressing from a normal zip deflated stream, which means it was impossible to seek in it. So, if an access was made at the end of the library, it was necessary to decompress the whole library. With some nasty binary reordering, and some difficulty, it was possible to avoid accessing the end of the library, but the approach is very much fragile. It only takes an unexpected code path to make things much slower than they should be. Consequently, for the past months, I ve been working on improving the original idea and, with some assistance from Julian Seward, implemented the scheme with seekable compressed streams. Instead of letting the zip archive creation tool deflate libraries, we store specially crafted files. Essentially, files are cut in small chunks, and each chunk is compressed individually. This means a less efficient compression, but it also means random access to chunks is possible. However, instead of stacking on top of our existing custom linker, I started over, from the ground up. First, because it needed a serious clean up (a good part of linker.c is leftovers from the Android linker that we don t use, and APKOpen.cpp is a messy mix of JNI stubs, library decompression handling (which in itself was also a mess) and Gecko initialization code) and most importantly, because it relied on some Android system linker internals and thus required binary compatibility with the system linker. Which, according to Google engineers that contacted us a few months ago, was going to break in what we now know will be called Android Jelly Bean. The benefit of the clean slate approach is that the new code is not tied to Gecko at all and was designed to work on Android as well as on desktop Linux systems (which made debugging much much easier). We re thus releasing the code as a separate project: faulty.lib. It is licensed under the Mozilla Public License version 2.0. Please feel free to try, contribute, and/or fork it. This dynamic linker is not meant to completely follow standard ELF rules (most notably for symbol resolution), and as a result does some assumptions. It s also still work in progress, with some obvious optimizations pending (like, avoiding to resolve the same symbols again and again during relocations), and some features missing (for example, symbol versioning). The next blog post will give some information about how to build Firefox for Android to benefit from on-demand decompression. I will also detail a few of the tricks involved in this dynamic linker in subsequent blog posts.

26 February 2012

Joachim Breitner: GHC 7.4.1 speeds up arbtt by a factor of 22

More than two years ago I wrote arbtt, a tool that silently records what programs you are using and allows you to do statistics on that data later, based on rules that you define afterwards, hence the name automatic rule based time tracker. I wasn t doing much with it recently (the last release has been half a year ago), but it nevertheless was running on my machine and by now has tracked a total time span of 248 days in 350000 records. Yesterday, I had a use for it again: measuring the time spent creating a certain document with LaTeX. So I added a rule to my categorize.cfg and ran arbtt-stats. I knew that it was not very fast, and that my data set has grown considerably since I last used it. And indeed, it took more than 6 minutes to process the data and spit out the result. Since I m currently working on the GHC 7.4.1 transition in Debian anyways, I decided to check what happens if I compile the code with that version of the Haskell compiler, instead of the previous version 7.0.4. And behold: The whole process took merely 17.3 seconds to complete! At first I did not believe it, but the result was identical, both binaries were built with the same option, i.e. no profiling enabled or anything like that. Wouldn t you also like to have such speed ups for free, just by waiting for someone else to improve their work?

I tried to find out the reason for the speed up and created profiling output from both the old and the new binary. The old binary spends 83% of the time in Categorize.checkRegex, which basically just call Text.Regex.PCRE.Light.match. Since the version of pcre-light is the same in both binaries, I conclude that the Foreign Function Interface that GHC provides to interact with C libraries (libpcre in this case) is much faster now, although I do not find any mention in the release notes. And even if I do not count the 83% time spent in checkRegex, the code from the new compiler is still 2.7 times faster. Thanks, GHC devs, great work!

Flattr this post

23 February 2012

Russ Allbery: Debian Policy 3.9.3.0

I've just uploaded version 3.9.3.0 of Debian Policy. The really big change in this version is the 1.0 version of the copyright format specification, but that will get a separate announcement once the www team has had a chance to put it in place on the web site. There are a bunch of other fixes, though, including some normative changes. Here's the upgrading checklist (duplicated from my announcement, but formatted a bit prettier):
2.4
New archive sections education, introspection, and metapackages added.
5.6.8
The Architecture field in *.dsc files may now contain the value any all for source packages building both architecture-independent and architecture-dependent packages.
7.1
If a dependency is restricted to particular architectures, the list of architectures must be non-empty.
9.1.1
/run is allowed as an exception to the FHS and replaces /var/run. /run/lock replaces /var/lock. The FHS requirements for the older directories apply to these directories as well. Backward compatibility links will be maintained and packages need not switch to referencing /run directly yet. Files in /run should be stored in a temporary file system.
9.1.4
New section spelling out the requirements for packages that use files in /run, /var/run, or /var/lock. This generalizes information previously only in 9.3.2.
9.5
Cron job file names must not contain "." or "+" or they will be ignored by cron. They should replace those characters with "_". If a package provides multiple cron job files in the same directory, they should each start with the package name (possibly modified as above), "-", and then some suitable suffix.
9.10
Packages using doc-base do not need to call install-docs anymore.
10.7.4
Packages that declare the same conffile may see left-over configuration files from each other even if they conflict.
11.8
The Policy rules around Motif libraries were just a special case of normal rules for non-free dependencies and were largely obsolete, so they have been removed.
12.5
debian/copyright is no longer required to list the Debian maintainers involved in the creation of the package (although note that the requirement to list copyright information is unchanged).
copyright-format
Version 1.0 of the "Machine-readable debian/copyright file" specification is included.
mime
This separate document has been retired and and its (short) contents merged into Policy section 9.7. There are no changes to the requirements.
perl
Packages may declare an interest in the <perl-major-upgrade> trigger to be notified of major upgrades of perl.
virtual
ttf-japanese- mincho,gothic have been renamed to fonts-japanese- mincho,gothic .
Please review and update your packages as necessary the next time you make an upload.

22 February 2012

Michal &#268;iha&#345;: Secure your phpMyAdmin

phpMyAdmin is quite popular software (to give some numbers let's mention 10000 downloads daily on SourceForge.net or 122685 reports in Debian's popcon) and as such is quite attractive target for various scripted attacks. If you run phpMyAdmin installation somewhere you should really make sure it is enough secured, so that these script kiddies don't get through. In past month I've looked at what kind of attacks are these guys trying and in all cases these are pretty old vulnerabilities, some of them fixed years ago. So the first thing you should do is to update. It is always good to run latest stable version, but in case you can not for whatever reason, try at least taking the most important fixes and using them. In ideal world your distribution would do this job for you, but in case it did not, you can for example take patches from Debian, which is pretty good at taking our patches (surprisingly it is not much related to my involvement there). To check which patches they have applied you can use excellent patch-tracker tool, which exposes patches from all released packages. To give you overview of which issues are mostly being attempted to exploit by script kiddies right now, here is the list: If you have fixed these, you should be pretty safe for now, but follow our security announcements for possible future issues (you can use RSS feed or subscribe to news mailing list, where all security issues are announced as well). However there are more things you can do to keep you safer: So these are the basic steps which will help you against possible compromise, I might return to some of these in more details in future posts.

Filed under: Debian English Phpmyadmin Suse 0 comments Flattr this!

Cyril Brulebois: Wrench days

Fixing hardware Thanks to the nice Debian Swiss Knives initiative! Fixing software Besides trying to maintain the whole X stack as usual, and preparing the funky wayland, mesa, and weston trio for experimental, I got myself into other areas. Debugging gzip Jakup Wilk noticed a while ago that some multiarchified packages couldn t be co-installed due to gzipped files being different between architectures. That was reported as #647522 against the gzip package. While some developers consider the lack of determinism (as in: it s not in the gzip specification, and the current implementation generates different packages anyway!) as a dead-end for the multiarch experiment, it didn t look like undebuggable or unfixable. And indeed, as confirmed by a quick analysis: a lack of clean-up when several files are processed at once can lead to different results, depending on the order in which files are specified on the command line; the patch for that is trivial, and was later made smaller and the need for it was explained. Working in the Release Team I reviewed/approved a few packages for squeeze, but also took care of handling or finishing a few transitions (as seen in my hints file), among which the funny iceweasel9/libmozjs ones; release tools are nice, but one gets to learn lots of stuff at once (which isn t a negative aspect). I hacked a tiny collision detector so that packages involved in several transitions can be easily spotted. Given the huge number of ongoing transitions we had lately, that didn t look overkill. Squashing bugs It had been a very long time since my last working on totally random RC bugs. The Bug Squashing Party organized at IRILL was a very nice opportunity to get back to totally crazy bugs in unknown packages, but also to meet with other Debian contributors; sponsoring maintainers with good patches on their first try was nice, but walking other contributors through their first patches sent to the BTS or through their first NMUs was nice too. Waving good bye to yada Back in the M rida QA meeting I attended in 2007, there were already jokes about how bad yada was, and how it should die in a fire etc. When I started looking at the RC bug list, I quickly switched to scrolling it backwards, and I wondered how much work was left. The details can be seen in #660548, and the result is: yada is gone! sanity++; packages.debian.org Both long descriptions and translations for packages in some suites disappeared after the switch to Description-md5. Thanks to a quick and reduced packages.debian.org setup (mostly: en+de for squeeze+sid+experimental), I managed to find my way through Perl and DB files to propose patches for #657557. I m still waiting for a confirmation, but in case it works fine, we could even get a fix for DDTP/DDTSS.

21 February 2012

Russ Allbery: Debian packaging updates

Now is the point in the Debian release cycle where I usually try to go through my packages and bring them generally up to date. In case other people are doing the same thing, here are a few new facilities or techniques that I'm rolling out across my packages. (This is apart from the obvious stuff, like multiarch where appropriate and debhelper 9, and the older stuff, like using dh.) It's also worth mentioning that Ubuntu was responsible for breaking a lot of ground here. Due to bug reports and patches submitted from Ubuntu, several of my packages already had hardening build flags and --as-needed issues fixed before this round of packaging refresh, which made adding these features much easier than it would be otherwise.

19 February 2012

Gregor Herrmann: RC bugs 2012/07

thanks to the Paris BSP & other activities we're seeing a nice decline in RC bugs. here are my recent contributions:

18 February 2012

Stefano Zacchiroli: GPL-d Debian software skew (?)

At FOSDEM, John Sullivan delivered an interesting talk titled Is copyleft being framed? to verify alleged claims on the decline of GPL-d software. (Slides are available.) The crux of the talk is the analysis he performed on the Debian archive to discover the amount of software we distribute that is covered by GPL, LGPL, or AGPL ("GPL-d" for short in the remainder). John's talk steps in an interesting and long running debate (a recent summary of which is available in this ITWire article). The most interesting part is the discrepancy among John's results and Blackduck's, which are often used to argue how the popularity of the GPL license is declining. That might be the case. Or not. The more analyses we do to find it out, the better. The underlying assumption on John's work is that Debian is a representative sample of the Free Software out there, which I think is a reasonable assumption. I find the analysis presented in the talk completely satisfactorily from a purely scientific point of view. The same cannot be said about Blackduck's result: both their methods and data are secret, making it impossible to reproduce their experiments. Highly unscientific. Still, John's results are surprising: as much as 87 percent of Lenny's packages and 93 percent of Squeeze's are GPL-d. That seems a lot. Puzzled about that, John discussed with me the issue before his talk, in search for pitfalls in his methods or data. Finding none, I pointed him to the almighty DktrKranz for some extra review; who found nothing either. To stay on the safe side, even during his talk John called for independent reviews of his results. What could be wrong? The tool used to gather the data is license-count from the debian-policy package. Input data are the debian/copyright files of all Debian source packages. If license-count is not bugged, our debian/copyright files might be. One thing that occurred to me only a few days ago is the habit of declaring a different license for Debian packaging (the files under debian/) than the software being packaged itself. That's a bad habit because it might cause unwanted license mixtures via patches that live under debian/ but I've seen several occurrences of it in the Debian archive. For name and (self-)shame: I've also been guilty of it in the past, when I was young . Is that reason enough to skew results and overestimate GPL-d software? I don't think so, I hope not, but ultimately I don't know. It'd be nice to rule out the possibility entirely. So if anyone is willing to do some sampling of affected debian/copyright files and propose patches for license-count to exclude those "false positives", please shout. (As a bonus point: that would also help to take more sound decision for the typical use case of license-count, i.e. deciding when a license should be added to /usr/share/common-licenses.) Other independent reviews of the results are equally welcome. Note: the above, as well as John's analysis, would be a trivial exercise if DEP-5 were already widely deployed in the Debian archive.
Update: add link to John's slides
Update 19/02/2012: Russ Allbery, author of license-count, posted a way more likely cause of data skew in John's analysis: double counting among the different types of copyleft licenses

Benjamin Mako Hill: Advice for Prospective Doctoral Students

There is tons of advice on the Internet (e.g., on the academic blogs I read) for prospective doctoral students. I am very happy with my own graduate school choices but I feel that I basically got lucky. Few people are saying the two things I really wish someone had told me before I made the decision to get a PhD: Most People Getting Doctorates Probably Shouldn't In most fields, the only thing you need a PhD for is to become a professor -- and even this requirement can be flexible. You can have almost any job in any company or non-profit without a PhD. You can teach without a PhD. You can write books without a PhD. You can do research and work in thinktanks without a PhD. You don't even always need a PhD to grant PhDs to other people: two of my advisors at the Media Lab supervised PhD work but did not have doctorates themselves! Becoming a tenured professor is more difficult without a doctorate, but it is not impossible. There are grants and jobs outside of universities that require doctorates, but not nearly as many as most people applying for PhDs programs think. Getting a doctorate can even hurt: If you want to work in a company or non-profit, you are usually better off with 4-6 years of experience doing the kind of work you want to do than with the doctorate and the less relevant experience of getting one. Starting salaries for people with doctorates are often higher than for people with masters degrees. But salaries for people with masters degrees and 5 years of experience are even higher -- and that's before you take into account the opportunity costs of working for relatively low graduate student wages for half a decade. PhD take an enormous amount of time and, in most programs, you spend a huge amount of this time doing academic busy work, teaching, applying for grants or fellowships, and writing academic papers that very few people read. These are skills you'll need to be a successful professor. They are useful skills for other jobs too, but not as useful as the experience of actually doing those other jobs for the time it takes to get the degree. Evaluating Graduate Programs If you are still convinced you need a doctorate, or any graduate degree for that matter, you will need to pick a program. Plenty of people will offer advice on how to pick the right program and trying to balance all the complicated and contradictory advice can be difficult. Although I love my program and advisors, I've known many less happy students. Toward that end, there are two pieces of meta-advice that I wish everybody was told before they applied:
  1. Find recent graduates of the program you are considering, and the faculty advisor(s) you are planning on working with, and look at where they are now. Are these ex-students doing the kind of work that you want to do? Are they at great programs at great universities? Chances are good that a PhD program and its faculty will prepare future students to be like, and do work like, the students they have trained in the past. Programs that consistently make good placements are preparing their students well, supporting them, making sure they have the resources necessary to do good work, and helping their students when they are on the job market. A program whose students do poorly, or just end doing work that isn't like the kind you want to do, will probably fail you too.

  2. If recent graduates seem to be generally successful and doing the kind of work you want to do, find one who looks most like the kind of academic you want to become and talk to them about their experience. Chances are, your faculty advisors will overlap with theirs and your experience will be similar. Ex-students can tell you the strengths of weaknesses of the program you are considering and what to watch out for. If they had a horrible experience, there's a decent chance you will too, and they will tell you so.

Doing these two things means you don't have to worry about trying to think of all the axes on which you want to evaluate a program or pour through admissions material which is only tangentially connected to the reality you'll live for a long time. What matters most is the outcomes, of course, because you're be living the rest of your life for a lot longer than you'll be in the PhD program.

15 February 2012

Mike (stew) O'Connor: Udev and multiple keyboards

I've started using my kinesis keyboard with my laptop a lot during the workday, but not the rest of the time, which means my USB keyboard is being plugged and unplugged often. I prefer to have a lot of keys remapped on the kinesys, So I was having an annoying time having to run xmodmap (possibly without the benefit of having Mod4 to get the window manager to give me a shell) to set these keys. I survived about two weeks before this bugged me enough to figure out what could be done to solve this. I had solved it once before by dropping a magically crafted file into /etc/hal/fdi/policy, but that system is long gone. Luckily it looks like thanks (at least in part) to Martin Pitt, udev has a keymap facility for handling this type of scenario. /usr/share/doc/udev/README.keymap.txt.gz has a bunch of good hints that got me started.
$ /lib/udev/findkeyboards
USB keyboard: input/event2
AT keyboard: input/event0
Finds my keyboard devices. I'm interested in the USB connected keyboard. So I run:
/lib/udev/keymap -i input/event2
To discover scan codes my keyboard generates for the keys I want to remap. here #654928 got me, the trick is to issue the above command from the laptop keyboard, so that the terminal you launched keymap in doesn't get wonky because it never got the key-release event from the enter key. When I hit keys on my external keyboard now, I get stuff like:
scan code: 0x7004A   key code: home
scan code: 0x700E7   key code: rightctrl
They key codes coming out match what is printed on the keycaps. From here I generate a file matching files in /lib/udev/keymaps, but change the keycodes to the keycodes I want to remap to, ending up with:
0x7004A leftmeta 
0x700E7 rightalt
This file goes into /lib/udev/keymaps/kinesis-stew. It appears the things like leftmeta/rightalt which are valid come from /usr/include/linux/input.h. It also seems to be valid to put hex there, but I didn't find a good way to figure out what hex codes go with which key codes. Then I look for the VendorID/DeviceID by looking in the output of "udevadm info --export-db" for input/event2 and find:
ID_VENDOR_ID=05f3
ID_MODEL_ID=0007
So now I'm ready to create my udev rule (which I put into /etc/udev/rules.d/99-stew-keymap.rules
ATTRS idVendor =="05f3", ATTRS idProduct =="0007", RUN+="keymap $name kinesis-stew"
And now everything seems to be good. I can plug and unplug my external keyboard without having to run xmodmap to get the keybindings I want, and the keybindings for the built-in laptop keyboard are not disturbed.

14 February 2012

Martin-&#201;ric Racine: Adopted: ispell-et, myspell-lv, rus-ispell.

Over recent months, I have been pondering the relevance of my Free Software involvement a lot: More than anything, it constantly feels more and more like several important distributions and projects are moving in directions that break with the old ways far too radically, by breaking software usability rules such as "least surprise", "works out of the box", and "if it ain't broken, don't fix it" either to try new approaches to addressing existing needs or simply for change's sake. Another aspect lies in my usual motivation for packaging or adopting software: because I need it and nobody else bothered doing it. Further reflection made me ponder what, correspondingly, are my usual reasons for dropping software: because what exists already works and because everybody and their grand-mother already invented a million of ways to handle the same issue. Which brings us to today's news. What prompted me to hand the dictionary packages I maintained over to someone else was a combination of "no longer need it" and "it already works fine as it is." For instance, the Estonian dictionaries haven't seen any upstream release since the initial one in 2003. While I still use Estonian daily in my online communications, the last 9 years have mostly been spent making minor changes to the dependencies and maintainer scripts, just to keep the package compliant with current Debian packaging policies and with ongoing improvements to the dictionary-common maintainer tools. This is not the sort of work that requires any knowledge of the Estonian language, so I felt that having dictionary-common developers handle those technical transitions in a clean and across-the-board way for as many dictionary packages as possible would be more productive than me trying to keep track of those changes by myself. A similar case applies to the Latvian dictionaries. While there have been occasional upstream releases, some of which required patching the source code, maintaining the package has mostly been about tracking Debian policy changes and dictionary-common functional changes. Additionally, my interest for learning Latvian has dramatically dropped over the years, so I no longer saw any point in remaining involved in maintaining the package. Ditto for the Russian dictionaries: occasional upstream releases, occasional patches, regular packaging upgrades to keep up with the Debian policy and with dictionary-common functionality, but no longer much of anything than a passive interest in practicing my Russian. One of the developers behind dictionary-common, Agustin Martin Domingo, frequently helped me make sense of the changes I needed to track in the past, so he gladly accepted taking over the technical maintenance of all 3 packages. The Estonian dictionaries, while extremely skim in the breadth of vocabulary they cover, remain useful, but are essentially deprecated, as the upstream author is working on a spell checking engine similar to what Voikko does for Finnish, which is why maintaining them will be a rather easy task for Agustin. Meanwhile, Aigars Mahinovs passively remains on board for the Latvian dictionaries, while the Russian dictionaries have been adopted by Mikhail Gusarov, in both cases with Agustin assisting on technical matters. Basically, I feel that passing the maintenance over to people whose motivation remains high is a better way to guarantee those packages' future than leaving them in my unmotivated hands and I'm glad that I found someone to keep on packaging them. PS: happy Valentine's day to everyone!

9 February 2012

Lior Kaplan: Debian packaging for beginners @ FOSDEM

After not attending FOSDEM for a few years, this year I decided to attend and also give a talk about Debian packaging for beginners , a replay of a talk given by Gergely Nagy in Debconf11 (video). As the per distribution devrooms were replaced a few years ago with the cross-distributions rooms, I thought it might be a good chance to have this kind of introduction talk to help people started contributing to Debian (or derived distributions). The talk wasn t meant to replace reading the documentation (new maintainer s guide, developer s references and the debian policy), but it s a good start for those who want some hands-on experience. The idea is to start with a just the upstream directory and progress trough the various errors and warnings we get during the build process using dpkg-buildpackage. The outcome is of course very basic, but enough to get people ready to do things on their own, including various QA tests on the package (e.g. lintian and debdiff). The presentation covers the important points of the 3 main files in the debian directory: control, changelog and rules. It addition it holds some information about the various tools one can used to test the packages. I hope to make another version of the presentation to be more standalone than just having the main points during the talk itself.
Filed under: Debian GNU/Linux, FOSDEM

8 February 2012

Ian Wienand: Python and --prefix

Something interesting I discovered about Python and --prefix that I can't see a lot of documentation on... When you build Python you can use the standard --prefix flag to configure to home the installation as you require. You might expect that this would hard-code the location to look for the support libraries to the value you gave; however in reality it doesn't quite work like that. Python will only look in the directory specified by prefix after it first searches relative to the path of the executing binary. Specifically, it looks at argv[0] and works through a few steps is argv[0] a symlink? then dereference it. Does argv[0] have any slashes in it? if not, then search the $PATH for the binary. After this, it starts searching for dirname(argv[0])/lib/pythonX.Y/os.py, then dirname(argv[0])/../lib and so on, until it reaches the root. Only after these searches fail does the interpreter then fall back to the hard-coded path specified in the --prefix when configured. What is the practical implications? It means you can move around a python installation tree and have it all "just work", which is nice. In my situation, I noticed this because we have a completely self-encapsulated build toolchain, but we wish to ship the same interpreter on the thing that we're building (during the build, we run the interpreter to create .pyc files for distribution, and we need to be sure that when we did this we didn't accidentally pick up any of the build hosts python; only the toolchain python). The PYTHONHOME environment variable does override this behaviour; if it is set then the search stops there. Another interesting thing is that sys.prefix is therefore not the value passed in by --prefix during configure, but the value of the current dynamically determined prefix value. If you run an strace, you can see this in operation.
readlink("/usr/bin/python", "python2.7", 4096) = 9
readlink("/usr/bin/python2.7", 0xbf8b014c, 4096) = -1 EINVAL (Invalid argument)
stat64("/usr/bin/Modules/Setup", 0xbf8af0a0) = -1 ENOENT (No such file or directory)
stat64("/usr/bin/lib/python2.7/os.py", 0xbf8af090) = -1 ENOENT (No such file or directory)
stat64("/usr/bin/lib/python2.7/os.pyc", 0xbf8af090) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/python2.7/os.py",  st_mode=S_IFREG 0644, st_size=26300, ... ) = 0
stat64("/usr/bin/Modules/Setup", 0xbf8af0a0) = -1 ENOENT (No such file or directory)
stat64("/usr/bin/lib/python2.7/lib-dynload", 0xbf8af0a0) = -1 ENOENT (No such file or directory)
stat64("/usr/lib/python2.7/lib-dynload",  st_mode=S_IFDIR 0755, st_size=4096, ... ) = 0
Firstly it dereferences symlinks. Then it looks for Modules/Setup to see if it is running out of the build tree. Then it starts looking for os.py, walking its way upwards. One interesting thing that may either be a bug or a feature, I haven't decided, is that if you set the prefix to / then the interpreter will not go back to the root and then look in /lib. This is probably pretty obscure usage though! All this is implemented in Modules/getpath.c which has a nice big comment at the top explaining the rules in detail.

7 February 2012

Rapha&#235;l Hertzog: Dpkg with multiarch support available in Debian experimental

As I announced on debian-devel, Guillem Jover uploaded a snapshot of dpkg s multiarch branch to experimental (version 1.16.2~wipmultiarch). Beware: There will
likely be some small interface changes between this version and the version that will be released later in unstable (possibly in the output of dpkg --get-selections, dpkg --list, maybe other commands). multiarch allows you to install packages from different architectures on the same machine. This can be useful if your computer can run programs from 2 architectures (eg. x86 CPU supporting i386 and amd64), or if you often need to cross-compile software and thus need the libraries of your target architecture. Test dpkg with multiarch support If you want to test multiarch support in dpkg, install the package from experimental (apt-get install dpkg/experimental assuming you have experimental in your sources.list). Then you can add a supplementary architecture to your system by doing sudo dpkg --add-architecture <arch> (e.g. i386 if you are on amd64, and vice-versa). APT will automatically pick up the new architecture and start downloading the Packages file for the new architecture (it uses dpkg --print-foreign-architectures to know about them). From there on you can install packages from the foreign architectures with apt-get install foo:<arch> . Many packages will not be installable because some of their dependencies have not yet been updated to work with in a multiarch world (libraries must be installed in a multiarch-compliant path so as to be co-installable, and then marked Multi-Arch: same ). Other dependencies might need to be marked Multi-Arch: foreign . See wiki.debian.org/Multiarch/Implementation for more HOWTO-like explanations. Now is a good time to see if you can install the foreign packages that you could need in such a setup and to help to convert the required libraries. You can also read Cyril Brulebois article which quickly shows how to hunt for the problematic packages which have not been converted to multiarch (in his sample, ucf is not ready. Since it s an Architecture: all package which can run on any architecture, it means that it s lacking a Multi-Arch: foreign field). Report bugs If you discover any bug in dpkg s multiarch implementation, please report it to the Bug Tracking System (against dpkg with the version 1.16.2~wipmultiarch ). If you notice important libraries or packages which are not yet multiarch ready, please open wishlist bug reports requesting the conversion and point the maintainers towards the wiki page linked above. Even better, prepare patches and submit those with your bug reports. Again, you can follow the lead of Cyril Brulebois who filed 6 bugs! Review the multiarch implementation If you re a C programmer and have some good knowledge of dpkg (or are willing to learn more of it), we would certainly benefit from more eyes reviewing the multiarch branch. If you want to discuss some design issues of the multiarch implementation in dpkg (or have questions related to your review), please get in touch via debian-dpkg@lists.debian.org. The latest version of the branch is pu/multiarch/master in Guillem s personal repository. I have my own version of the branch (pu/multiarch/full) which is usually a snapshot of Guillem s branch with my own submitted fixes.
$ git clone git://git.debian.org/dpkg/dpkg.git
$ cd dpkg
$ git remote add guillem git://git.hadrons.org/git/debian/dpkg/dpkg.git
$ git remote add buxy git://git.debian.org/~hertzog/dpkg.git
$ git fetch guillem && git fetch buxy
If you followed the instructions above, the relevant branches are thus guillem/pu/multiarch/master and buxy/pu/multiarch/full. Both branches are regularly rebased on top of master where Guillem merges progressively the commits from the multi-arch branch as his review progresses. Thank you in advance for your help bringing multiarch in shape for Debian Wheezy,

6 comments Liked this article? Click here. My blog is Flattr-enabled.

Next.

Previous.