This year is in its last throes and a few things, that don't really warrant their own blog post, have piled up here. To get them out of the door before 2012 starts, I'd like to go over them in the next few lines. First: there seem to be a group of rather cheap people who want to have the Humble Indie Bundle #4, but don't like to give at least 0.01 USD. Really guys, you seem to have enough money for a fast-enough computer and a (broadband) internet connection (otherwise downloading the circa 2.8 GB of games won't be a lot of fun), but you don't have the 0.01 USD to spare? If you don't like closed-source games (a sentiment I can truly understand, in fact, that was something I really pondered before buying) then just don't play them. But searching for a torrent for the HIB#4 while it is still being sold to just avoid paying the minimal fee, I can't understand. Now, some might wonder how I know this. The easy answer is, I blogged about buying the HIB#4 and mentioned, that I downloaded the games through a torrent (you can choose between direct downloads and torrents on your "login page") and a lot of people started hitting the blog entry with a referrer from a search engine, showing they'd searched for "humble indie bundle 4 torrent". Please make a decision: don't play the games or if you wish to, then pay something. The people behind those games make it easy and, in comparison to many other game studios, a fair offer. That should be worth something. For those of you who loved Josephine, there is a new awesome piece of art, Creative Commons licensed, by An-U (German homepage). The title (and the drawing) are derived from the equally awesome Katzenjammer song "On the Devil's Back" (in case you're still looking for a present, you might want to consider one of their CDs or a ticket to their next concert near you). Then, I wanted to thank all those people, who donated something over the course of 2011! Thanks a lot, especially for showing me, that you liked my services/work enough to even consider a donation! It means a lot to me, to know this. At year's end everybody is doing statistics and while I won't bore you with long in-depth analysis, I thought I throw out, what the top posts/things where (so far). Maybe you find yourself represented in there or maybe you find something new. In no particular order:

• My unofficial wine-unstable packages generated a lot of traffic and back links.
• The mini-tips series had many hits.
• My two posts (1, 2) about the Linux graphics stack garnered a lot of attention.
• The note about removing Sun's Java implementation was the first post to be mentioned in a Debian Project News issue.
• Right after my wine-unstable packages, my post about OSM and OpenLayers generated the most e-mails.
• The YouTube playlist fetcher is also pretty popular, though the Greasemonkey script collection has been picking up speed recently.

I hope your 2011 was a good year and I wish you all the best for 2012! I'm pretty sure Debian will continue to grow, but there is also a lot of work ahead, maybe you can join the effort and make Debian that much more awesome!

In 1977 Tal Brody said after winning the Euroleague Basketball cup with Maccabi Tel Aviv as sentence which became quite famous in Israel:

We are on the map! And we are staying on the map not only in sports, but in everything.

I found out through Invar Hovav that there s a regression in the Ubuntu installation doesn t believe this is true anymore, as according to lp#905754, you can t select Israel or its major cities, and instead get suggestions for Gaza and Hebron. This is a regression from past installers when you could at least choose Tel Aviv, Jerusalem and Gaza. It used to be hard to pin point each location, but was still possible. This might be part of a bigger problem of selecting close/adjacent cities, but the default behavior in the past was different. From the technical point of view, the Gaza and Hebron TZ files are weird (tzdata version 2011n-1). Both don t have info for 2012 daylight saving time (usually start in late March or early April). Hebron switches twice a year to DST, hopefully nothing critical uses this time zone.

$ zdump -v Asia/Hebron grep 2011
Asia/Hebron Fri Apr 1 10:00:59 2011 UTC = Fri Apr 1 12:00:59 2011 EET isdst=0 gmtoff=7200
Asia/Hebron Fri Apr 1 10:01:00 2011 UTC = Fri Apr 1 13:01:00 2011 EEST isdst=1 gmtoff=10800
Asia/Hebron Sun Jul 31 20:59:59 2011 UTC = Sun Jul 31 23:59:59 2011 EEST isdst=1 gmtoff=10800
Asia/Hebron Sun Jul 31 21:00:00 2011 UTC = Sun Jul 31 23:00:00 2011 EET isdst=0 gmtoff=7200
Asia/Hebron Mon Aug 29 21:59:59 2011 UTC = Mon Aug 29 23:59:59 2011 EET isdst=0 gmtoff=7200
Asia/Hebron Mon Aug 29 22:00:00 2011 UTC = Tue Aug 30 01:00:00 2011 EEST isdst=1 gmtoff=10800
Asia/Hebron Thu Sep 29 23:59:59 2011 UTC = Fri Sep 30 02:59:59 2011 EEST isdst=1 gmtoff=10800
Asia/Hebron Fri Sep 30 00:00:00 2011 UTC = Fri Sep 30 02:00:00 2011 EET isdst=0 gmtoff=7200

And Gaza only have a 3 months daylight saving time, while Israel has 6 and European eastern time has 7 months (all are UTC+2 region).

$ zdump -v Asia/Gaza grep 2011
Asia/Gaza Sat Apr 2 10:00:59 2011 UTC = Sat Apr 2 12:00:59 2011 EET isdst=0 gmtoff=7200
Asia/Gaza Sat Apr 2 10:01:00 2011 UTC = Sat Apr 2 13:01:00 2011 EEST isdst=1 gmtoff=10800
Asia/Gaza Sun Jul 31 20:59:59 2011 UTC = Sun Jul 31 23:59:59 2011 EEST isdst=1 gmtoff=10800
Asia/Gaza Sun Jul 31 21:00:00 2011 UTC = Sun Jul 31 23:00:00 2011 EET isdst=0 gmtoff=7200

p.s.
This isn t a political post, don t make it into one.
Filed under: Debian GNU/Linux, Ubuntu

DRBD is a system for replicating a block device across multiple systems. It s most commonly used for having one system write to the DRBD block device such that all writes are written to a local disk and a remote disk. In the default configuration a write is not complete until it s committed to disk locally and remotely. There is support for having multiple systems write to disk at the same time, but naturally that only works if the filesystem drivers are aware of this. I m installing DRBD on some Debian/Squeeze servers for the purpose of mirroring a mail store across multiple systems. For the virtual machines which run mail queues I m not using DRBD because the failure conditions that I m planning for don t include two disks entirely failing. I m planning for a system having an outage for a while so it s OK to have some inbound and outbound mail delayed but it s not OK for the mail store to be unavailable. Global changes I ve made in /etc/drbd.d/global_common.conf In the common section I changed the protocol from C to B , this means that a write() system call returns after data is committed locally and sent to the other node. This means that if the primary node goes permanently offline AND if the secondary node has a transient power failure or kernel crash causing the buffer contents to be lost then writes can be lost. I don t think that this scenario is likely enough to make it worth choosing protocol C and requiring that all writes go to disk on both nodes before they are considered to be complete. In the net section I added the following: sndbuf-size 512k;
data-integrity-alg sha1; This uses a larger network sending buffer (apparently good for fast local networks although I d have expected that the low delay on a local Gig-E would give a low bandwidth delay product) and to use sha1 hashes on all packets (why does it default to no data integrity). Reserved Numbers The default port number that is used is 7789. I think it s best to use ports below 1024 for system services so I ve setup some systems starting with port 100 and going up from there. I use a different port for every DRBD instance, so if I have two clustered resources on a LAN then I ll use different ports even if they aren t configured to ever run on the same system. You never know when the cluster assignment will change and DRBD port numbers seems like something that could potentially cause real problems if there was a port conflict. Most of the documentation assumes that the DRBD device nodes on a system will start at /dev/drbd0 and increment, but this is not a requirement. I am configuring things such that there will only ever be one /dev/drbd0 on a network. This means that there is no possibility of a cut/paste error in a /etc/fstab file or a Xen configuration file causing data loss. As an aside I recently discovered that a Xen Dom0 can do a read-write mount of a block device that is being used read-write by a Xen DomU, there is some degree of protection against a DomU using a block device that is already being used in the Dom0 but no protection against the Dom0 messing with the DomU s resources. It would be nice if there was an option of using some device name other than /dev/drbdX where X is a number. Using meaningful names would reduce the incidence of doing things to the wrong device. As an aside it would be nice if there was some sort of mount helper for determining which devices shouldn t be mounted locally and which mount options are permitted it MIGHT be OK to do a read-only mount of a DomU s filesystem in the Dom0 but probably all mounting should be prevented. Also a mount helper for such things would ideally be able to change the default mount options, for example it could make the defaults be nosuid,nodev (or even noexec,nodev) when mounting filesystems from removable devices. Initial Synchronisation After a few trials it seems to me that things generally work if you create DRBD on two nodes at the same time and then immediately make one of them primary. If you don t then it will probably refuse to accept one copy of the data as primary as it can t seem to realise that both are inconsistent. I can t understand why it does this in the case where there are two nodes with inconsistent data, you know for sure that there is no good data so there should be an operation to zero both devices and make them equal. Instead there The solution sometimes seems to be to run drbdsetup /dev/drbd0 primary - (where drbd0 is replaced with the appropriate device). This seems to work well and allowed me to create a DRBD installation before I had installed the second server. If the servers have been connected in Inconsistent/Inconsistent state then the solution seems to involve running drbdadm -- --overwrite-data-of-peer primary db0-mysql (for the case of a resource named db0-mysql defined in /etc/drbd.d/db0-mysql.res). Also it seems that some commands can only be run from one node. So if you have a primary node that s in service and another node in Secondary/Unknown state (IE disconnected) with data state Inconsistent/DUnknown then while you would expect to be able to connect from the secondary node is appears that nothing other than a drbdadm connect command run from the primary node will get things going.

Damn those unshaven yaks I m trying to write a Nagios plugin for work that will comprehensively monitor network interfaces and make sure they re up, passing traffic, all those sorts of things. Of course, I m doing it all in Ruby, because that s how I roll. So, I need to Know Things about the interface. Everyone does that with ethtool. Right? Sure, if your eyeballs are parsing it. But have you ever tried to machine parse it? To put it as eloquently as possible:

# ethtool eth0
Settings for eth0:
 Supported ports: [ TP MII ]
 Supported link modes:   10baseT/Half 10baseT/Full 
                         100baseT/Half 100baseT/Full 
                         1000baseT/Half 1000baseT/Full 
 Supports auto-negotiation: Yes
 Advertised link modes:  10baseT/Half 10baseT/Full 
                         100baseT/Half 100baseT/Full 
                         1000baseT/Half 1000baseT/Full 
 Advertised pause frame use: No
 Advertised auto-negotiation: Yes
 Link partner advertised link modes:  10baseT/Half 10baseT/Full 
                                      100baseT/Half 100baseT/Full 
                                      1000baseT/Half 1000baseT/Full 
 Link partner advertised pause frame use: No
 Link partner advertised auto-negotiation: Yes
 Speed: 1000Mb/s
 Duplex: Full
 Port: MII
 PHYAD: 0
 Transceiver: internal
 Auto-negotiation: on
 Supports Wake-on: pumbg
 Wake-on: g
 Current message level: 0x00000033 (51)
 Link detected: yes

Parse that, bitch!

Or perhaps not. At any rate, I decided that it would be most advantageous if I went straight to the source and twiddle the ioctl until it did my bidding. And thus, about 5 hours later, was Rethtool born. Once I worked out a less-than-entirely-crackful way of dealing with C structs in Ruby (after a bit of digging around, I went with the appallingly-undocumented-but-sufficiently-featureful CStruct), and after I finally worked out I was passing the wrong damned struct to ioctl(SIOCETHTOOL) (speaking of appallingly-undocumented: fuck you, ioctl, and all your twisty-passages children), it was smooth sailing. So, if you re one of the eight or so people on earth who will ever need to get at the grubby internals of your network interfaces using Ruby (and can t do it via some sysfs magic), Rethtool is for you.

Yesterday I bought my copy of the Humble Indie Bundle #4. And while the donation part is a good thing, I directed most of my payment to the developers of the games to allow them to make more games, which run natively on Linux. I haven't played much yet (no time) and just tried out the game for which the Torrent finished first (Bit.Trip.Runner). The nice thing about that game is, it is available as a Debian package, the not-so-nice thing about said package is, that it installs stuff in places where it shouldn't and has tons of wrong file properties like ownership of the files... The Depends line is also more or less pointless, as it just lists the needed packages, but not the minimal versions. Hopefully these things will be fixed in a future package version. Apart from that, the game in its retro look is sort of addictive. Yes, it's "just" simple Jump'n'Run, but it is a lot of fun to play. Partly because you "play" a simple melody, when you clear obstacles and collect the gold and other goodies. Anyway, long story short, if you want a few nice DRM-free games running natively on Linux (in my case with the Mesa3D drivers for my Radeon), I can recommend this bundle. And maybe the games get open-sourced in the future as has been done previously. None of these games are so called AAA titles, but that doesn't mean they can't provide some fun hours. And to top it off you don't need Wine or something else to run them on your Linux system.

Today QApt was accepted into Debian's archives. It took me some time, in fact a lot more than I hoped it'd take, to get the final pieces into place [UPDATE](the inital work was done by Jos , the co-maintainer of QApt)[/UPDATE], but now you're able to use QApt as a simple-to-use wrapper around the whole APT, DPKG and Xapian stack. The target users are C++ programs using the Qt framework. The use cases range from graphical package manager to allowing easy updates of certain packages from within another program. My interest in QApt comes from me being co-maintainer of Kingston, the update notification Plasmoid for KDE. We have a wishlist bug open against Kingston, requesting the addition of a "do update" button. We could have used the update-manager infrastructure, but both Sune and myself weren't too thrilled with that. Luckily there is an alternative in the form of QApt. I'll hope I find some time during the next weeks to implement the update mechanism for Kingston. The next things to come for QApt will be minor cleanups, already waiting in Git, a new upstream bug fix release (no date is set yet) and a multiarchification (already partly done in a local branch).

When I started using Linux, I started with a desktop environment (KDE) and then tried a lot of (standalone) window managers, including but not limited to Enlightenment, Blackbox, Fluxbox and Sawfish. But I was never really satisfied as it felt as if something was missing.
It then came, that I became a user of a desktop environment again. Now I have been a GNOME user for at least five years.

gsettings set org.gnome.desktop.session session-name 'gnome-fallback'

Last night, the text below went out to r-sig-finance along with updates to the R/Finance website and its Call for Papers page; followed by some tweeting and Goggle+'ing (and please do feel free to retweet and share at will...)

Call for Papers: R/Finance 2012: Applied Finance with R
May 11 and 12, 2012
University of Illinois, Chicago, IL, USA
The fourth annual R/Finance conference for applied finance using R will be held on May 11 and 12, 2012 in Chicago, IL, USA on the campus of the University of Illinois at Chicago. The two-day conference will cover topics including portfolio management, time series analysis, advanced risk tools, high-performance computing, market microstructure, and econometrics. All will be discussed within the context of using R as a primary tool for financial risk management, portfolio construction, and trading. Over the past three years, R/Finance has included attendees from around the world and featured keynote presentations from prominent academics and practitioners. We anticipate another exciting line-up for 2012 --- including keynote presentations from Blair Hull, Paul Gilbert, Rob McCulloch, and Simon Urbanek. We invite you to submit complete papers or one-page abstracts (in txt or pdf format) for consideration. Academic and practitioner proposals related to R are encouraged. We welcome submissions for full talks, abbreviated "lightning talks", and for a limited number of (longer) pre-conference seminar sessions. Presenters are strongly encouraged to provide working R code to accompany the presentation/paper. Data sets should also be made public for the purposes of reproducibility (though we realize this may be limited due to contracts with data vendors). Preference may be given to presenters who have released R packages. Travel and accommodation grants may be available for selected presenters at the discretion of the committee. In addition, the conference will award prizes for best papers. To be eligible for a best paper award, a submission must be a full paper. Extended abstracts, even if a full paper by conference time, are not eligible for a best paper award. Please send submissions to: committee at RinFinance.com. The submission deadline is January 31, 2012. Submitters will be notified of acceptance via email by February 28, 2012. Notification of whether a presentation will be a long presentation or a lightning talk will also be made at that time. Additional details will be announced at this website as they become available. Information on previous year's presenters and their presentations are also at the conference website R/Finance 2009, 2010 and 2011. For the program committee:

Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson,
Dale Rosenthal, Jeffrey Ryan, Joshua Ulrich

So see you in Chicago in May! Update: Corrected urls to past conference thanks to heads-up by Josh. Thanks!

The French government has published a call for tender for a 2 million euro contract to support Debian and CentOS systems throughout the French public administration. The news is a month old, but the call for bids is open until 9 January 2012. There is more coverage on a EC blog post as well as on Le Monde Informatique (in French). "Customers" of the bid winner will be almost all ministries of the French administration --- from prime minister to justice, from defense to sports, from education to culture --- for a 3-year period. The call requires ability to support most of the "usual suspects" among popular FOSS applications, but explicitly focuses on Debian and CentOS as distributions. It is by far not the first time it has happened, but it is always great for me to see major public administrations choosing community based distros (and Debian in particular ).

Lately I have been playing a little bit with different Platform as a Service providers. So far, the most developer-friendly one seems to be Heroku. In particular, Heroku provides an incredibly straightforward way to get started. After creating an account on their website, you just have to install the appropriate gem: sudo gem install heroku The command heroku help will then show you all the possible invocations of the command line client, just to give you an idea of what you can do with it (everything). Alright, that s not particularly interesting without an application to play with, is it? So let s create a new Sinatra application. In order to do that, we need a new directory containing a file like the following one:

# webapp.rb
require 'rubygems'
require 'sinatra'
# Add other gems you might need here
get '/' do
  "Your Sinatra app is working!"
end

Also add a Gemfile with all the required dependencies, in our trivial example sinatra is actually the only gem you need:

source :rubygems
gem "sinatra"

Lastly, you need a config.ru file where you can require our newly created Ruby program and specify that it is a Sinatra application. In this example, the Ruby file is called webapp.rb.

require './webapp'
run Sinatra::Application

A very useful program when developing a Sinatra application is shotgun. gem install shotgun and you will be able to run your new shiny app by executing the command shotgun. While developing on your machine, shotgun will take care of reloading the application for every request, so that you don t have to restart the server after every modification to the source code. Run bundle install to easily install all the required dependencies in your development environment, and shotgun to start the app locally. Pointing your browser to http://127.0.0.1:9393/ you can check if everything works as expected. If nothing went wrong, you can create a git repository to track changes to the source code:

git init
git add . 
git commit -m "Initial commit"

Perfect! Now, how do you create and deploy your application on Heroku?

heroku create
git push heroku master

Heroku will automatically find out that you deployed a Sinatra application, it will install all the required dependencies and start serving requests. A few interesting commands you might want to run to see what s going on are heroku info, heroku ps and heroku logs. The client also provides a command, heroku open, that opens your browser at the right production URL. So, you now have a working application under revision control. Dependencies are handled by specifying them into the Gemfile, and the required gems can be installed locally using bundle. Shotgun is our friend when it comes to use the application for development purposes. A git push heroku master is all we need to deploy a new version of the code. Of course there is way much more about Sinatra and Heroku than what I ve covered in this mini-intro. Please refer to the Sinatra Book and the Heroku Dev Center for more information!

Yesterday some confusion arised on my side when I saw a new facebook advertising campaign in my facebook account (yes, I am a member of facebook, although I'm aware of the privacy concerns). Basically it was saying that I should try the friends finder and that some of my friends (showing and naming me three of them) would have used it already.

Some background:
The friends finder feature of facebook is a feature that asks for the password of your e-mail account. It will then crawl through your emails to find contacts that might already be on facebook but not connected to you (in a facebook sense).

My first feeling was: Oh my god. How can it be that friends (and family) of mine are so naive? Especially since there were people included which I consider to be quiet clever. But honestly: Who would be so naive to give an unknown company direct unsupervised (you can't tell what they really do) access to your mail account? Would you give it to your friend? Your husband? Your father? I guess the answer will be "No" in the most cases and these people are most likely people you trust. Well, I know, you could state similar things for Googlemail who crawl your mails to show you personalized advertising. And in fact you are right. But if one decides to use gmail for your email hosting you have to trust them anyway. Like you have to trust anybody else you retain to host your mails (who could do the same but just don't tell you). But in this case its a third party, Facebook.

But if you now think that this becomes a rant against my friends: You are wrong. Over the day I found out that it basically shows up all my friends in a rotation. Every time I open the start page it randomly picks three people from my friend list and shows them to me, telling the same lie to me all over again.

So what do we have here? Facebook tries to advertise the most dubious feature they have in the most aggressive way one could imagine. By pretending wrong facts. Wouldn't that even be an element of crime in Germany ("Irref hrung" 5 UWG or maybe also 4 UWG, "unsachliche Beeinflussung")?

Facebook, you can do better.

Update: Stefano raised a good point. I didn't actually make clear that it has been verified that those people actually did not use the "feature":

1. I asked some of them. They said, they didn't use it and were in fact surprised that I asked.
2. Others told me they saw that ad stating that I used the feature. I definitely did not use this "feature".
3. Probably weak: IMHO its highly unlikely that all of my friends used that feature and at the point I've written this it had already shown my whole friend list.

I just discovered that bug report over at Ubuntu.
Short summary:
They have a script in upstart which is not meant to be run manually and if you do it will erase your whole file system. Additionally it seems that the fact that you shall not run that script is not communicated anywhere.

That alone isn't the most spectacular about it. Bugs happen. Whats spectacular about it is how a Canonical employee and member of the TechBoard (for people who don't know it: The people who decide about the technical direction Ubuntu takes) handles that bug. One quote of him to reflect it all:

Sorry, the only response here is "Don't Do That Then"

So what we have here is a classical case of bad programming. The problem in question is that the script expects a certain environment variable to be set. Fair enough. However it does not check if its set at all and instead of failing or using a sensible default it simply sticks to undefined behaviour. What we have here is a classical programming mistake every beginner tends to do. People who start programming often forget (or don't know) that every external value we rely on must be considered untrustworthy. Therefore a good practice is to check those values.In this case someone decided that this is useless because they suffer from the wrong assumption that nobody ever calls it manually and the other wrong assumption that caller of the scriptwill always set the environment variable correctly. This is a double-fail.
Now the developer in question does not accept that (someone else indicated why the behaviour of the script is dangerous), he simply says that the bug is invalid. Thats really a pity.

Recently I decided to invest in another harddisk for my atom system. That system, I built up almost two years ago, has become the central system in my home network, serving as a fileserver to host my personal data, some git repositories etc., streaming server and since I switched to a cable internet connection it also serves as a router/firewall.Originally, I bought that disk to backup some data, of the systems in the network, but I realized that all data on this system were hosted on a single 320GB 2,5" disk and it became clear to me, that, in absense of a proper backup strategy, I at least should provide some redundancy. So I decided, once the disk was in place, that the whole system should move to a RAID1 over the two disks. Basically this is not that hard as it may seem at a first glance, but I had some problems due to a new sector size in some recent harddisks, which is called Advanced Format. But lets begin from the start. The basic idea of such a migration is:

1. Install mdadm with apt-get. Make sure to answer 'all' to the question which devices need to be activated in order to boot the system.
   
   
2. Partition the new disk (almost) identical.Because the new drive is somewhat bigger that wouldn't make sense, but at least the two partitions which should be mirrored on the second disk, need to be identical.Usually this is achieved easily by using

   sfdisk -d /dev/sda sfdisk /dev/second/sdb

   In this case, it wasn't that easy. But I will come to that in a minute.
   
   
3. Change the type of the partitions to 'FD' (Linux RAID autodetect) with fdisk
   
   
4. Erase evidence of an eventual old RAID from the partitions, which is probably pointless on a brand-new disk, but we want to be sure:

   mdadm --zero-superblock /dev/sdb1mdadm --zero-superblock /dev/sdb2

5. Create two DEGRADED raid1 arrays from the partitions:

   mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 missingmdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sdb2 missing

6. Create filesystem on the first raid device, which will become /boot.
   
   
7. Mount that filesystem somewhere temporary and move the contents of /boot to it:

   mount /dev/md0 /mnt/somewhere

8. Unmount /boot, edit fstab to mount /boot from /dev/md0 and re-mount /boot (from md0)
   
   
9. Create mdadm configuration with mdadm and append it to /etc/mdadm/mdadm.conf:

   mdadm --examine --scan >> /etc/mdadm/mdadm.conf

10. Update the initramfs and grub (no manual modification needed with grub2 on my system)and install grub into the MBR of the second disk.

    update-initramfs -uupdate-grubgrub-install /dev/sdb

11. The first point to pray: Reboot the system to verify it can boot from the new /boot.
    
    
12. Create a physical volume on /dev/md1:

    pvcreate /dev/md1

13. Extend the volume group to contain that device:

    vgextend <volgroup_name> /dev/md1</volgroup_name>

14. Move the whole volume group physically from the first disk to the degraded RAID:

    vgmove <volgroup_name> /dev/md1</volgroup_name>

    (Wait for it to complete... takes some time ;)
    
    
15. Reduce first disk from the VG:

    vgreduce <volgroup_name> /dev/sda2</volgroup_name>

16. Prepare it for addition to the RAID (see step 3 and 4) and add it:

    mdadm --add /dev/md0 /dev/sda1mdadm --add /dev/md1 /dev/sda2

17. Hooray! Watch into /proc/mdstat. You should see that the RAID is recovering.
    
    
18. When recovery is finished pray another time and hope that system is still booting with it running from the RAID entirely. If it does: Finished :-)

Now to the problem with the advanced format:There is some action taking place with the hardware vendors to move to a new sector size. Physically my new device has a size of 4096 bytes per sector. Somewhat different to the 512 bytes disks used to have the last decade. Logically it still has 512 bytes per sector. As far as I understand this is achieved by placing 8 logical sectors into one physical sector, so when partitioning a new disk the alignment of the disk has to be so that partitions start in a sector which is a multiple of 8. That, obviously, wasn't the case with the old partitioning on my first disk. So I had to manually create partitions by specifying start points manually and making sure they are dividable by 8.Otherwise fdisk would complain about the layout on the disk.This does not work with cfdisk, because it does not accept manual alignment parameters and unfortunately the partitions it creates do have a wrong alignment. So good old fdisk and some calculations how many sectors are needed and where to start, to the rescue. So the layout is now:

Device Boot Start End Blocks Id System
/dev/sdb1 2048 291154 144553+ fd Linux raid autodetect
/dev/sdb2 291160 625139334 312424087+ fd Linux raid autodetect

Last night, the text below went out to r-sig-finance along with updates to the R/Finance website and its Call for Papers page; followed by some tweeting and Goggle+'ing (and please do feel free to retweet and share at will...)

Call for Papers: R/Finance 2012: Applied Finance with R
May 11 and 12, 2012
University of Illinois, Chicago, IL, USA
The fourth annual R/Finance conference for applied finance using R will be held on May 11 and 12, 2012 in Chicago, IL, USA on the campus of the University of Illinois at Chicago. The two-day conference will cover topics including portfolio management, time series analysis, advanced risk tools, high-performance computing, market microstructure, and econometrics. All will be discussed within the context of using R as a primary tool for financial risk management, portfolio construction, and trading. Over the past three years, R/Finance has included attendees from around the world and featured keynote presentations from prominent academics and practitioners. We anticipate another exciting line-up for 2012 --- including keynote presentations from Blair Hull, Paul Gilbert, Rob McCulloch, and Simon Urbanek. We invite you to submit complete papers or one-page abstracts (in txt or pdf format) for consideration. Academic and practitioner proposals related to R are encouraged. We welcome submissions for full talks, abbreviated "lightning talks", and for a limited number of (longer) pre-conference seminar sessions. Presenters are strongly encouraged to provide working R code to accompany the presentation/paper. Data sets should also be made public for the purposes of reproducibility (though we realize this may be limited due to contracts with data vendors). Preference may be given to presenters who have released R packages. Travel and accommodation grants may be available for selected presenters at the discretion of the committee. In addition, the conference will award prizes for best papers. To be eligible for a best paper award, a submission must be a full paper. Extended abstracts, even if a full paper by conference time, are not eligible for a best paper award. Please send submissions to: committee at RinFinance.com. The submission deadline is January 31, 2012. Submitters will be notified of acceptance via email by February 28, 2012. Notification of whether a presentation will be a long presentation or a lightning talk will also be made at that time. Additional details will be announced at this website as they become available. Information on previous year's presenters and their presentations are also at the conference website R/Finance 2009, 2010 and 2011. For the program committee:

Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson,
Dale Rosenthal, Jeffrey Ryan, Joshua Ulrich

So see you in Chicago in May!

The Good Every other year or so I feel the need to clean up my Debian system and remove the installed packages I m not interested in anymore. I remember there was a nice aptitude pattern to search for packages which I have manually installed (i.e. which were not installed to satisfy a dependency). Ideally I would then go through the (presumably short) list of packages and remove the ones I don t need any more. Since I always forget the aptitude pattern to search for those packages, I google for something like list of manually installed packages and find a solution like: aptitude search ~i !~M . Although this solution is not wrong, it is not quite what I was looking for. Sure, it will find you all packages which are installed and not installed to satisfy a dependency, but it also contains packages of priorities: required, important and standard which you usually don t want to deinstall. So the actual solution should also exclude those packages.

aptitude search '~i !~M !~pstandard !~pimportant !~prequired'

The above line will do the trick and present you a list of packages which are: installed and not dragged in to satisfy a dependency and not of priority standard, important or required. If you where only interested in the solution of the quest of getting a list of all manually installed packages you can stop reading here. The Bad If you re running your system for longer than, say, a few weeks you will probably notice that the list is very long and contains much more packages than you actually installed manually. You will probably notice a lot of libs and other packages in this list and wonder why they aren t marked auto as they should since you didn t install them on purpose. Unfortunately I don t have an answer for that question. I m using aptitude exclusively to install packages and rely on the fact that aptitude does install the dependencies automatically and more importantly: also removes them if not needed any more. But it seems like something is broken in aptitude (or somewhere else) and packages simply lose their auto status instead of being removed and thus stay on your system forever if you don t manually clean your system. The Ugly To solve that issue I started aptitude and filtered the package tree with the above pattern (press l in aptitude and enter ~i !~M !~pstandard !~pimportant !~prequired). Now the package view shows only packages which are safe to remove since they are not of of priority standard or higher and manually installed by you. Now I navigate with the cursor to the list of installed packages an mark all of them as auto by pressing M. For each package in this list, aptitude will try to find an installed package which depends on that package and mark that package auto. The remaining packages which no other packages depend on, aptitude will mark for deletion. If you now press g one time, aptitude will show you which packages will be removed. I then go through that list and mark every single of the remaining packages I really want on my system with + as manually installed. After I m done I proceed by pressing g again which will remove all remaining packages. In my case nearly 600 packages (1GB) where removed. Bugreport! I assume there is a bug in aptitude or some related package which sometimes(!) removes a packages auto status instead of removing it. Unfortunately I did not find a way to reproduce that (except that it obviously does happen). If someone could find a way to reproduce that problem, one could fill a bugreport and help to get that bug fixed. Right now, a Debian system suffers (like many other operating systems) from a slow but steadily growing bloat of installed software. In my case 600 packages with 1GB accumulated over one or two years.

The route cache is a Linux kernel component enabling route lookups to be faster by caching the results in some table and checking it before issuing a regular lookup in the route tables. When using Linux as a router, the inefficiency of the route cache can hinder the performances of your box. The documentation on this component is scarce and it is difficult to find up-to-date bits on how the route cache works and how to tune it. The book Understanding Linux Network Internals from O Reilly is an exception and contains valuable information on how the route cache works. Even if the book is targeted at 2.6.12, the part on the route cache is still quite accurate. Unfortunately, it fails to provide appropriate tips on how to monitor and tune the route cache. I hope to provide here a concise view of the route cache, as it is implemented in Linux 3.1. It is protocol-dependent¹ and I will cover only the IPv4 version here.

Overview of the route cache subsystem To handle an incoming or outgoing IP datagram, the kernel needs to issue a lookup in the route tables. While it seems to be quite trivial, several questions need to be answered:

• Does the source address and the destination address appear to be valid?
• Is the source address a martian address²?
• Is the destination address mine or should I forward the packet?
• Which route table should I use?
• Does the destination address match this route? And this one?
• Can I currently contact the gateway that I should use?

Those checks can be a bit time consuming, even for small route tables. To avoid them for each packets, Linux maintains a route cache which is queried before doing a regular lookup and updated after each one. You can dump it with ip -s route show cache:

$ ip -s route show cache
198.51.100.7 from 203.0.113.2 via 192.0.2.1 dev eth1
  cache used 7 age 2sec ipid 0x1fce rtt 131ms rttvar 45ms cwnd 10
198.51.100.17 from 203.0.113.15 via 192.0.2.1 dev eth1
  cache used 3 age 0sec ipid 0xc3bd
local 192.0.2.18 from 203.0.113.15 dev lo src 192.0.2.18
  cache <local> used 154 age 1sec iif eth0

Here are two examples:

• If Linux receives a packet from 198.51.100.17 to 203.0.113.15, it will find this flow in the route cache. Therefore, it already knows that it should forward the packet to 192.0.2.1. No checks needed.
• If it receives a packet from 198.51.100.16 to 203.0.113.15, there is no appropriate entry in the route cache and therefore, the system will have to look at the route tables. It is likely to use the exact same entry than if the source was 198.51.100.17 but maybe there is some policy routing requesting the use of a special route table or 198.51.100.16 is a local address and the packet will therefore be classified as martian.

The schema below shows how this cache is implemented³. It uses a separate hash table (BSD systems keep the cache in the routing table). Each bucket is a chained list of route cache entries. Once an entry has been added to the route cache, there are several ways to remove it. Most entries are removed by the garbage collector which will scan the route cache and remove invalid and older entries. It will be triggered when the route cache is full or at regular interval, once a certain threshold has been met.

Available knobs There are several values you can tune. Most of them are available in sysctl.

• rhash_entries is the size of the hash table⁴. If you don t specify it on the kernel command line, it is computed dynamically based on the memory available on your system. You can view its value by looking at something like IP route cache hash table entries: 262144 (order: 9, 2097152 bytes) in the kernel logs.
• net.ipv4.route.max_size is the maximum number of entries in the route cache. Except under exceptional circumstances, this value is never exceeded.
• net.ipv4.route.gc_elasticity is the target average length of a chain in the route cache. The garbage collector will work harder if this value is exceeded. This means that if you multiply this value by the value of rhash_entries, you will get the target average number of entries in the route cache.
• net.ipv4.route.gc_min_interval_ms is the minimum delay between two runs of the garbage collector, except when the cache is full. The default value should be fine.
• net.ipv4.route.gc_thresh is a threshold triggering the garbage collector every net.ipv4.route.gc_min_interval_ms milliseconds.
• net.ipv4.route.gc_timeout is the base value to determine if an entry is old enough to be removed or not. Whatever its value, the garbage collector will attempt to remove the same number of entries. However, this value could potentially influence its efficiency. See below for more details on this.

You may find documentation about those obsolete sysctl values:

• net.ipv4.route.secret_interval has been removed in Linux 2.6.35; it was used to trigger an asynchronous flush at fixed interval to avoid to fill the cache.
• net.ipv4.route.gc_interval has been removed in Linux 2.6.38. It is still present until Linux 3.2 but has no effect. It was used to trigger an asynchronous cleanup of the route cache (but in a different way of what the garbage collector does). The garbage collector is now considered efficient enough for the job.

Statistics & monitoring Linux maintains some statistics about the use of the route cache. You can find them in /proc/net/stat/rt_cache. The command lnstat can print them nicely for you:

$ lnstat -s1 -i1 -c-1 -f rt_cache
rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache rt_cache 
 entries   in_hit in_slow_ in_slow_ in_no_ro   in_brd in_marti in_marti  out_hit out_slow out_slow gc_total gc_ignor gc_goal_ gc_dst_o in_hlist out_hlis 
                       tot       mc      ute            an_dst   an_src              _tot      _mc                ed     miss  verflow  _search t_search 
 3096848    42309      686        0        0        0        0        0        3        0        0      674      672        0        0    27644        8 
 3096984    41405      636        0        0        0        0        0        3        0        0      623      621        0        0    28189        8 
 3097160    42483      700        0        0        0        0        0        5        0        0      694      692        0        0    29506       12 

Except for the first column, lnstat outputs values in units per second. Let s review some of those values:

• rt_cache_entries is the number of entries in the route cache. You should compare it with net.ipv4.route.max_size and ensure that the cache is never full to avoid triggering the garbage collector too often.
• rt_cache_in_hit and rt_cache_out_hit are the number of regular lookups avoided because the result was found in the cache for incoming and outgoing packets, respectively. When the system is a router, most lookups only happen on the incoming side. You should compare this value with rt_cache_in_slow_tot and rt_cache_in_out_slow_tot which are the number of lookups in the route tables. On this system, the efficiency of the route cache is more than 98% which is quite good.
• rt_cache_gc_total is how often the garbage collector was requested to be triggered while rt_cache_gc_ignored corresponds to how often it was finally not run because it has already been triggered shortly beforehand (less than net.ipv4.route.gc_min_interval_ms milliseconds). The difference between those two values should be very small to ensure that the garbage collector does not run more a handful of times per second.
• rt_cache_gc_goal_miss is how often the garbage collector was not able to fulfill its goal. This value should rarely be different of 0.
• rt_cache_gc_dst_overflow is how often the route cache is bigger than the allowed maximum size. This should never happen except when you try to shrink the cache.
• rt_cache_in_hlist_search and rt_cache_out_hlist_search is how often a lookup in the cache had to look at the next entry in the chain for the computed bucket: each time Linux has to follow the next pointer in a cache entry, it increments one of those counters. It is a clue on the average length of chains in the route cache hash table. Compare those values with the number of cache lookups (hit and miss).

As an illustration, here is a plot of the various statistics exposed above for a router whose route cache receives about 1000 routes par second: rhash_entries is 1,048,576, as is net.ipv4.route.gc_thresh. Therefore, the garbage collector was requested to be run when the number of entries goes above this level. Because the cache is not full, it is only really run twice per seconds (the value of net.ipv4.gc_interval_ms is 500). net.ipv4.gc_elasticity is set to 3. This explains why the garbage collector is aggressive when the number of entries reached 3,145,728. As you can see, the efficiency is near 100% all the time. The percentage of collisions is rt_cache_in_hlist_search ratio to the sum of rt_cache_in_hit and rt_cache_in_slow_tot.

Tuning Do you need to modify any of those values? You have two questions to ask yourself:

1. How much efficiency do you want to get from the route cache?
2. How much memory do you want to dedicate to the route cache?

As a rule of thumb, two millions entries eat about 500 MB of memory on a 64bit system. You should be able to compute the average memory usage and the maximum memory usage from the values of net.ipv4.route.max_size, rhash_entries and net.ipv4.route.gc_elasticity. For example, if the route cache hash table has 262,144 buckets, the maximum allowed number of entries in the cache is 4,194,304 and net.ipv4.route.gc_elasticity is set to 8, the memory usage will be 500 MB on average and 1 GB max. If this is too much, you will need to lower some values. Look at the previous section to compute the current efficiency of your route cache. It should be above 90%. If you are dissatisfied with that, you could increase the cache size. If you want to double the number of entries, double net.ipv4.route.max_size, net.ipv4.route.gc_thresh and rhash_entries but keep net.ipv4.route.gc_elasticity to 8. For the garbage collector to be efficient, the route cache should not be filled too fast. The garbage collector should be able to cope with this situation but this may impact performance because it needs to walk several times the route cache to find entries to expire. Watching gc_goal_miss may give you a hint about this: if this value starts to be different of 0, lower net.ipv4.gc_timeout.

In-depth look into the garbage collector Those different pieces of advice may have puzzled you. We need to understand how the garbage collector works to better cope with them. The garbage collector is triggered when a new entry needs to be added to the cache and the number of entries is superior to net.ipv4.route.gc_thresh. It is implemented in rt_garbage_collect() function. It will do nothing if it has been called less than net.ipv4.route.gc_interval_ms milliseconds and the cache is not full (more than net.ipv4.route.max_size entries).

Setting a goal The garbage collector will first assess the situation. It will look how the number of entries currently in the cache compares to the product of rhash_entries and net.ipv4.gc_elasticity:

1. Above this limit, it will try to remove at least rhash_entries entries.
2. Otherwise, it will try to remove no more than half the difference.

If you look at the previous plot, you can easily see the difference when the garbage collector is not aggressive (above 1,048,576 entries but below 3,145,728) and when it is (above 3,145,728 entries). If we assume the garbage collector is able to meet its goal, we can easily simulate its algorithm: The first plot shows what would happen if about 2000 routes are added per second with rhash_entries equal to 262,144 and net.ipv4.route.gc_elasticity set to 8. When net.ipv4.route.gc_threshold is met, the garbage collector has almost no effect. However, when the number of entries reaches 2,097,152, the garbage collector sets its goal to 262,144. From this point, the number of entries oscillates around 2 millions entries. On the second plot, rhash_entries is now equal to 1,048,576 but net.ipv4.route.gc_elasticity has been set to 2. Therefore, the aggressive part of the garbage collectors kicks at the same threshold than for the previous plot. However, its goal is now four times larger and the oscillations have a larger amplitude. The fact that the aggressive part of the garbage collector kicks less often is nullified by the fact that it needs to remove more entries each time. Because of the slight impact on cache efficiency, it seems better to keep net.ipv4.route.gc_elasticity around 8, or 4 if we want to keep shorter chains (but there seems to be no improvement to do so). The other plots show what happens when there is a sudden surge in the number of routes added or when there is a pause.

Meeting the goal Now that we understand how rhash_entries, net.ipv4.route.gc_elasticity and net.ipv4.route.gc_threshold interact, let s look at net.ipv4.route.gc_timeout. Once the garbage collector has set its goal and if it is positive, it needs to choose which entries to remove from the cache. It walks the hash table from the position of its last run and remove entries until its goal is met. If the entry is not current anymore (for example, the network interface associated to it has changed its IP configuration), it is removed. Otherwise, the system looks at the age of the entry and its position in the chain. If the entry is the first in the chain, it is kept only if its age is below net.ipv4.route.gc_timeout. If it is second, it is kept only if its age is below half of net.ipv4.route.gc_timeout. If it is third, the threshold is a quarter of net.ipv4.route.gc_timeout, and so on. The garbage collector will favor short chains. If after a full run of the hash table, the garbage collector was not able to meet its goal, it will start again but will behave more aggressively, as if net.ipv4.route.gc_timeout is set to half of its value. It will do as many passes as necessary until its goal is met or until there is no way to remove any entry (or until it has spent too much time). Once the garbage collector has switched to this more aggressive behavior, it will keep being aggressive for a few cycles (a bit less for each cycle). With a very large value of net.ipv4.route.gc_timeout, the garbage collector will have a hard time to find entries to expire. It will need to do several passes until it is able to expire some entries. On the other hand, if you choose a very small value, the garbage collector may remove entries that were just added, even if there are older entries further in the hash table. For a more in-depth coverage of how the route cache works, look at chapter 33 of Understanding Linux Network Internals. Just be aware that multipath route caching has been removed (and was never really used) and asynchronous cleanup (rt_periodic_timer) does not exist anymore.

Conclusion When tuning the route cache, rhash_entries, net.ipv4.route.gc_elasticity, net.ipv4.route.max_size and net.ipv4.route.gc_threshold are related and should not be modified independently. net.ipv4.route.gc_thresh should be below the product of net.ipv4.route.gc_elasticity and rhash_entries while net.ipv4.route.max_size should be above this value. Linux exposes several interesting metrics related to the route cache. Monitoring them allows one to watch the efficiency of the route cache and may uncover some anomalies (garbage collector running too often, difficulty to remove routes from the cache, ). The route cache subsystem still evolves and some old behaviors have been dismissed. The best documentation is, unfortunately, still the code and other documentations tend to become obsolete. The IPv6 route cache is quite different of the one presented here. Don t blindly apply the same tuning for IPv6.

---

1. Linux provides a protocol-independent destination cache subsystem (DST). This component is not a generic cache layer and only enables loose coupling with external subsystems.
2. Martian addresses are addresses that cannot be used as a source address, either because they are reserved for special-use (like a multicast address) or because of the use of reverse path filtering which checks if a packet received on one interface would be answered on the same interface, as defined in RFC 3704. This feature is enabled by setting rp_filter in Linux.
3. For more details, you may want to look at include/net/dst.h, include/net/route.h and net/ipv4/route.c.
4. In fact, the size of the hash table is always a power of two. If specified, rhash_entries is rounded to the next power of two and, internally, is stored as rt_hash_mask + 1.

Hello As Damyan mentioned last summer in Debconf, the current command interface to Config::Model Perl module is rather cumbersome as it involves quite a lot of typing. First, config-edit command name is too long. Even with the help bash completion, this does not compare well with git or hg command. Then all operation require options. For instance, the -application option is required to specify what you will be working on. A required option is a good example of an oxymoron. Which may be funny but which is not user friendly So, here s the synopsis of the new command I m working on:

# edit dpkg config with GUI (Tk or curses, or readline)
cfg edit dpkg 
# just check the validity of the file, show warnings
# and suggested fixes
# does not modify the file
cfg check dpkg
# check the file, remove deprecated parameters,
# migrate data to new parameters and save
cfg migrate dpkg 
# like migrate, but also apply all suggested fixes
cfg fix dpkg 
# like migrate and modify configuration
# with command line argument
cfg modify dpkg source format="quilt (3.0)"
# access dpkg parameters through a fuse file
# system. data are saved when fs is unmounted
cfg fusefs dpkg fuse-dir
# dump configuration data in config-model format
cfg dump dpkg   
# list all available applications
cfg list 
# more general synopsis
cfg [ global_options ] command application [ options ] arguments

This synopsis use dpkg as an example, but it could be used for any of the available models (e.g. ssh, lcdproc, multistrap ). The main question I have for you is: Is cfg name good ? I mean it s short, descriptive enough. But it s also quite ugly. Does anyone have a better idea ? And what do you think of the synopsis shown above ? Are the sub-command name descriptive enough ? All the best

(It s a new day, it s a new bug ) A regression appeared in the synaptics input driver, between the 1.4.1 and the 1.5.0 release, as reported in Debian #649002 by many users. Basically, no more touchpad on PowerPC (hi, iBook G4 users!). :-( What follows is how I tracked it down. The upstream bug report is fdo#43728. Anyone could have done so, there s no black magic involved. A little hint maybe: git bisect is really easy on X drivers, since one can build and test without the Debian packaging bits. 1st step: Getting started You need sources and build dependencies, nothing fancy. Since packages might be named a bit differently, there s a reference to the relevant upstream repositories in debian/watch for X packages.

# You need git here:
debcheckout xserver-xorg-input-synaptics synaptics.git
cd synaptics.git
# Let s get the upstream tags and branches:
cat debian/watch
git remote add upstream git://anongit.freedesktop.org/xorg/driver/xf86-input-synaptics
git fetch --all
# Let s install build dependencies:
sudo apt-get build-dep xserver-xorg-input-synaptics

2nd step: Reproducing It was said 1.5.0 was the first known buggy version, so let s make sure:

# Get the appropriate tag:
git checkout xf86-input-synaptics-1.5.0
# Prepare the build system:
autoreconf -vfi
# X finds its modules under /usr, not /usr/local:
./configure --prefix=/usr
# Build and install:
make && sudo make install

Now time to restart the display manager, and confirm that the pointer is not moving bug reproduced. Now let s check the 1.4.1 release isn t affected:

# Clean everything:
git clean -xdf
git checkout xf86-input-synaptics-1.4.1
autoreconf -vfi
./configure --prefix=/usr
make && sudo make install

And after a display manager restart, the pointer is moving again, good! For reference, the log has very different lines about appletouch, so it can be used to programmatically learn about the status (good or bad). 3rd step: Narrowing it down So, we have a known good version and a known bad version. Time for a binary search to find the guilty commit which introduced that regression!

git bisect start
git bisect good xf86-input-synaptics-1.4.1
git bisect bad  xf86-input-synaptics-1.5.0

You don t even need to know that 1.4.x and 1.5.x live in different branches, git figures that out for you:

Bisecting: a merge base must be tested
[de0dfb76444ad4160468d00515876c91a9fa20bf] synaptics 1.4.0

It s just a matter of running autoreconf && ./configure && make && sudo make install && sudo /etc/init.d/xdm restart every step of the way, so it s pretty easy. No wonder, 1.4.0 was working OK, so it can be recorded as such through git bisect good, and the binary search goes between that commit and 1.5.0. After a few iterations of git bisect good and git bisect bad, we get to the commit reported on the upstream bug. As an extra bonus, mentioning the upstream bug number/URL in the Debian bug means it can be marked as forwarded there and we ll receive automatic notifications when its status changes, thanks to bts-link. Now, if you want to provide a patch for this bug, you may want to try and revert this commit. 4th step: Providing a patch Let s go back to the affected release and try to revert the guilty commit:

git checkout xf86-input-synaptics-1.5.0
git revert 13543b156d78bc4d01a19844a5ee8f283269621b

Unfortunately, some later commits modified the affected areas so we re running into conflicts:

error: could not revert 13543b1... eventcomm: replace synaptics-custom TEST_BIT with server's BitIsOn.
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'

Let s try to understand what the offending commit did: it replaced a set of macros with a single one, taken from the server. Both the old TEST_BIT() and the new BitIsOn() take 2 arguments, but the order is reversed. Reverting it should only be a matter of introducing the TEST_BIT() macro again (along with the two other ones it needs), and using it. The problem is replacing all BitIsOn(foo,bar) with TEST_BIT(bar,foo): foo and bar might not be trivial, they could be complex C expressions, and a replacement in your favourite editor might not get all occurrences right. Thankfully, there s a nice tool to handle such things, called coccinelle. I won t go into details, just show how it can help here. Basically you just need to craft a tiny patch, called a semantic patch, which describes what I wrote in the previous paragraph. You only need to declare two expressions, say a and b, and declare that all BitIsOn(a,b) must be replaced with TEST_BIT(b,a). Here s the syntax:

@@
expression a,b;
@@
-BitIsOn(a,b)
+TEST_BIT(b,a)

Save it under testbit.cocci and apply it through spatch (from the coccinelle package), asking it to perform in-place replacement (we re in a git checkout copy, remember):

# Apply:
spatch -sp_file testbit.cocci -in_place .
# Profit:
git diff

Then you can run git commit -s, write a nice commit message, send it upstream, and enjoy the rest of the night. Coming soon to a mirror near you: fixing commit. As mentioned on the upstream bug report, I got the Debian reference wrong in the commit message, I really meant #649002.

Mako's IronBlogger is a great idea. I often find myself postponing blog posts for a very long time, simply out of laziness. IronBlogger provides a nice community incentive to counter (my) laziness and blogging more often. As a related challenge, we have to face the fact that different subsets of our communities use different media to stay informed: mailing lists, blog (aggregators), social media, IRC, etc. Disparities in how they stay informed are a pity and can be countered using multiple medias at a time. Although I haven't blogged very often as of lately, I managed to keep the Debian (Developer) community informed of what happens in "DPL land" on a monthly basis, by the means of bits from the DPL mails sent to d-d-a. While the target of bits mails perfectly fits d-d-a, there is no reason to exclude a broader public from them. After all, who knows, maybe we'll find the next DPL victim^W candidate among Planet readers! Bonus point: blogging this also helped me realize that my mails are not as markdown-clean as I thought they were. I still have no IronBlogger squad, though. (And sharing beers with folks in the Boston area is not terribly handy for me ). Anyone interested in setting up a BloggeurDeFer in the Paris area? (SCNR)

---

Dear Project Members,
another month has passed, it's time to bother you again about what has happened in DPL land in November (this time, with even less delay than the last one, ah!). Call for Help: press/publicity team I'd like to highlight the call for help by the press / publicity teams. They are "hiring" and have sent out a call for new members a couple of weeks ago. The work they do is amazing and is very important for Debian, as important as maintaining packages or fixing RC bugs during a freeze. It is only by letting the world know what Debian is and what we do, that we can keep the Project thriving. And letting the world know is exactly what the publicity and press teams do. If you're into writing, blogging, or simply have a crush for social media, please read the call and "apply"! Interviews November has apparently been the "let's interview the DPL" month. I've spent quite some time giving interviews to interested journalists about various topics. For both my embarrassment and transparency on what I've said on behalf of Debian, here are the relevant links:

• audio interviewwith Karen Sandler for the Free as in Freedom oggcast by herself and Bradley M. Kuhn
• text interview with Raphael Hertzog for the People behind Debian series
• text interview for the Linux Certification Institute Italy (LICI) (note: the original of this one is in Italian; the English version is a translation by LICI)
• video interview with Amber Granger during last UDS

Assets

• the Video Team sprint approved in October has happened at beginning of November; report is still pending but the organizers are on it
• Christian Perrier reported about his trip to India mini-DebConf, even that he attended on behalf of the Project. I find amazing both the outcome of the event and Christian's report, a very good example of how to inform the Project when acting on behalf of it
• the hardware replacement for ftp,lists .d.o I've mentioned last month has been ordered, has arrived, and DSA have already put their hands on it for the setup. Many thanks to DSA and in particular to Martin Zobel-Helas for the logistics
• as an administratrivia, I've asked SPI to setup for Debian a "gift" credit card, i.e. a credit card with a limited, refillable budget. It is needed for using (with a safeguard) credits offered to the Project by public cloud providers (e.g. AWS' donation), as well as to use various kinds of services that we need from time to time

Legal advice (work in progress)

• I've sought and got from a former SPI lawyer a first legal advice on the relationships among trademark law and the procedures of a Free Software distro like Debian when distributing software subject to trademarks. I'm still waiting to seek a second advice from the current SPI lawyers (whom I've been keeping busy with other stuff). In the meantime I've pending a summary of what I've got thus far in the trademark-vs-DFSG thread. Stay tuned there!
• I've been working with SPI lawyers on a proposal of software patent policy that I think we should advertise to make explicit the position of the Debian Project on software patents. I see it as a continuation/completion of the FAQ we have already published on the subject. I'll let you know as soon as we reach a stable draft, that should be pretty soon.

Relationships with others

• I've been approached by members of the board of the OpenSource Initiative (OSI), about the soon to be open OSI affiliation program. They'd like to know if Debian is interested in being affiliated to OSI. I think we should and I'll eventually start a discussion on this topic. In the meantime, I welcome feedback from anyone who is interested in the topic.
• Thanks to the interest of Andrew Pollock, LaMont Jones, Florian Weimer and the Security Team, we are discussing with ISC to have Debian --- as a project, rather than as individuals who just happen to maintain ISC software in Debian --- become member of all forums relevant for software we distribute (BIND, NTP, DHCP). They are doing all the coordination work, but I've anticipated I'll be happy to pledge for Debian membership.
• I've taken the first step to reach out to companies that have a strategic interest in Debian. According to discussion I've had with representative of such companies, they often face similar challenges in offering Debian support and in contributing back to Debian. I think we should offer a forum for them to discuss and find synergies, as well as explain our needs to them. An official mailing list might be such a place. Once we have a suitable forum ready, I'll propose to the press team to send out a corresponding call for companies. In the meantime, if you happen to know, or even work for, companies that might be interested, please ask them to get in touch with me.

Miscellanea

• I've had a chance to discuss with the editors of the SPDX specification about its relationship with DEP-5. Thanks to the help of Dominique Dumont work is already in progress to have a mutual converter among the two formats. Also, although with some bumps, work on finalizing DEP-5 has finally restarted.
• an interesting thread of a while ago is in need of some work to produce a useful outcome, i.e. a set of security guidelines for DDs. If you're willing to help, have a look at the thread
• some more mediation-fu has been going on during November; have a look at my daily logs below if you care about the details

Thanks for reading thus far,
and happy hacking.

---

PS as usual, the boring day-to-day activity log is available at master:/srv/leader/news/bits-from-the-DPL.*