At least 150 people were reported killed today in an air crash at Madrid. Rest in peace, all of you will be missed.

You can say that DebConf8 is officially is over for you, when you're finally home, after three weeks in Mar del Plata; go to a primary school friend's house to celebrate his birthday, and end the day watching a movie alone. It's also when you delete the reminder for the two weekly IRC meetings, when you have to go back to classes, convince your boss not to kill you because of all the work hours spent on organisation, make your pet cat recognise you again, and be able to make a blog post about it. What can I say about DC8? It's been madness, rush, fun, tiredness beyond belief, intoxication, mao, meetings every day with the hotel, friends that you get to see only once a year, musicians out of key, and pride. Pride of seeing people enjoying themselves, things going more or less as were planned, and being part of a group of people that worked hard to make this dream come true: thanks to all of you! One little thing I brought from MDQ is a copy of In Bruges, a movie I didn't know anything about, but whose title caught me. That's the movie I've just seen and loved because it brought me memoirs of Bruges and Belgium. City and country I visited last year after DebConf, and fell in love with. That's something else that DebConf has been for me: the opportunity to visit countries far away from mine, and not wanting to ever stop. Anyway, I still want to see some fellow debianites before they go to their respective countries. So a barbecue will take place at my home this weekend, just to compete with Steve's. If you're still around, just talk to me to arrange details and eat a little more beef if you hadn't got an overdose yet! By the way, I totally fell in love with Cl mence Po sy after this movie :-) Tags: Planet Lugfi, Planet Debian

Update: There are more notes on how aptitude performs as a Sudoku solver here. Russell Coker recently wrote a post entitled Ownership of the Local SE Linux Policy. This post has nothing to do with the substance of his post, which is a discussion of how distributions should configure SELinux by default. I know nothing about SELinux, but something that Russell said in passing caught my attention:

I am not aware of the Debian package dependencies (or those of any other distribution) being about to represent that the postfix package depends on selinux-policy-default-postfix if and only if the selinux-policy-default package is installed. Please note that I am not suggesting that we add support for such things, a package management system that can solve Sudoku based on package dependency rules is not something that I think would be useful or worth having.

(emphasis added) As it happens, a little-known fact about the Debian packaging system is that you can, in fact, describe Sudoku puzzles in it! Note for the impatient: If you want to skip all this talk and jump to the code, see debsudoku.py. For those of you who, like me, are not Sudoku-heads, here's a quick summary of the rules (for the common case of a game with 3 blocks):

1. The game is played by filling in a 9-by-9 grid with numbers between 1 and 9, inclusive. The board is divided into 9 3-by-3 blocks of grid cells in the obvious way.
2. All the cells in a column must contain different numbers.
3. All the cells in a row must contain different numbers.
4. All the cells in a block must contain different numbers.

There are many ways that you could convert a Sudoku problem into a Debian package archive, but here's a particularly simple one that is close to my summary of the rules. Create 9 * 9 * 9 = 729 packages, named cellR.C-V where R stands for row, C for column, and V for value. Each of these packages represents writing the value V in the cell (R, C). To describe the relationships of each cell to the other cells in the problem, we can create several virtual packages:

• The virtual package cellR.C represents the statement The cell (R, C) has been filled in with some value. It will be used to ensure that only one value is placed in any particular cell, and to ensure that all the cells in the puzzle are filled in with something.
• The virtual package blockR.C-V represents the statement A cell in the block (R, C) has been filled in with the value V. The block coordinates are the block-row and block-column containing the cell; for instance, the cell at (5, 8) is in block (2, 3). This package will be used to ensure that only one cell in a given block contains the same value.
• The virtual package rowR-V represents the statement A cell in the row R has been filled in with the value V. It will be used to ensure that no two cells in any given row have the same value.
• The virtual package colC-V represents the statement A cell in the column C has been filled in with the value V. It will be used to ensure that no two cells in any given column have the same value.

Converting these requirements to package dependencies is quite straightforward. For the package cellR.C-V, in block row BR and block column BC, we add these lines to the package's definition:

Provides: cellR.C, blockBR.BC-V, rowR-V, colC-V
Conflicts: cellR.C, blockBR.BC-V, rowR-V, colC-V

For instance, the cell at (5, 8) will get these provides/conflicts for V=2:

Provides: cell5.8, block2.3-2, row5-2, col8-2
Conflicts: cell5.8, block2.3-2, row5-2, col8-2

This relies on the fact that self-conflicts are ignored (see Debian Policy section 7.4). To actually describe the puzzle we want to solve, we need to add another package that requires two things:

1. Every cell contains a value.
2. The cells that the problem fixes have their fixed values.

For instance,

Package: puzzle
Depends: cell1.1, cell1.2, ..., cell5.9-3, cell6.1, ....

if we want to require that the cell at (5, 9) contains the value 3. Proof-of-concept As a proof-of-concept, I have written a hacky Python script, named debsudoku.py, that can convert ksudoku saved games into Packages files suitable for use with apt-get or aptitude. In fact, it has two modes: conflicts, which implements the conversion described above, and depends, which implements a different but logically equivalent conversion using only Depends instead of Conflicts. To use the script, save a ksudoku board without doing any work on it, then run

python debsudoku.py /path/to/the/ksudoku/board

I've only done minimal testing on this, so don't use it to fly airplanes (in fact, please don't use any Sudoku player to fly airplanes), but I did use it, in combination with aptitude, to solve a single ksudoku puzzle. Real-world evaluation Interestingly, in a test with an easy puzzle generated by ksudoku, aptitude was able to solve the puzzle in just a few seconds when it was expressed with the conflicts reduction, but failed to find a solution at all using depends (I stopped it after several minutes). I think this is due to how the depends reduction is performed. Take rows: instead of generating conflicts between cells in a row, it requires that each row contain every number between 1 and 9. The problem with this is that aptitude's resolver functions poorly when there are non-obvious conflicts in the dependency structure. It will decide to try a combination of packages that can't be installed together, but it won't realize it. As a result of this, it ends up trying many possible combinations of packages. For instance, it might try to place the number 3 in both (2, 5) and (5, 5). There is no dependency outlawing this in the depends reduction, so aptitude will happily continue trying to resolve dependencies. Since it has used 3 twice in the fifth column, it will not be able to find any way to resolve all the dependencies that state that all the numbers between 1 and 9 appear in the 5th column, but it lacks the higher-level reasoning that would allow it to break out; it will end up trying many or most of the possible ways of filling cells in before it gives up on the offending re-use of 3. Luckily, real-world package archives (what the aptitude resolver is designed to run on) don't usually look like this.

I have gotten quite some response to my blog post about the hardware test proposal thingy from Bdale. It would seem I haven't been entirely clear Someone referred me to a page by Kenshi Muto that parses 'lspci' output into a hardware compatibility list. This doesn't help. It ignores x.org stuff (which is important, too) and the page itself clearly says that it cannot guarantee whether a piece of hardware will actually work. For clarity: when I said 'Apparently all the other vendors have it, too', I really meant 'Apparently all the other GNU/Linux vendors have it, too'. People have suggested some cooperation between kernel/xorg upstream, other vendors, and perhaps something like freedesktop.org; and while that may be a good idea in and of itself, in the end Debian will have to provide something which it will call 'official' and which will tell a vendor whether or not Debian is supported on their hardware. This thing may not be fully automated or polished; it may need interactivity; but it must be something which will give a result that a product manager may want to lose some sleep over if it's not good enough. Holger also suggested we try implementing this with Debian Live, rather than debian-installer. This may be a good suggestion; a debian-live image will have a full Debian system available rather than the somewhat limited d-i environment, so writing a test should be easier. Also, I don't even want to think how testing X.org drivers would work out from within d-i. So what's left is a way to figure out how to do such a live system. I'll be posting some suggestion to the debian-devel mailinglist 'soon'.

Some interesting two days have just passed. I took the airplane from Mar del Plata to Buenos Aires on the 17th, slept one night in a hostel on a mattress that felt like stone, and went to the 'Circulo del Officieles de la Mar' (or some such, I don't know spanish) where I had to give a talk. The oral feedback I've gotten so far indicates that people did appreciate it, which is good. I had originally planned to do some sightseeing in Buenos Aires after my talk, but me being tired and not immediately finding anyone else to go with me changed those plans. Instead, I just stayed at the C.O.M., where we played some Mao. Amongst other things. About an hour before I left the place, Holger Levsen gave me a tripod, with the request to take it with me to Belgium, and keep it until FOSDEM. As that would also give me the use of an tripod, and being the photography hobbyist that I am, I was too happy to oblige. There was just one problem: the suitcase that contained my luggage already had a bulge from here to tokio, so adding it was a no-no. No problem, I thought. I'll just keep it in my carry-on luggage. There's some space left there, and I certainly didn't have 5kg of carry-on luggage yet. Problem solved. The 'security' in EZE is a joke; the security officer doesn't even look at the screen while X-raying my luggage. Well done. So I fly from EZE to MAD on the first, 11-and-a-half-hour leg of my flight. Madrid Barajas is a crazy airport, with its own private metro. Somewhere between gates R-something (where IB6844 from EZE arrived) and J40 (where the flight to BRU will depart from), we need to go through a security checkpoint. Strange, since I hadn't had to do that on my way in from BRU to EZE; but of course you have no choice, so I comply. As I've gotten used to by now, what with the mess of cabling in my laptop bag and the other metal objects there, I get a baggage check. The security officer who goes through my very messy bag asks me to open the bag containing the tripod, so he can see it. He then promptly decides that it is a "sharp" object, and that it is "not acceptible". Here the fun begins. Of course, trying to argue with a security guard isn't the best idea if you want to be allowed on any flight, ever, so I don't even try that. I ask him 'can I check that in, instead?' and he tells me how to go back, to the exit, and to the departures hall where I can have that little tripod get checked in. So I go there. The lady at the check-in desk tells me that she cannot check the tripod in, because the flight has already been closed and I'm too late. I tell her I'm from a connecting flight, and that I can't help that. She says she knows. So I ask her what my other options are. She tells me I could try to get on the next flight to Brussels, which is at 20:00 (rather than the 16:20 flight that I was booked on). I consider that an acceptable compromise, so ask her how to get that done, and she refers me to the ticket sales desk, some 10 meters to my left. Unfortunately, the 20:00 flight turns out to be completely booked, and there are no other flights on the same day. The best alternative he can give me is a flight to Amsterdam at 19:20, which would cost 'a little more'. I ask him 'how much more'. He starts looking up things in his computer, and after a few minutes tells me that he cannot refund my ticket for the 16:20 flight and that a flight to amsterdam would cost me 429. At this point I get somewhat angry, and tell him that this is not acceptable. He refers me to the customer service desk right across. The lady at the customer service desk is friendly, but firm: she cannot help me. Either I leave the tripod behind, or I pay whatever the ticket service asks me—she takes my word that it is 429, because she cannot check it, and cannot change it, either. I get more agitated, and ask to see the supervisor. She points me to a lady a little across who can 'call' the supervisor. At this point it is 15:55 As I reach that lady, apparently someone else had just asked for the same thing, and starts arguing with a guy in black uniform, who appears to be the supervisor, about the fact that he's got more than 30kg of luggage which he isn't allowed to take along with him. Dude, not a single airline will let you do that, no matter how much you argue. I let them argue for about 10 minutes, all the while nervously looking at my watch, but then I interrupt with 'please, I don't have much time left before my flight leaves'. That gets his attention. In our two-minute conversation, the guy in black backs up everything the other three people have told me, adding that '429 is reasonable; your luggage is already on the flight, and it will cost Iberia a lot of money to delay the flight and get it off'. I curse. He also seems surprised at the notion that I'm not allowed to take a tripod in my carry-on luggage. Hmm. Figuring I have nothing to loose, I curse one more time, and make a run for it. When I reach the security checkpoint, I ask to be allowed to skip the queue in front of it, on the grounds that I have only 15 minutes left before my flight leaves. Mind you, this is not the same security checkpoint as the one where I was rejected about an hour earlier; this is the security checkpoint between the departures hall and the secure area, whereas the other security checkpoint was one between two different terminals. I am granted my request, and quickly throw everything in the X-ray boxes. They need to go through my bag again. I help them, showing where everything conspicuous is, and opening my flute box so they know what's in there. He looks at everything, and signals that I can move on. Hang on, I think. This can't be right. 'Okay,' I ask? 'Yes,' he says. I run for the nearest info desk, show them my boarding pass, and tell them 'please, my flight leaves in 10 minutes, can you tell me where to go?' They are quick and efficient. Gate J40. I make a run for it again, and just make it in time. Of course, I could forget about my plans of finding a power outlet, plugging in my cell phone and calling my parents, but that's not a big deal—there's a train station at Brussels Airport. So what have we learned? Security on airports is random, arbitrary, non-standardized, and utterly stupid. Items that pose no danger at all—plastic bottles containing more than 100ml of water, small quantities of yoghurt that are over 100grammes but not over 100ml, tripods—are not allowed, while on the other hand I've occasionally gotten through security carrying knives and tools that are clearly forbidden by the rules. We've also learned that the rules very much depend on the particular person checking you. If you're refused for an item that is not a knife or a gun or something else similarly problematic, it may help to just find a different security checkpoint and try again. It's not as if they make any sense, anyway.

I'm developing a tool that will take a loooong time to complete, so I'd like to know what's the progress of its execution. Basically I'd like something to tell me "Processed: x of y" where x is the current position and y is the total to process.

On Python Cookbook there are three articles about it, but none of them satisfied me.

The I remembered that there's a program, written in Python, that does what I need: dput! So I wrote a simple script (if you want to look at it, use wget, not open with browser) to mimic what dput does.

At the end, I finally find a really nice module: progressbar. I'm preparing a package for Debian for it, since it contains many nice kind of progress bars. Anyhow, I needed to create a patch to add "Processed: 5 of 47"-kinda output, but it's really useful even as it is, even for you.

Steve's docs on keys on a USB stick needed a few tweaks for my own systems - first thing I noticed was that I had /dev/loop0 instead of /dev/loop/0. As my configuration needs to use an SDcard instead of a USB stick or pen, I've changed the references from 'pen' to 'card'.
SDcards on my systems come out on /dev/mmcblk0 with partitions having a 'p[0-9]' suffix (p1, p2 etc.) so the DEVBASE setting needed a tweak. (This line needs to be modified for each user case anyway because the size of the partition will vary so the '62390272' value needs to be modified.)
DEVBASE= fdisk -l /dev/mmcblk0 2>/dev/null grep ^Disk.*62390272\ bytes sed 's?^.*/d?/d?g;s?:.*$??g'
A few other notes for those using Sid instead of Etch:

1. kernel module needed: loop-aes-modules-2.6.26-1 - getting the modules to load requires a reboot.


2. loop-aes-utils the actual package that provides the diverted mount binaries, instead of just "loop-aes"

Barring a few typos I made in transcribing, the rest of the documentation works fine.

Alternative keys
I have two keys, one is intended to remain on the laptop and has comments to that effect (0xA897FD02). The other key, (0x28BCB3E3), is my Debian upload key so needs to be protected a little more than the ordinary laptop key (which I'll use when signing email on the laptop).
'debsign' does not have a way to specify a different --homedir or secret keyring or any other option that would allow me to have an alternative gpg configuration (maybe it should?). So I've copied debsign into my home scripts directory to create a slightly modified version and then I followed Steve's advice for ~/.gnupg/ to create a duplicate directory with the same permissions and a much reduced public keyring file (just exported my own keys). The secring.gpg file for this alternative homedir is the on the SD card with the autofs symlink (so the only file on the encrypted SDCard is the secret keyring).
Then it is a simple case of manually calling debsign (e.g. after using debuild -uc -us):
~/scripts/debsign -k0x28bcb3e3 /path/to/.changes

+CARD=" --no-default-keyring --homedir=/home/neil/.gcard "

...

-                   $signcommand --local-user "$2" --clearsign \

+                   $signcommand $CARD --local-user "$2" --clearsign \

...

-                   $signcommand --local-user "$2" --clearsign \

+                   $signcommand $CARD --local-user "$2" --clearsign \

...

-       $signcommand -u "$2" +clearsig=on -fast  "$1" > "$ASCII_SIGNED_FILE"

+       $signcommand $CARD -u "$2" +clearsig=on -fast  "$1" > "$ASCII_SIGNED_FILE"

When you have a server on the Internet, you get lots of "brute force" attacks on the SSH daemon, trying plausible logins with a variety of passwords. Even with good passwords, these attacks might eventually succeed (and they're annoying even when they don't), so you want to thwart them. One way is to use fail2ban, a script that monitors the failed connections, and sets up firewall rules (for instance) blocking further connections from the attacking IP addresses. It's good, but it fills your logs with messages about IPs getting banned and unbanned after a while. And you're still at risk that the multiple connections crash the SSH daemon, or trigger a bug in it, or whatever. A second layer of protection can be to block all SSH connection attempts except when they come from known IP addresses, but that doesn't work when you're away from home, and you're locked out. Been there, done that. So, some wise people have devised a trick called "port-knocking". It's similar to only opening the door to people who use a special knock (think "That's all, folks"): the firewall stays closed, but it opens a tiny targeted hole to some IP addresses for a limited length of time, based on a secret handshake. The window for attack is therefore very small, and the SSH daemon stays idle most of the time. And you can still log on your hosted server when you're attending conferences. There are a variety of implementations for this concept. Some could be web-based (you need to submit the right password to a web page), some could use other services or a dedicated daemon. But when I started investigating port-knocking, I wanted something simple, preferably with no dependencies on a daemon that would need to be exposed to the net and potentially crash. I found an article on the Debian Administration website, but I wasn't entirely satisfied with it. The principles appealed to me (netfilter-only, secret handshake in the form of opening connections to secret ports), though, so I evolved it into my own implementation, which I proudly present to you today. The goals of this implementation were:

• flexibility;
• robustness;
• resistance to replay attacks;
• no dependency on user-land daemons;
• simple, easily auditable code;
• must not be a pain in the neck to use.

The bulk of the work therefore stays in the kernel's netfilter (that's for robustness and no user-land dependency), but the control interface is integrated with the usual firewalling script. Resistance to replay attacks is achieved by choosing hard-to-predict ports. So if someone snoops the wireless while I'm at a conference and catches my secret handshake, it'll only be valid for a short period of time, hopefully short enough to prevent dictionary attacks. The handshake is therefore calculated as a function of the current date and time, with an added secret seed. The following shell function calculates 5 port numbers within a given range (requires dc to be installed, for big-integer arithmetic):

calc_knock_ports ()  
    secret=$1
    bottomport=$2
    topport=$3
    nbports=$(( $topport - $bottomport + 1 ))
    hash=$(TZ=UTC date +%Y-%m-%d-%H-$secret   md5sum   awk ' print $1 '   tr a-z A-Z)
    num=$(echo 16i $hash f   dc)
    pk_port1=$(echo $num $nbports 0 ^ / $nbports % $bottomport + f   dc)
    pk_port2=$(echo $num $nbports 1 ^ / $nbports % $bottomport + f   dc)
    pk_port3=$(echo $num $nbports 2 ^ / $nbports % $bottomport + f   dc)
    pk_port4=$(echo $num $nbports 3 ^ / $nbports % $bottomport + f   dc)
    pk_port5=$(echo $num $nbports 4 ^ / $nbports % $bottomport + f   dc)
 

Okay. So this function calculates ports, now what? Now we're going to define a few chains by which netfilter will store states of IP addresses as they progress through the handshake:

setup_portknocking_tables ()  
    iptables -N portknock_into_phase1
    iptables -A portknock_into_phase1 -m recent --name PK_PHASE1 --set
    # iptables -A portknock_into_phase1 -j LOG --log-level notice --log-prefix "INTO PK_PHASE1: "
    iptables -N portknock_into_phase2
    iptables -A portknock_into_phase2 -m recent --name PK_PHASE1 --remove
    iptables -A portknock_into_phase2 -m recent --name PK_PHASE2 --set
    # iptables -A portknock_into_phase2 -j LOG --log-level notice --log-prefix "INTO PK_PHASE2: "
    iptables -N portknock_into_phase3
    iptables -A portknock_into_phase3 -m recent --name PK_PHASE2 --remove
    iptables -A portknock_into_phase3 -m recent --name PK_PHASE3 --set
    # iptables -A portknock_into_phase3 -j LOG --log-level notice --log-prefix "INTO PK_PHASE3: "                     
    iptables -N portknock_into_phase4
    iptables -A portknock_into_phase4 -m recent --name PK_PHASE3 --remove
    iptables -A portknock_into_phase4 -m recent --name PK_PHASE4 --set
    # iptables -A portknock_into_phase4 -j LOG --log-level notice --log-prefix "INTO PK_PHASE4: "
    iptables -N portknock_into_phase5
    iptables -A portknock_into_phase5 -m recent --name PK_PHASE4 --remove
    iptables -A portknock_into_phase5 -m recent --name PK_PHASE5 --set
    iptables -A portknock_into_phase5 -m recent --name PK_ESTABLISHED --set
    # iptables -A portknock_into_phase5 -j LOG --log-level notice --log-prefix "INTO PK_PHASE5: "
    iptables -N portknock_accept
    iptables -A portknock_accept -m limit -j LOG --log-level notice --log-prefix "ACCEPTED AFTER PORTKNOCKING: "
    # iptables -A portknock_accept -m recent --name PK_PHASE5 --remove
    iptables -A portknock_accept -j ACCEPT
    iptables -N portknocking
 

These chains use the recent module, which seems to be commonly available in standard kernels. You'll notice how, as one packet goes through these rules, its originating IP address moves from one set of "recent" addresses to the next. But no logic exists yet to make the packet actually go through these rules, so here comes the glue:

refresh_portknocking ()  
    calc_knock_ports f00b4r 10000 10999
    iptables -F portknocking
    iptables -A portknocking -p tcp --dport $pk_port1 -m state --state NEW                                                 -j portknock_into_phase1
    iptables -A portknocking -p tcp --dport $pk_port2 -m state --state NEW -m recent --rcheck --name PK_PHASE1 --seconds 5 -j portknock_into_phase2
    iptables -A portknocking -p tcp --dport $pk_port3 -m state --state NEW -m recent --rcheck --name PK_PHASE2 --seconds 5 -j portknock_into_phase3
    iptables -A portknocking -p tcp --dport $pk_port4 -m state --state NEW -m recent --rcheck --name PK_PHASE3 --seconds 5 -j portknock_into_phase4
    iptables -A portknocking -p tcp --dport $pk_port5 -m state --state NEW -m recent --rcheck --name PK_PHASE4 --seconds 5 -j portknock_into_phase5
    # echo clear > /proc/net/ipt_recent/PK_DONE
    echo clear > /proc/net/ipt_recent/PK_PHASE1
    echo clear > /proc/net/ipt_recent/PK_PHASE2
    echo clear > /proc/net/ipt_recent/PK_PHASE3
    echo clear > /proc/net/ipt_recent/PK_PHASE4
    echo clear > /proc/net/ipt_recent/PK_PHASE5
 

Right. This function adds rules to the portknocking chain. A packet injected into this ruleset will, depending on its destination port and whether its source IP address has already been seen, end up in one of the PK_PHASE* sets. All we have to do now is therefore to send some packets to this portknocking chain, and use the port-knocking sets to decide whether to accept incoming connections or not:

iptables -A INPUT -j portknocking
iptables -A INPUT -m recent --rcheck --seconds 5 --name PK_PHASE5 -m state --state NEW -p tcp --dport ssh -j portknock_accept

This example only mentions accepting incoming SSH connections, but it's in no way a limitation: a server of mine uses similar rules to DNAT certain ports to internal IP addresses. And there we have it for the server part: incoming SSH connections are usually ignored (well, handled by the rest of the firewall script, but let's assume that it drops these packets by default), but if one IP address knows the appropriate ports and sends a connection attempt to them in order, then it'll be able to open SSH connections for a little while after that. Of course, it's going to be boring if one has to send these packets by hand, but it can be easily automated by a script. Here's a ~/bin/portknock.sh I have:

#! /bin/sh
host=$1
port=$2
calc_knock_ports ()  
[...]
 
calc_knock_ports f00b4r 10000 10999
for i in $pk_port1 $pk_port2 $pk_port3 $pk_port4 $pk_port5 ; do
    nc -w 1 $host $i < /dev/null > /dev/null 2>&1
done
nc $host $port

It's designed to be called with two parameters, a host and a port, and it needs netcat in addition to dc. Why the last line, I hear you cry? Because then I can just add the following lines to my ~/.ssh/config:

Host blahblah
  IdentityFile foobar
  ProxyCommand /home/roland/bin/portknock.sh %h %p

...and SSH will automagically tunnel its network socket through the script, which will in turn happily tunnel that through netcat after completing the secret handshake. And when I type ssh myserver on my laptop, interesting stuff happens behind the scenes, and a special, just-for-me hole is opened in the server firewall, just for the few seconds I need to establish the SSH session (packets belonging to established TCP sockets are allowed by the firewall's connection tracking). Note: This article is deliberately short on details and ready-to-run scripts. Firstly because firewall scripts vary wildly so any script would have to be adapted anyway, but mostly because security is best handled with one's brain switched on. Fiddling with a firewall can easily open gaping holes or lock everyone out. So please make sure you understand what goes on before blindly pasting stuff into your own setup. Some of the lines that are commented out may also be of interest, and were left as an exercise for the reader. Other lines were not included, and are also left as a rather important exercise to the reader; note in particular how the netfilter rules as currently established do not mitigate the replay attacks...

I haven't posted for a long while about ctioga, but that does not mean that development has stopped. It goes on, and features get added basically every single time I need them. Recent additions make it possible to tune a little more the look of the legends and add pure text lines in them. I've however realized that now, what ctioga is missing the most is real flexibility over the coordinates. For now, all coordinate positions are either relative to the frame (such as for --inset) or to the graph (for --draw). What I really wish is a way to mix all kind of coordinates: real size (1cm), frame position and graph position. That will require quite a fair deal of work, of course, but that shouldn't be too difficult... I'll go on posting about it here.

A large part of the disagreement about the way to manage the policy seems to be based on who will be the primary “owner” of the policy on the machine. This isn’t a problem that only applies to SE Linux, the same issue applies for various types of configuration files and scripts throughout the process of distribution development. Having a range of modules which can be considered configuration data that come from a single source seems to make SE Linux policy unique among other packages. The reasons for packaging all Apache modules in the main package seem a lot clearer. One idea that keeps cropping up is that as the policy is modular it should be included in daemon packages and the person maintaining the distribution package of the policy should maintain it. The reason for this request seems to usually be based on the idea that the person who packages a daemon for a distribution knows more about how it works than anyone else, I believe that this is false in most cases. When I started working on SE Linux I had a reasonable amount of experience in maintaining Debian packages of daemons and server processes, but I had to learn a lot about how things REALLY work to be able to write good policy. Also if we were to have policy modules included in the daemon packages, then those packages would need to be updated whenever there were serious changes to the SE Linux policy. For example Debian/Unstable flip-flopped on MCS support recently, changing the policy packages to re-enable MCS was enough pain, getting 50 daemon packages updated would have been unreasonably painful. Then of course there is the case where two daemons need to communicate, if the interface which is provided with one policy module has to be updated before another module can be updated and they are in separate packages then synchronised updates to two separate packages might be required for a single change to the upstream policy. I believe that the idea of having policy modules owned by the maintainers of the various daemon packages is not viable. I also believe that most people who package daemons would violently oppose the idea of having to package SE Linux policy if they realised what would be required of them. Caleb Case seems to believe that ownership of policy can either be based on the distribution developer or the local sys-admin with apparently little middle-ground [1]. In the section titled “The Evils of Single Policy Packages” he suggests that if an application is upgraded for a security fix, and that upgrade requires a new policy, then it requires a new policy for the entire system if all the policy is in the same package. However the way things currently work is that upgrading a Debian SE Linux policy package does not install any of the new modules. They are stored under /usr/share/selinux/default but the active modules are under /etc/selinux/default/modules/active. An example of just such an upgrade is the Debian Security Advisory DSA-1617-1 for the SE Linux policy for Etch to address the recent BIND issue [2]. In summary the new version of BIND didn’t work well with the SE Linux policy, so an update was released to fix it. When the updated SE Linux policy package is installed it will upgrade the bind.pp module if the previous version of the package was known to have the version of bind.pp that didn’t allow named to bind() to most UDP ports - the other policy modules are not touched. I think that this is great evidence to show that the way things currently work in Debian work well. For the hypothetical case where a user had made local modifications to the bind.pp policy module, they could simply put the policy package on hold - I think it’s safe to assume that anyone who cares about security will read the changelogs for all updates to released versions of Debian, so they would realise the need to do this. Part of Caleb’s argument rests on the supposed need for end users to modify policy packages (IE to build their own packages from modified source). I run many SE Linux machines, and since the release of the “modular” policy (which first appeared in Fedora Core 5, Debian/Etch, and Red Hat Enterprise Linux 5) I have never needed to make such a modification. I modify policy regularly for the benefit of Debian users and have a number of test machines to try it out. But for the machines where I am a sysadmin I just create a local module that permits the access that is needed. The only reason why someone would need to modify an existing module is to remove privileges or to change automatic domain transition rules. Changing automatic domain transitions is a serious change to the policy which is not something that a typical user would want to do - if they were to do such things then they would probably grab the policy source and rebuild all the policy packages. Removing privileges is not something that a typical sysadmin desires, the reference policy is reasonably strict and users generally don’t look for ways to tighten up the policy. In almost all cases it seems best to consider that the policy modules which are shipped by the distribution are owned by the distribution not the sysadmin. The sysadmin will decide which policy modules to load, what roles and levels to assign to users with the semanage tool, and what local additions to add to the policy. For the CentOS systems I run I use the Red Hat policy, I don’t believe that there is a benefit for me to change the policy that Red Hat ships, and I think that for people who have less knowledge about SE Linux policy than me there are more reasons not to change such policy and less reasons to do so. Finally Caleb provides a suggestion for managing policy modules by having sym-links to the modules that you desire. Of course there is nothing preventing the existence of a postfix.pp file on the system provided by a package while there is a local postfix.pp file which is the target of the sym-link (so the sym-link idea does not support the idea of having multiple policy packages). With the way that policy modules can be loaded from any location, the only need for sym-links is if you want to have an automatic upgrade script that can be overridden for some modules. I have no objection to adding such a feature to the Debian policy packages if someone sends me a patch. Caleb also failed to discuss how policy would be initially loaded if packaged on a per-module basis. If for example I had a package selinux-policy-default-postfix which contains the file postfix.pp, how would this package get installed? I am not aware of the Debian package dependencies (or those of any other distribution) being about to represent that the postfix package depends on selinux-policy-default-postfix if and only if the selinux-policy-default package is installed. Please note that I am not suggesting that we add support for such things, a package management system that can solve Sudoku based on package dependency rules is not something that I think would be useful or worth having. As I noted in my previous post about how to package SE Linux policy for distributions [3] the current Debian policy packages have code in the postinst (which I believe originated with Erich Schubert) to load policy modules that match the Debian packages on the system. This means that initially setting up the policy merely requires installing the selinux-policy-default package and rebooting. I am inclined to reject any proposed change which makes the initial install of of the policy more difficult than this. After Debian/Lenny is released I plan to make some changes to the policy. One thing that I want to do is to have a Debconf option to allow users to choose to automatically upgrade their running policy whenever they upgrade the Debian policy package, this would probably only apply to changes within one release (IE it wouldn’t cause an automatic upgrade from Lenny+1 policy to Lenny+2). Another thing I would like to do is to have the policy modules which are currently copied to /etc/selinux/default/modules/active instead be hard linked when the source is a system directory. That would save about 12M of disk space on some of my systems. I’ve taken the unusual step of writing two blog posts in response to Caleb’s post not because I want to criticise him (he has done a lot of good work), but because he is important in the SE Linux community and his post deserves the two hours I have spent writing responses to it. While writing these posts I have noticed a number of issues that can be improved, I invite suggestions from Caleb and others on how to make such improvements.

• [1] http://www.calebcase.com/node/2


• [2] http://lists.debian.org/debian-security-announce/2008/msg00201.html


• [3] http://etbe.coker.com.au/2008/08/18/se-linux-policy-packaging-distribution/

Share This

What I liked

• Excellent organization, orga team fully deserved all the praise they got.
• Meeting people - it was the first DebConf I attended in person, so it was a pleasure to meet lots of folks whom I’ve only known online before.
• Quite a few interesting talks (videos are available, or will be available shortly, in DebConf8 video archive). Particularly recommended: quality assurance talks/BOFs by Lucas Nussbaum, BTS development progress by Don Armstrong, method diffusion in large volunteer projects by Martin Krafft, talk and debate on OpenSSL security issue by Luciano Bello, various lightning talks.
• Deserving a special mention, a talk by Mark Shuttleworth on Debian and Ubuntu relationship. I thought that he said all the right words and, judging by reactions, attitude towards Ubuntu has changed significantly during the last year, turning from hostility to peaceful cooperation.
• Argentinian food: steaks, empanadas, alfajores.

What I did not like

• People pay much less attention to fixing RC bugs than they used to, the two BSPs organized during DebConf8 attracted only 3-5 people.
• Some nice ideas being thrown around without anyone volunteering to work on them. Not really surprising, given that most of the core teams are short on manpower.
• Almost complete lack of events targeted at engaging users and beginner contributors.
• Quite a few specialized talks, interesting (IMHO) to only a minor percentage of the attendees.
• Lots of discussions with few or no specific decisions or action items.
• Pretty long trips to and from Mar del Plata (about 30 hours door-to-door travel time for me), luggage was lost for 2 days on the way there.

What I did

• Attended about 75% of talks, tried to contribute to discussion in constructive, non-flamy ways .
• Participated in BSPs, contributed debugging information, patches or fixes for about a dozen of release critical bugs, started triaging sparc-specific bugs.
• Started working on updating the Debian kernel handbook for Lenny, about 1/3 done.
• Gave a “Google and Open Source” lightning talk.

I will probably never find time to blog about Debconf 8 and write down the pleasure I had to attend it, so let's use jetlag as an excuse for the short length of this blog entry and just mention that Debconf 8 rocked. The DC8 organizers rocked. The hotel rocked. The network team rocked. The DC8 attendees rocked. The speakers rocked. And jetlag sucks. Soon to come: my online gallery...

Caleb Case (Ubuntu contributer and Tresys employee) has written about the benefits of using separate packages for SE Linux policy modules [1]. Firstly I think it’s useful to consider some other large packages that could be split into multiple packages. The first example that springs to mind is coreutils which used to be textutils, shellutils, and fileutils. Each of those packages contained many programs and could conceivably have been split. Some of the utilities in that package are replaced for most use, for example no-one uses the cksum utility, generally md5sum and sha1sum (which are in the same package) are used instead. Also the pinky command probably isn’t even known by most users who use finger instead (apart from newer Unix users who don’t even know what finger is). So in spite of the potential benefit of splitting the package (or maintaining the previous split) it was decided that it would be easier for everyone to have a single package. The merge of the three packages was performed upstream, but there was nothing preventing the Debian package maintainer from splitting the package - apart from the inconvenience to everyone. The coreutils package in Etch takes 10M of disk space when installed, as it’s almost impossible to buy a new hard drive smaller than 80G that doesn’t seem to be a problem for most users. The second example is the X server which has separate packages for each video card. One thing to keep in mind about the X server is that the video drivers don’t change often. While it is quite possible to remove a hard drive from one machine and install it in another, or duplicate a hard drive to save the effort of a re-install (I have done both many times) they are not common operations in the life of a system. Of course when you do require such an update you need to first install the correct package (out of about 60 choices), which can be a challenge. I suspect that most Debian systems have all the video driver packages installed (along with drivers for wacom tablets and other hardware devices that might be used) as that appears to be the default. So it seems likely that a significant portion of the users have all the packages installed and therefore get no benefit from the split package. Now let’s consider the disk space use of the selinux-policy-default package - it’s 24M when installed. Of that 4.9M is in the base.pp file (the core part of the policy which is required), then there’s 848K for the X server (which is going to be loaded on all Debian systems that have X clients installed - due to an issue with /tmp/.ICE-unix labelling [2]). Then there’s 784K for the Postfix policy (which is larger than it needs to be - I’ve been planning to fix this for the past four years or so) and 696K for the SSH policy (used by almost everyone). The next largest is 592K for the Unconfined policy, the number of people who choose not to use this will be small, and as it’s enabled by default it seems impractical to provide a way of removing it. One possibility for splitting the policy is to create a separate package of modules used for the less common daemons and services, if modules for INN, Cyrus, distcc, ipsec, kerberos, ktalk, nis, PCMCIA, pcscd, RADIUS, rshd, SASL, and UUCP were in a separate package then that would reduce the installed size of the main package by 1.9M while providing no change in functionality to the majority of users. One thing to keep in mind is that each package at a minimum will have a changelog and a copyright file (residing in a separate directory under /usr/share/doc) and three files as part of the dpkg data store, each of which takes up at least one allocation unit on disk (usually 4K). So adding one extra package will add at least 24K of disk space to every system that installs it (or 32K if the package has postinst and postrm scripts). This is actually a highly optimal case, the current policy packages (selinux-policy-default and selinux-policy-mls) each take 72K of disk space for their doc directory. One of my SE Linux server sytems (randomly selected) has 23 policy modules installed, if they were in separate packages there would be a minimum of 552K of disk space used by packaging, 736K if there were postinst and postrm scripts, and as much as 2M if the doc directory for each package was similar to the current doc directories). As the system in question needs 5796K of policy modules, the 2M of overhead would make it approach 8M of disk space. So it would only be a saving of 16M over the current situation. While saving that amount of disk space is a good thing, I think that when balanced against the usability issues it’s not worth-while. Currently the SE Linux policy packages will determine what applications are installed and automatically load policy packages to match. I don’t believe that it’s possible to have a package post-inst script install other packages (and if it is possible I don’t think it’s desirable). Therefore to have separate packages would make a significant difference to the ease of use, it seems that the best way to manage it would be to have the core policy package include a script to install the other packages. Finally there’s the issue of when you recognise the need for a policy module. It’s not uncommon for me to do some work for a client while on a train, bus, or plane journey. I will grab packages needed to simulate a configuration that the client desires and then work out how to get it going correctly while on the journey. While it would not be a problem for me (I always have the SE Linux policy source and all packages on hand) I expect that many people who have similar needs might find themself a long way from net access without the policy package that they need to do their work. Sure such people could do their work in permissive mode, but that would encourage them to deploy in permissive mode too and thus defeat the goals of the SE Linux project (in terms of having wide-spread adoption). My next post on this topic will cover the issue of custom policy. Updated to note that Caleb is a contributor to Ubuntu not a developer.

• [1] http://www.calebcase.com/node/2


• [2] http://marc.info/?l=selinux&m=121789588709977&w=2

Share This

I decided to expand the scope of my blog to non-Debian subjects and non-English posts, so from now on Debian-related posts will appear in the /debian/ category (which is the only one syndicated to Planet Debian). Interested readers may subscribe to my complete feed (whose URL has not changed) or read and comment the blog in the it.fan.marco-ditri newsgroup.

I’ll be in Great Britain next week. I’ll come to the Debian Party 2008 in Cambridge on Saturday 23rd August and will be back to London to visit ’till Wednesday 27th August. I’m staying by myself in a nice little hotel in Bloomsbury. It will be very interesting to meet Debian people and probably do some pitching for my Aptitude GTK project which is making nice progress. I’ve never been to London before. In fact, I’ve been once when very young and all that I can remember is that the hot dogs seemed nice :). I’m currently reading the Lonely Planet book about London and filling my program. Hi, London!

Someone asked me the other day what it would take to add Debian support for the D-Link DNS-323. Since we support a number of Orion based devices in debian-installer already, adding support for another device is typically fairly easy. I don't have a D-Link DNS-323 myself, but I looked around the useful DNS-323 wiki and this is what I came up. I'm sharing this list in the hope that other people are interested in working on DNS-323 support.

• The Orion kernel in Debian has support for the D-Link DNS-323 but it needs testing. Also, some patches, such as support for the fan speed chip, are not included in the mainline kernel yet.
• flash-kernel (which writes the kernel and ramdisk to flash) needs support for the DNS-323. I actually implemented this in the meantime but it's completely untested.
• oldsys-preseed reads the network configuration from your existing firmware and uses it to configure the network for debian-installer. This also needs support for the DNS-323.
• Some tools to generate proper firmware images need to be packaged for Debian.
• The debian-installer needs to generate a boot image for the DNS-323 (easy once the tools are packaged).
• Apparently the MAC address is not set automatically in u-boot and you have to run a tool called mac_read to set it. This is problematic because at the moment there's no code to set the MAC address in d-i and to make sure the newly installed system will automatically set the MAC address. This needs some work.

So something Bdale came up with this morning at breakfast (and during his talk too, apparently, which I didn't attend) was this idea of a "hardware compatibility test"): a bootable image that hardware vendors could run to see whether their hardware would run Debian. Apparently all the other vendors have it, too, and the lack of it may be one of the main reasons why Debian isn't currently supported by a whole lot of hardware vendors yet. Such a test wouldn't have to do all that much; just boot the machine (if it can) with the kernel that would be used for the installer and the system that is eventually installed; then run through a check of the available hardware, and finally come up with some kind of score that tells the vendor whether their hardware is supported at all, or if not, what they could do to improve the score. It seemed to me (and to Mark Hymers, who was seated to my left) that this is something that could be done fairly easily with a slightly modified version of debian-installer. It would be okay if there was a different version for every Debian release we do; and I tend to think it's not even going to be a problem if the first time we don't make the release, but release such a test slighty after the release of Debian. Having such a test would certainly give hardware vendors an incentive to improve their Debian support, especially if it's a simple thing that they can have some summer student do over all their hardware who'd then store the results in a database of sorts. Or so. Additionally, if we do this right, we could diversify between 'a wireless driver that will probably work if you load ndiswrapper or something similar' (which would get a score that tells them 'yes, it will run Debian', but no perfect score) and 'a wireless driver that works with free drivers and no additional firmware required' (which would get a perfect score if there's nothing more). By doing this, we would put Debian's collective driving force behind a move to better and more Linux-friendly hardware, which can only be a good thing. Bdale seems to thing this could be an industry-changing thing, and I can't think of a reason why it wouldn't work. Except for one: I'm not sure I'll have the time to work on this myself; and even if I would be sure of that, it's not going to be something that I can do all by myself -- other people would have to run the test on their hardware and communicate the results. So here's a request: is anyone else interested in this kind of thing? It doesn't sound like something too complicated; and given my business, it surely is something I have a personal interest in, so I will try to make time for it. But I can't do it alone...

After a long night of debugging and while preparing, reviewing and pushing final patches before a SymPy release, I got this:

$ hg qpush 
applying 1645.diff
Unable to read 1645.diff
** unknown exception encountered, details follow
** report bug details to http://www.selenic.com/mercurial/bts
** or mercurial@selenic.com
** Mercurial Distributed SCM (version 1.0.1)
Traceback (most recent call last):
  File "/usr/bin/hg", line 20, in 
    mercurial.dispatch.run()
  File "/var/lib/python-support/python2.5/mercurial/dispatch.py", line 20, in run
    sys.exit(dispatch(sys.argv[1:]))
  File "/var/lib/python-support/python2.5/mercurial/dispatch.py", line 29, in dispatch
    return _runcatch(u, args)
  File "/var/lib/python-support/python2.5/mercurial/dispatch.py", line 45, in _runcatch
    return _dispatch(ui, args)
  File "/var/lib/python-support/python2.5/mercurial/dispatch.py", line 364, in _dispatch
    ret = _runcommand(ui, options, cmd, d)
  File "/var/lib/python-support/python2.5/mercurial/dispatch.py", line 417, in _runcommand
    return checkargs()
  File "/var/lib/python-support/python2.5/mercurial/dispatch.py", line 373, in checkargs
    return cmdfunc()
  File "/var/lib/python-support/python2.5/mercurial/dispatch.py", line 356, in 
    d = lambda: func(ui, repo, *args, **cmdoptions)
  File "/var/lib/python-support/python2.5/hgext/mq.py", line 1942, in push
    mergeq=mergeq)
  File "/var/lib/python-support/python2.5/hgext/mq.py", line 833, in push
    top = self.applied[-1].name
IndexError: list index out of range

I am using:

$ hg --version 
Mercurial Distributed SCM (version 1.0.1)

Copyright (C) 2005-2008 Matt Mackall  and others
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

And I just got pissed off and I am switching to git for good. I've been using mercurial every day since about November 2007, and I think I am quite proficient in it. But we were missing some features, like recording a patch by hunks (git add -p), so Kirill Smelkov (another SymPy developer) just implemented this to mercurial queues, it is now included in hg 1.0. But I am constantly beaten by stupid bugs, mercurial broke into pieces several times already (last time with hg bisect, before it with mercurial queues and file renaming), that just should not happen in a production version. Another problem is that hgweb is constantly using 100% of CPU on my server whenever someone clicks to see the contents of the README file. Again I can spend time debugging it, but I will just use git and if it works, I'll stay with it.

I started learning git recently and it is just better in every aspect. Bigger community, it has all the features that I was always missing in mercurial (like rebase, hunks recording, diff diff --color-words, better gitk). And it's superfast.

Unless I find some showstopper bugs in git too, I don't think I am coming back.

We'll have to create a live mercurial mirror of SymPy, so that people can continue using mercurial, but I'll myself just be using git. We wrote a simple translation table for anyone using mercurial to get started:

http://wiki.sympy.org/wiki/Git_hg_rosetta_stone

I love Python and I like that Mercurial is just a small C core and the rest is in Python, but for some reason, it still has the baby bugs, that should have been fixed years ago and why should we spend our time fixing and improving Mercurial, if other people have already done the job for us (and usually a better job than I could do) in git?

Another reason for switching is that in Debian, mercurial is not used for packages, while git is used a lot.