I recently wrote about wanting to have a teletype. Well, I have since realized that teletypes weigh hundreds of pounds, draw hundreds of watts, and aren't available on eBay for a reasonable price. Well I knew the hundreds of pounds bit, but still. I pretty well have had to give up on a real teletype.

So, now on to the next best thing: a teletype simulator. Enter the two free dot-matrix printers that found their way to my office earlier this week. One of them even works. I bicycled to the awesome local office supplies store (about 11 miles away) to buy a ribbon for it. This is the place that's been there since the 1890s. They still stock dot matrix ribbons, typewriter ribbons, and even fanfold paper.

On the project. Linux has its heritage in Unix, which was used with these devices. It can be made to work with them even now. But there's a trick: teletypes used a bidirectional serial link. Dot matrix printers have no keyboard. So we have to take input from a different device than we send output to.

A simple trick will do for that:

TERM=escpterm telnet localhost > /dev/lp0

Now, here's the next problem. Dot-matrix printers have a line buffer. They don't start printing the line at all until they see CR or LF. Makes it annoying for interactive use. So I wrote a quick tool to insert into that pipeline. After a certain timeout after the input stops, it will force the printer to flush its buffer. Took a little while to figure out how to do that, too; turns out there's a command ESC J that takes an increment for vertical spacing in 1/216 inch, and accepts 0. So I can send \x1BJ\x00 to flush the buffer. I can run it like this:

TERM=escpterm telnet localhost escpbuf > /dev/lp0

That leaves another problem, though: the printhead is right over the text. (Even though it moves to the right of the printing position, and then moves back left for the next character to print.) I modified the program to roll the paper out a bit, and then reverse feed it to continue printing the line. But that is slow and, I suspect, tough on the stepper motor.

Also, I have crafted a terminfo file for the Epson-compatible dot-matrix printers (which are almost all of them), which can also be found at the above link.

So here's the question, for anyone that has used a real teletype:

Did the printhead obscure the text there too, or could you see the entire current line at all times?

All "low quality" videos (which in fact have better quality than the streams had) from DebConf8 are now available for download at the usual location. Enjoy!

I expect the rest of the high quality ones to appear during the next 48h, after that I'll start with the remaining post processing, which will take a bit longer, probably some weeks, hopefully only a few though :-)

Currently there are 57 events available in low quality (4,9G in size) and 30 in high (14G). The SPI event is the only one I know that is missing, but I still need to check if we have it on tape and if there are more missing.

If you notice any non-obvious problems with these videos, please don't tell me, but do describe the problem in our todo list. Thanks!

To make that a bit more clear: There is only one kind of obvious problem: more than one file per event. It's obvious that these need to be merged. All other issues, like a completly missing event, missing beginning of an event, or silence instead of audio, etc. should be noted in our todo-list. Your help here is very much appreciated.

For some time there have been two mainstream Mandatory Access Control (MAC) [1] systems for Linux. SE Linux [2] and AppArmor [3]. In late 2007 Novell laid off almost all the developers of AppArmor [4] with the aim of having the community do all the coding. Crispin Cowan (the founder and leader of the AppArmor project) was later hired by Microsoft, which probably killed the chances for ongoing community development [5]. Crispin has an MSDN blog, but with only one post so far (describing UAC) [6], hopefully he will start blogging more prolifically in future. Now SUSE is including SE Linux support in OpenSUSE 11.1 [7]. They say that they will not ship policies and SE Linux specific tools such as “checkpolicy”, but instead they will be available from “repositories”. Maybe this is some strange SUSE thing, but for most Linux users when something is in a “repository” then it’s shipped as part of the distribution. The SUSE announcement also included the line “This is particularly important for organizations that have already standardized on SELinux, but could not even test-drive SUSE Linux Enterprise before without major work and changes“. The next step will be to make SE Linux the default and AppArmor the one that exists in a repository, and the step after that will be to remove AppArmor. In a way it’s a pity that AppArmor is going away so quickly. The lack of competition is not good for the market, and homogenity isn’t good for security. But OTOH this means more resources will be available for SE Linux development which will be a good thing.

• [1] http://en.wikipedia.org/wiki/Mandatory_access_control


• [2] http://www.nsa.gov/selinux/


• [3] http://en.wikipedia.org/wiki/Apparmor


• [4] http://news.cnet.com/8301-13580_3-9796140-39.html


• [5] http://blogs.msdn.com/michael_howard/archive/2008/01/17/crispin-cowan-joins-the-windows-security-team.aspx


• [6] http://blogs.msdn.com/crispincowan/default.aspx


• [7] http://news.opensuse.org/2008/08/20/opensuse-to-add-selinux-basic-enablement-in-111/

Share This

I’m sure this was an honest mistake, but it sure was amusing:

This week has been a little hectic, as I’ve been struggling with testing different versions of the GNU/Linux kernel.

Docking the FreeRunner into the laptop Earlier in the day, I wrote:

So yes, my laptop can now be turned into a phone charger with networking, DNS and apt cache services. I shall look into hooking that script into dbus to have it run automatically when the phone is plugged and unplugged.

It's not dbus, it's udev, and I've managed to do it. It's an udev rule:

# cat /etc/udev/rules.d/z60_openmoko_net.rules
ACTION=="add", SUBSYSTEM=="net", ATTRS idVendor =="1457", ATTRS idProduct =="5122", RUN+="/root/bin/share-openmoko"
ACTION=="remove", SUBSYSTEM=="net", INTERFACE="usb0", RUN+="/root/bin/share-openmoko"

And a script:

# cat bin/share-openmoko 
#!/bin/sh
if [ "$ACTION" == "add" ]
then
    ACTION=start
fi
if [ "$ACTION" == "remove" ]
then
    ACTION=stop
fi
INTERFACE=$ INTERFACE:-"usb0" 
ACTION=$ ACTION:-"$1" 
case "$ACTION" in
    start)
        logger -t openmoko "Connected, setting up network"
        iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE -s 192.168.0.200/29
        ifconfig usb0 192.168.0.200 netmask 255.255.255.248
        /etc/init.d/dnsmasq start
        echo 1 > /proc/sys/net/ipv4/ip_forward
        ;;
    stop)
        logger -t openmoko "Disconnected, bringing down network"
        echo 0 > /proc/sys/net/ipv4/ip_forward
        /etc/init.d/dnsmasq stop
        iptables -t nat -F POSTROUTING
        ifconfig usb0 down
        ;;
esac

The script has extra cruft that makes it double as a udev script and as a init.d style script, just because I didn't feel like abandoning the init.d style interface. However, udev handles it perfectly, so there's probably no use at all for the start/stop part. This means that the interface and all supporting services will be brought up and down when the phone is connected/disconnected, and also when the phone is suspended/resumed. Of course, more fancy things can be plugged into the script, like syncing PIM info, file systems, turning on applets in panels, mounting phone file systems and whatnot.

Unpacking the new FreeRunner I got myself a FreeRunner. Here are some notes from the first few days: Available operating systems The FreeRunner comes with the Om 2007.2 distribution: it works for basic phone things, and it has an opkg package manager that you can use to install all sort of extra software. Only issue: the SMS application was rather unstable with my SIM card. I then tried Om 2008.8, which is a major new redesign, partly based on Qtopia. It's definitely more advanced on how it looks, with smooth animations all around, but I did not manage to get the GSM part to work, because I did not manage to get it to ask for a PIN. I then tried plain Qtopia and I got what looks and feels like a properly working mobile phone. The only issue that it has is that I did not manage to make it suspend, so battery life is much shorter than it could be. Apparently, this should improve as kernel 2.6.26 reaches the phone. I kept Qtopia installed in the phone flash as a "stable" phone system. And finally, Debian. Debian is based on the freesmartphone.org software stack, which is an attempt to create a UI agnostic DBUS frontend to the hardware that supports multiple applications running on top of it. It is a young project, but it can already drive a mobile phone in a useful way. It has a demo interface that is basically a showcase of what is implemented in the DBUS frontend, but does most of the basic things you need from a mobile phone, including taking and making phone calls. And then it has Debian behind, all of it. I need to buy a bigger microsd card. Tips and tricks Flashing things

1. apt-get install dfu-util
2. Start the Freerunner bios/bootloader, by holding down Power and then pressing AUX (standard bootloader), or by holding down AUX and then pressing Power (factory, unbrickable, read only fail safe bootloader).
3. dfu-util -l shows you a list of devices it can access. If you see more than once, you need to specify in all dfu-util commands which one you want, using -d USBID. In my case, I have to always use dfu-util -d 0x1d50:0x5119
4. To flash the kernel: dfu-util -a kernel -R -D /path/to/uImage
5. To flash the root file system: dfu-util -a rootfs -R -D rootfs_filename.jffs2
6. To flash the bootloader: dfu-util -a u-boot -R -D uboot_filename.bin
7. To flash the u-boot configuration: dfu-util -a u-boot_env -D env.new

You can also download all of these things from the FreeRunner by using -U instead of -D. Networking via USB All of the distributions I tried, by default configure the USB as a gadget with ethernet over usb. You can form a lan with it using the cdc_ether module, and you will find your phone preconfigured as 192.168.0.202 expecting to find a gateway at 192.168.0.200. I made myself this script to start and stop networking with the phone:

#!/bin/sh
case "$1" in
    start)
        iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE -s 192.168.0.200/29
        ifconfig usb0 192.168.0.200 netmask 255.255.255.248
        /etc/init.d/dnsmasq start
        echo 1 > /proc/sys/net/ipv4/ip_forward
        ;;
    stop)
        echo 0 > /proc/sys/net/ipv4/ip_forward
        /etc/init.d/dnsmasq stop
        iptables -t nat -F POSTROUTING
        ifconfig usb0 down
        ;;
esac

Besides doing masquerading, it also brings up dnsmasq so that the phone can always find a DNS together with the router. Another useful trick is to configure the phone to share the approx cache with the laptop:

deb http://192.168.0.200:9999/debian unstable main
deb http://192.168.0.200:9999/debian experimental main
deb http://pkg-fso.alioth.debian.org/debian unstable main

So yes, my laptop can now be turned into a phone charger with networking, DNS and apt cache services. I shall look into hooking that script into dbus to have it run automatically when the phone is plugged and unplugged. Changing the ringtone in Debian In case you don't like the default ringtone, this is how to change it:

1. vi /usr/share/python-support/fso-frameworkd/framework/subsystems/oeventd/receiver.py
2. look for def _play( self ):
3. change the codec in decoder = gst.element_factory_make, if needed. The list of available plugins is on the gstreamer website.
4. change the path in filesrc.set_property
5. /etc/init.d/fso-frameworkd restart and maybe /etc/init.d/zhone-session restart

Yes, the ringtone is currently hardcoded, but it does say it's a prototype after all. Or, if you prefer, it's fully configurable and the configuration can be found in /usr/share/python-support/fso-frameworkd. Configuring the bootloader You can connect to the u-boot bootloader via a serial terminal on /dev/ttyACM0. Type help and you will find that it can do a lot of things. The OpenMoko wiki has pages on the bootloader itself, its commands and the environment. The environment is the configuration of the bootloader, similar somehow to /boot/grub/menu.lst. Unlike grub, you can edit the environment from within the bootloader and then save it using the saveenv command. What I did:

1. A boot entry for Debian, with the kernel on an ext2 partition instead of fat: setenv menu_3 Boot from microSD (FAT+ext2): setenv bootargs \$ bootargs_base rootfstype=ext2 root=/dev/mmcblk0p2 rootdelay=5 \$ mtdparts ro\; mmcinit\; fatload mmc 1 0x32000000 \$ sd_image_name \; bootm 0x32000000
2. Stay in the bootloader until a choice has been made: setenv bootdelay -1
3. Don't power down the bootloader when idle: setenv boot_menu_timeout 99999
4. Always show the menu at boot: setenv stop_in_menu yes
5. saveenv

So now my phone dual boots, and I can choose if I want a more reliable phone now (Qtopia) or if I want to play with my future phone (Debian). Configuring ssh for two host keys on the same host Minor issue, but annoying: since both QTopia and Debian show up on 192.168.0.202, ssh will complain about changed host keys. Here is how to configure ssh to avoid the problem (in ~/.ssh/config):

Host debian
HostName 192.168.0.202
User root
HostKeyAlias debian
Host qtopia
HostName 192.168.0.202
User root
HostKeyAlias qtopia

I’ve just read an amusing series of blog posts about bad wiring [1]. I’ve seen my share of wiring horror in the past. There are some easy ways of minimising wiring problems which seem to never get implemented. The first thing to do is to have switches near computers. Having 48 port switches in a server room and wires going across the building causes mess and is difficult to manage. A desktop machine doesn’t need a dedicated Gig-E (or even 100baseT) connection to the network backbone. Cheap desktop switches installed on desks allow one cable to go to each group of desks (or two cables if you have separate networks for VOIP and data). If you have a large office area then a fast switch in the corner of the room connecting to desktop switches on the desks is a good way to reduce the cabling requirements. The only potential down-side is that some switches are noisy, the switches with big fans can be easily eliminated by a casual examination, but the ones that make whistling sounds from the PSU need to be tested first. The staff at your local electronics store should be very happy to open one item for inspection and plug it in if you are about to purchase a moderate number (they will usually do so even if you are buying a single item). A common objection to this is the perceived lack of reliability of desktop switches. One mitigating factor is that if a spare switch is available the people who work in the area can replace a broken switch. Another is that my observation is that misconfiguration on big expensive switches causes significantly more down-time than hardware failures on cheap switches ever could. A cheap switch that needs to be power-cycled once a month will cause little interruption to work, while a big expensive switch (which can only be configured by the “network experts” - not regular sysadmins such as me) can easily cause an hour of down-time for most of an office during peak hours. Finally the reliability of the cables themselves is also an issue, having two cables running to the local switch in every office can allow an easy replacement to fix a problem - it can be done without involving the IT department (who just make sure that both cables are connected to the switch in the server room). If there is exactly one cable running to each PC from the server room and one of the cables fails then someone’s PC will be offline for a while. In server rooms the typical size of a rack is 42RU (42 Rack Units). If using 1RU servers that means 42 Ethernet cables. A single switch can handle 48 Ethernet ports in a 1RU mount (for the more dense switches), others have 24 ports or less. So a single rack can handle 41 small servers and a switch with 48 ports (two ports to go to the upstream switch and five spare ports). If using 2RU servers a single rack could handle 20 servers and a 24port switch that has two connections to the upstream switch and two spare ports. Also it’s generally desirable to have at least two Ethernet connections to each server (public addresses and private addresses for connecting to databases and management). For 1RU servers you could have two 48 port switches and 40 servers in a rack. For 2RU servers you could have 20 servers and either two 24port switches or one 48port switch that supports VLANs (I prefer two switches - it’s more difficult to mess things up when there are two switches, if one switch fails you can login via the other switch to probe it, and it’s also cheaper). If the majority of Ethernet cables are terminated in the same rack it’s much harder for things to get messed up. Also it’s very important to leave some spare switch ports available as it’s a common occurrence for people to bring laptops into a server room to diagnose problems and you really don’t want them to unplug server A to diagnose a problem with server B… Switches should go in the middle of the rack. While it may look nicer to have the switch at the top or the bottom, that means that the server which is above or below it will have the cables for all the other switches going past it. Ideally the cables would go in neat cable runs at the side of the rack but in my experience they usually end up just dangling in front. If the patch cables are reasonably short and they only dangle across half the servers things won’t get too ugly (this is harm minimisation in server room design). The low end of network requirements is usually the home office. My approach to network design for my home office is quite different, I have no switches! I bought a bunch of dual-port Ethernet cards and now every machine that I own has at least two Ethernet ports (and some have as many as four). My main router and gateway has four ports which allows connections from all parts of my house. Then every desktop machine has at least two ports so that I can connect a laptop in any part of the house. This avoids the energy use of switches (I previously used a 24 port switch that drew 45W [2]), switches of course also make some noise and are an extra point of failure. While switches are more reliable than PCs, as I have to fix any PC that breaks anyway my overall network reliability is increased by not using switches. For connecting the machines in my home I mostly use bridging (only the Internet gateway acts as a router), I have STP enabled on all machines that have any risk of having their ports cross connected but disable it on some desktop machines with two ports (so that I can plug my EeePC in and quickly start work for small tasks).

• [1] http://www.darkroastedblend.com/2008/08/crazy-wiring-part-5.html


• [2] http://doc.coker.com.au/environment/computer-related-power-use/

Share This

This week has been a little hectic, as I've been struggling with testing different versions of the GNU/Linux kernel. Specifically I've been trying to solve a problem where a Phenom processor, when coupled with 8Gb, would kernel panic under heavy load. After testing various patches, kernel versions, and random things I believe the problem is fixed in the kernel version 2.6.27RC4 - however nothing in the changelog appears relevant, so I guess only time will tell. Now we need to solve the problem of Atom processors panicing when attempting to boot 64-bit kernels. That is still present in the 2.6.27RC4 kernel. (ObRandom: If there are any interested parties I can provide remote serial console access to such a system.) Finally I've also been playing with PAM, the plugabble authentication module. Again specific use-case here. At work we want to allow people to ssh to some systems (to access serial consoles, etc), and we wish their connections to be tested against our internal single-sign-on mechanism. That could have meant a whole new PAM module, which would do XML-RPC-fu. Instead it meant packaging libpam-external - which is a neat PAM module allowing you to specify a shellscript to validate users & passwords. (libpam-external is very similar to mod_authnz_external which is a similar pluggable Apache2 module) So, this week "kernel hacking", & "pam hacking". Does that make me a real developer now? ObQuote: Time Bandits

Debian Integrated Solutions. (DIS, pronounced "dish".)

Or at least, that's what seems to become the result of this thread on thedebian-custom mailinglist (and some RL discussions at DebConf8).

I've not yet participated in the renaming discussions, at the moment I can only say what others already said: the terms "custom" and "distribution" both have several problems and renaming won't be easy. Now that I've said that, it occurs to me that I do have something to add: renaming established names is never easy, but can very well be worth the effort, if the established name is problematic / has shortcomings and the new one is way better.

And there I have my doubts. TTBOMK or IME, neither Debian Med nor Debian Edu qualify as "solutions", but are a (great) basis to build solutions...

Hmm, I mostly started to write this blog post, to make more people aware of this discussion and not to add new arguments. This should better be done on the debian-custom mailinglist. Please reply there :-)

In not totally unrelated news I'm back in Europe and catching up on my various backlogs. Tomorrow I plan to connect the harddrive with the Debconf8 videos and start/continue with that backlog... If you notice any problems with the already released videos, please don't tell me, but put them in that wiki page. Thanks!

Thanks to Anton Zinoviev, http://planet.debian.org/ is subscribed to my feed for months already, but I ve never gotten around to actually post something. So let s start in accordance with tradition! My name is Michael Schutte and I m yet another Debianist from Innsbruck, Austria. I have been around since about February, and I currently maintain kbd, python-odtwriter, and libgit-ruby. Then, I participate in the Debian/Ruby Extras team. I also contribute some QA work when time allows I even managed to fix an ugly security bug in an orphaned package once. I ll try to write down things here every once in a while (without breaking Planet too often).

Just as a personal reminder for the next time I need that:

If there's a machine, for example a Xen host, with just one harddisk and you want to be somewhat safe in case of a disk failure, you can add a second disk and setup a RAID1 (mirroring). It's most simple to shutdown the machine and setup the RAID1 for the dom0. Afterwards you can start the domU VMs again when you're using LVM for their virtual disks. Following these steps will add RAID1 underneath your LVM setup:

0. /dev/sda5 is your current physical volume for your LVM, on the second (equally partitioned) disk /dev/sdb5 will be your starting point for your RAID1 (/dev/md3)
1. create a degraded RAID1 on /dev/sdb5:

# mdadm --create /dev/md3 --level=1 -f --raid-devices=1 /dev/sdb5
mdadm: array /dev/md3 started.
# cat /proc/mdstat
md3 : active raid1 sdb5[0]
135540288 blocks [1/1] [U]

2. prepare it for LVM and add it to your virtual group:

# pvcreate /dev/md3
# vgextend /dev/vg /dev/md3
Volume group "vg" successfully extended

3. move the extends from one disk to the other and be verbose to watch the progress:

# pvmove -v /dev/sda5
Finding volume group "vg"
Archiving volume group "vg" metadata (seqno 22).
Creating logical volume pvmove0
Moving 192 extents of logical volume vg/ts2-swap
Moving 1536 extents of logical volume vg/ts2-disk
[....]
Found volume group "vg"
Removing temporary pvmove LV
Writing out final volume group after pvmove
Creating volume group backup "/etc/lvm/backup/vg" (seqno 31)

4. remove the old partition from the virtual group:

# vgreduce /dev/vg /dev/sda5
Removed "/dev/sda5" from volume group "vg"

5. add the old partition to your raid and set it active:

# mdadm --add /dev/md3 /dev/sda5
mdadm: added /dev/sda5
# cat /proc/mdstat
md3 : active raid1 sda5[1](S) sdb5[0]
135540288 blocks [1/1] [U]
# mdadm --grow /dev/md3 --raid-devices=2
# cat /proc/mdstat
md3 : active raid1 sda5[2] sdb5[0]
135540288 blocks [2/1] [U_]
[>....................] recovery = 0.2% (284480/135540288) finish=39.6min speed=56896K/sec

6. let the RAID1 doing its sync and you're done.

Just as a small side note:
I don't know why the kernel don't want to boot when it get passed root=/dev/md2 as a kernel parameter. It works with root=/dev/sda3, though...

According to a recent article, 6 Common Website Mistakes That Cost Money are:

1. JavaScript or other crawler-unfriendly navigation that may impede indexing

This one is best avoided at design-time, by including Web Content Accessibility Guidelines 1.0 Level A in the design brief, but if you’ve ended up with JavaScript-based navigation on your site (check by doing “View Source” and searching for the code for your home page link and so on - if you can find it, then it’s probably not javascript) and it’s based on some template system, a webmaster can probably do a whole-site edit to put the navigation links in the page properly - or at least add a useful <noscript> tag.

2. Navigation that buries important pages within the site architecture.

The structure of the website’s files and the structure of the link menus do not need to match, so if there’s a page that you feel is important, get your webmaster to add it to the navigation links across the whole site.

3. Duplicate “pages” getting indexed under multiple URLs.

This usually happens for one of two reasons: one is inappropriately-parked domains, which is mistake 6 below and often fairly easy to fix; the other is a misbehaving web application, which you’ll need to get a programmer to fix. You can do a simple test of your web application by starting at your homepage and following links to a particular page; then open a new browser window and try to reach the same page by different links (or a site search) and compare the address bars (the bit of your browser showing http://) - do they match? If not, you’ve got this problem.

4. No keyword phrase focus in the content or conversely, keyword phrase stuffing

You can use a good word-counter on the text to see how common different keywords and keyphrases, or use the “webmaster tools” section of some search engines to see what they’re focusing on. Keyword-stuffing can be fairly easy to see. If you “View Source” on a page and there’s a large block of keyword-intensive text somewhere in it that doesn’t appear it when viewed in a browser, then it’s probably stuffing. Many sites regard stuffing as a sort of spam, so you don’t want to be found doing this. To fix these problems, rewrite the page text appropriately.

5. An optimized home page, but that’s it

Repeat your checks from problem 4 on a few pages other than the home page to discover whether you suffer from this. Also, see whether your website statistics show search engine visitors arriving at a variety of different pages (these are sometimes called “Entry Page” statistics).

6. Additional domains owned by the company are not properly redirected

If you usually use .co.uk, but you also have a .com domain, try visiting a random page on your website, then click in the address bar and replace the .co.uk with .com - what happens? If the page is Not Found, then your domains aren’t properly redirected and you need a Redirect adding to the second one. If the page displays but the address doesn’t change itself back, then your domains are probably pointing at the same webspace but aren’t properly redirected, which will mean you’re probably making mistake 3 above. Usually, the simplest way to fix this is with a conditional redirect. On Apache webservers, you can add a .htaccess file containing something like:-

RewriteEngine On
RewriteCond % HTTP_HOST  !^www\.yourdomain\.co\.uk$ [NC]
RewriteRule (.*) http://www.yourdomain.co.uk/$1 [R=permanent,L]

That even works on BT Business web hosting, by the way. If it doesn’t work, try asking your web hosting provider to enable mod_rewrite for you or ask them to suggest how to achieve the same result.

The above list of mistakes was published on Search Engine Land last week. The correction methods are all things used by my webmaster cooperative.

As everyone can see in the graph of release critical bugs, the number of RC bugs in etch had constantly risen since the release (which was already noted in some other blogs) up until last week. Since I suspected that this number included a lot of false positives, I began to triage this list last week during Debconf. I started with identifying all bugs that were filed against the version in etch, but really only meant the package in testing/unstable (but can't be interpreted correctly with version tracking alone since the version in stable, testing, and unstable is actually identical). There are a lot of reasons why an RC bug might be valid for testing/unstable, but not for stable, even though they share the same package:

Library transitions.

While most cases of library transitions can be handled with targeted binNMUs these days, there remain a few cases where a sourceful upload is needed, e.g. for renamed -dev packages and necessary source changes to adapt to new APIs.

Build-Problems caused by toolchain changes.

Newer gcc versions are often more strict about what constructs they allow. This can cause build problems in a lot of packages once the default version used is increased. Other packages that regulary cause new build problems are linux-libc-dev and dpkg-dev. Since we only require that a package is buildable in the release it is contained in, these are not valid RC bugs for stable.

Changes in the RC policy of the release team.

One example during the lenny release cycle was the use of invoke-rc.d in maintainer scripts to (re-)start daemons. Bugs were filed against a lot of packages not using invoke-rc.d before the etch release but they were only declared to be of RC severity after etch.

etc.

So as the first part of my triaging efforts I tried to identify these cases. I also looked for bugs that were listed as affecting etch due to incomplete version information (i.e. if there is no version given, the bug is assumed to affect all versions). As you can see from the bump in the graph I was able to identify about 200 bugs that met one of these conditions. We should avoid in the future to let the number of false positives grow to such large numbers. Everyone can help with that:

Maintainers

If you get bugs reported without a version number and you can verify that this bug was not present in an older version of the package, add correct "found" information. If you get a bug reported against a version that is both in stable and in testing/unstable, but not valid for stable, tag the bug appropriatly. I would recommend to err on the side of false positives though instead of on the side of false negatives!

Mass-bug filers

Most mass-bug filing that involve RC bugs should use tags to avoid creating false positives.

QA Group

Bugs filed about the proposed orphaning or removal of packages should usually be tagged, since only in very few cases these actually warrant any changes in stable.

At this point you probably ask yourself: What are the appropriate tags? This is much less clear than one might hope, since the involved tags (<suite>) changed/lost their meaning somewhat with the introduction of version tracking. Following my question about the appropriate tags on debian-release the release team and Don Armstrong for the debbugs maintainers seem to have agreed on their new meaning: A bug with suite tags affects the intersection of the set of suites indicated by its version information and the set of suites indicated by its suite tags. Next post: What about the other 600 RC bugs in stable?

Computer security is generally defined as the following:

1. The data should remain available to those who need it.
2. The data should not be available to those who don t have the right to.

The critical software component to the first item is your well-known nightmare: the backup software. Backups are easy! In many corporate environments, here is how you ensure you have backups:

• you install the backup agent,
• you call/mail the backup guy, telling him the IP of the machine and what to backup,
• you wait for his call/mail telling you it s ok.

Isn t that cool? So little work to have all your data completely safe! Wait if you look more closely, there is something wrong here. Nowhere in this process did you specify credentials to the backup guy, nor did you add anything to identify the backup server. Or to say it otherwise, once you have installed the backup agent, the backup server has full access to your data. But you didn t even specify anywhere the IP of the backup server. So this means that anyone on the network has full access to your data. Which, while trying to make your data secure, actually brings a gaping security issue on your system. Security by nullity When you install a well-designed backup agent, like Bacula, you have to specify a password for each client and make this password known to the backup server. A simple authentication protocol ensures that only the backup server is able to backup your data. However, when you install, for example, the HP Data protector agent, it starts listening on a TCP socket and (no kidding!) binds it to a restricted shell which has access to a small list of commands. The backup server only needs to connect to this TCP socket and issue commands. While this has the great advantage of simplifying the development process of the backup server, such a software has a name: a rootkit. Several other proprietary backup software have different implementations, like RPC-based or proprietary protocols, but the basis remains the same: you connect to the TCP port, and you have a way to read absolutely any file on the system. Of course, there are so-called security options, that you can buy besides the disk agent or the nifty web interface. Yes, when you buy software that is critical for your security, the very lowest level of security that you d expect from any software not turning your box into a self-service is a paying option. In the end, what prevents your data from being available to anyone on the intarweb is the ultimate solution to everything(tm): the firewall. As people are not stupid enough to use the same backup system in the DMZ, you can t simply bounce from it after having used a hole in the lousy PHP code that is never updated. (Well, that will still work in many cases since people actually are stupid enough.) No, what makes it very fun is the backup network. The hackup network A tape library is expensive hardware, therefore you only want to buy one. However, when you have several departments or, in outsourced environments, several clients, you can t simply access to the backup agents on all these servers, because of all these evil routers and firewalls that are blocking you. The solution is trivial: use the second network card on all servers to connect to a backup network. The backup network is entirely managed by backup people who have no idea of network security, and is routed all the way down to the backup server. If you are lucky, some ACLs set up by the network guy who plugged the switch will prevent some parts of the network to access some others. In all cases, it s very likely that many people in the company (or from other companies!) have full IP access to all your private servers where sensitive data is stored. That includes access to your databases setup without a password, and of course to your highly secure backup agent. That generally also includes full IP access to the lousy maintained backup server, on which security updates are never installed. But don t worry. Your data is safe. Note: no, this is not a worst-case scenario. Such badly-designed software is widespread. Such practice is widespread in several companies I know.

I’ve had little time in the four years I’ve lived in Roslindale to get involved in community volunteer efforts. My wife Rachele, however, has devoted several years to efforts such as the Roslindale Community Center, Roslindale Village Main Streets, and Roslindale Clean and Green. She has served as chair of both RCC and one of the RVMS committees. Over the past few months, I’ve been disappointed to see the RCC board endure repeated senseless attacks by a perhaps overzealous community member, Walter Michalik, who has effectively paralyzed the organization. I have nothing personal against this guy, but I thought I could do my part here by publishing an example of an email he broadcast to the committee, which I think is embarrassing enough on its own merits to need no editorializing from me, except to say any email that starts with “just so you won’t be surprised when the IRS launches its investigation” can be neither productive nor taken seriously. The context is a response to an announcement sent out pursuant to the Community Centers By-Laws scheduling a special meeting to discuss the recent upheaval on the Council. There’s no reason this should not be in the public record:

Rachele Just so you won’t be surprised when the IRS launches its investigation, be advised that YOU DO NOT HAVE THE AUTHORITY TO CALL FOR A SPECIAL MEETING. The By Laws do not allow the President/Chair to call for a Special meeting - that right is reserved for the MEMBERS to have a power over a hostile or non-performing President/Chair. It was created by a Membership wary of annimosity within its ranks in order to protect itself from falling into a useless dysfunctional body. That’s why our By Laws require the majority of the membership request it through the Chair and not the other way around. So you screwed that up too. Your continuous violation of rules and laws have proven beyond a doubt that this current Roslindale Community Center Council has violated the trust behind its IRS-designated not-for-profit status. The failures are on so many significant levels over 6 months it is readily apparent that the Roslindale Community is better served without a Community Center Council. There is no longer any excuse to allow the residents of Roslindale to be denied the services they deserve that you and the officers have no idea how to deliver. Because that’s what you are supposed to do - serve the residents of Roslindale and not preseide over a self-serving board. You’ve taken us from dysfunctional to non-functional over the past 60 days. You and your officers have already driven away our Archdale partners and now the merger is off. Congratulations. You got what you wanted. You and the officers have withheld information from the Members of the Council imposing gag orders along the way and in more than one instance spreading lies within this community. There are so very many lapses on your part and the rest of the officers that it is very apparent that the Roslindale Community Center exists IN SPITE OF the Roslindale Community Center Council - not because of it. It would be far better indeed, if the Center were run without the Council. What have you done in 6 months? All you do is avoid the work of leadership. Please, just gracefully step aside and let a real leader emerge - or dissolve the Council and let the Center run itself under BCYF before the IRS makes that happen. Of course this is my stated opinion. You can choose not to belive it. You can huddle with a group of self-serving officers and invent excuses or change the agenda away from the message and continue with your ad hominem attacks against the messenger. I’m thick-skinned. Feelings aren’t important to me - services to residents are important to me. While I have, in fact, enjoyed the confidences of others within this community and their true displesure with this Council, I will produce that to the appropriate authorities as I see fit along with the Bill of Particulars I have produced with documentation for all charges. As a founder of the Roslindale Community Center Council I can no longer support you or the officers as any legitimate authority in this community.

Technorati Tags: Boston, Roslindale, Walter Michalik

Share This

At the moment there are ongoing security issues related to web based services and DNS hijacking. the Daily Ack has a good summary of the session hijacking issue [1]. For a long time it has been generally accepted that you should configure a DNS server to not allow random machines on the Internet to copy the entire zone. Not that you should have any secret data there anyway, but it’s regarded as just a precautionary layer of security by obscurity. Dan Kaminsky (who brought the current DNS security issue to everyone’s attention) has described some potential ways to alleviate the problem [2]. One idea is to use random case in DNS requests (which are case insensitive but case preserving), so if you were to lookup wWw.cOkEr.CoM.aU and the result was returned with different case then you would know that it was forged. Two options which have been widely rejected are using TCP for DNS (which is fully supported for the case where an answer can not fit in a single UDP packet) and sending requests twice (to square the number of combinations that would need to be guessed). They have been rejected due to the excessive load on the servers (which are apparently already near capacity). One option that does not seem to get mentioned is the possibility to use multiple source IP addresses, so instead of merely having 2^16 ports to choose from you could multiply that by as many IP addresses as you have available. In the past I’ve worked for ISPs that could have dedicated a /22 (1024 IP addresses) to their DNS proxy if it would have increased the security of their customers - an ISP of the scale that has 1024 spare IP addresses available is going to be a major target of such attacks! Also with some fancy firewall/router devices it would not be impossible to direct all port 53 traffic through the DNS proxies. That would mean that an ISP with 200,000 broadband customers online could use a random IP address from that pool of 200,000 IP addresses for every DNS request. While attacking a random port choice out of 65500 ports is possible, if it was 65500 ports over a pool of 200,000 IP addresses it would be extremely difficult (I won’t claim it to be impossible). One problem with the consideration that has been given to TCP is that it doesn’t account for the other uses of TCP, such as for running DNS secondaries. In Australia we have two major ISPs (Telstra and Optus) and four major banks (ANZ, Commonwealth, NAB, and Westpac). It shouldn’t be difficult for arrangements to be made for the major ISPs to have their recursive DNS servers (the caching servers that their customers talk to) act as slaves for the DNS zones related to those four banks (which might be 12 zones or more given the use of different zones for stock-broking etc). If that was combined with a firewall preventing the regular ISP customers (the ones who are denied access to port 25 to reduce the amount of spam) from receiving any data from the Internet with a source port of 53 then the potential for attacks on Australian banks would be dramatically decreased. I note that the Westpac bank has DNS secondaries run by both Optus and Telstra (which makes sense for availability reasons if nothing else), so it seems that the Telstra and Optus ISP services could protect their customers who use Westpac without any great involvement from the bank. Banks have lots of phone lines and CTI systems. It would be easy for each bank to have a dedicated phone number (which is advertised in the printed phone books, in the telephone “directory assistance” service, and in brochures available in bank branches - all sources which are more difficult to fake than Internet services) which gave a recorded message of a list of DNS zone names and the IP addresses for the master data. Then every sysadmin of every ISP could mirror the zones that would be of most use to their customers. Another thing that banks could do would be to create a mailing list for changes to their DNS servers for the benefit of the sysadmins who want to protect their customers. Signing mail to such a list with a GPG key and having the fingerprint available from branches should not be difficult to arrange. Another possibility would be to use the ATM network to provide security relevant data. Modern ATMs have reasonably powerful computers which are used to display bank adverts when no-one is using them. Having an option to press a button on the ATM to get a screen full of Internet banking security details of use to a sysadmin should be easy to implement. For full coverage (including all the small building societies and credit unions) it would be impractical for every sysadmin to have a special case for every bank. But again there is a relatively easy solution. A federal agency that deals with fraud could maintain a list of zone names and master IP addresses for every financial institution in the country and make it available on CD. If the CD was available for collection from a police station, court-house, the registry of births, deaths, and marriages, or some other official government office then it should not have any additional security risks. Of course you wouldn’t want to post such CDs, even with public key signing (which many people don’t check properly) there would be too much risk of things going wrong. In a country such as the US (which has an unreasonably large number of banks) it would not be practical to make direct deals between ISPs and banks. But it should be practical to implement a system based on a federal agency distributing CDs with configuration files for BIND and any other DNS servers that are widely used (is any other DNS server widely used?). Of course none of this would do anything about the issue of Phishing email and typo domain name registration. But it would be good to solve as much as we can.

• [1] http://www.dailyack.com/2008/08/cookie-hijacking.html


• [2] http://www.doxpara.com/?p=1215

Share This

Ah, Dell. Seeming inventors of the tech support pit of bureaucratic indifferences, inventors of the flamingo pink Inspiron, perpetrators of fraud in New York...

I have, for over a year now, been on a crusade trying to get them to stop sending me their Dell Home and Home Office catalog to my mailbox. It has been a bundle of fun, let me tell you.

They have a nice-sounding privacy policy. It says you can opt out of all their mailings by filling out a form online. Yeah, good luck with that. First of all, there are different forms for different departments at Dell. I've filled out them all, multiple times. They do nothing whatsoever. Perhaps they use them as lists of known-good addresses to send new advertisements to, rather than lists of people to remove. Oh well.

Now, unfortunately I feel compelled to bore you with the saga so far, involving telephone hang-ups, broken privacy policies, and the like. But there is a silver lining at the end, in which I submitted a request to the postal service asking them to block Dell from sending me any more mail, and it appears that they are very likely to violate Federal Law any day now.

I have called them about it. Dealt with the old "let me transfer you to the correct department" then hang up on me ploy. Spoken to people that have promised up and down that I'll be off their list in 30-60 days. It's always 30-60 days, isn't it? Very convenient that I can't tell for 2 months whether or not they've processed my request.

I've tried their online chat. One of my attempts went like this:

Session Started with Agent (Sneha Ranga)

Agent (Sneha Ranga): "Due to circumstances that have affected Dell Communications I am temporarily unable to pull up any information. The down time is temporary. We apologize for the inconvenience, as we value your time as a customer. Please contact us back after an hour."

Session Ended

Ah Dell, only you could reach such a pinnacle of customer service. /kicking someone out of a chat room before they have a chance to say a word.

Finally, last fall, I blogged about the situation (that's the link above). Debbie from Dell red the post and emailed me. Great, I thought. She asked for my address information and catalog information and sent me a removal confirmation:

From: Debbie@Dell.com
Date: Thu, 18 Oct 2007 13:08:18 -0500
To: jgoerzen@complete.org
Subject: RE: Dell mailing list

Thank you, Mr. Goerzen, your request to have the below address
information removed from our marketing lists has been received:

[ my address here ]

We will process your request promptly. However, it may take several
weeks for some changes to take effect. If you are still receiving
catalogs after thirty (30) days feel free to email me. Sorry for any
inconvenience you may have experienced.

Thank you,
Debbie

So that was October. In December, I replied to that message, saying: "I received another mailing today, and it's been nearly 2 months since your initial message. If there's anything further you can do, I'd appreciate it." Debbie said, "I am very sorry Mr. Goerzen, I will resubmit your request." Guess how successful that was.

So in February, I manage to figure out a way to send in a support ticket without having a Dell system serial number. I wrote:

I keep getting your Dell Home and Home Office catalog. I have tried for months to get off your mailing list. I have called in, talked to people in multiple departments, who have promised to remove me from the list. I have contacted you online. NOTHING IS HELPING. This has gone on for MONTHS.

PLEASE NEVER SEND ME MAIL AGAIN, EVER, AND DO NOT RENT OR SHARE MY NAME.

My address is above.

The code on the mailing I received is: [ snipped ]

The form letter I got back said:

If you are currently receiving our catalog or mailings and would like to be removed, please visit the following web page and select the appropriate link under the "Opt-out of direct mail, phone or fax communications" heading: http://www.dell.com/OptOut

I replied, saying that form didn't work. Guess what I got back?

Thank you for signing up for Dell Email Subscriptions. Please save this email for your records.

Yes, that's right. Asking them to take me off their postal mailing lists got them to PUT ME ON their email lists. ARGH.

So they eventually manage to correctly take me off the email list, and of course promise to do the same with the postal list. This back in February.

I contacted them again in March and July, only to have a similar stupidity-laced run-in with clueless form-answer-laden Dell support reps. Each one claimed to have now, finally, and permanently removed me from the list. It never happened, and none of them lifted a finger to find out way, and no amount of begging could make them.

So, here's the good part.

Junkbusters has spent years educating people on how to get rid of unwanted mail, and documents getting a prohibitory order against the sender. It was originally designed for people that didn't want to receive obscene advertising mailings, but thanks to the happy fact that one non-adult-mailer challenged a prohibitory order all the way to the Supreme Court, you can now get prohibitory order against anyone. Yes, even Dell. (The supreme court's ruling even gave an example: you can prohibit a clothing catalog if you want.)

And last month, that's exactly what I did. The USPS sent me back a copy of the letter they sent to Dell, as well as a second page with instructions on reporting violations. Here's the letter they sent to Dell:



Somehow I get a chuckle over some Dell mail clerk trying to figure out how an 11-pound laptop is sexually provocative.

From August 25 on, it is a federal offense for Dell to send me another Home and Home Office catalog. This is a branch of criminal law, not civil law. That is, it's the maybe-go-to-jail branch of law.

How disappointed I was to receive yet another catalog from them today. If only they had waited 5 more days, I could have turned them in now.

Oh well. There's always next month's catalog. Let's just hope the clerk that received the USPS letter removed my name with a better system than everyone else at Dell uses, eh?

I’ve been installing free software for library cataloguing onto Apples recently. It’s not been great (GNU/Linux is still easier and faster, in my opinion) but it has worked much better than older Macs. As well as making servers that are finally getting easier, Apple lead the way in locked-down portable players, with their iPod and iPhone devices and iTunes service. Quite rightly, free software users campaign against Apple’s use of DRM/TPM (Technical Protection Measures). One of the rallying points has been Defective by Design (DbD), an FSF initiative since 2006. Long-time readers may remember that I took part in a DbD protest at an Apple Store in September 2006:-

“I noticed the huge till queue, which was attractive for three reasons: it had no security (I guess shoplifters rarely queue for the till), most of them are standing around waiting for something to read and they are almost certainly Apple customers, so who better to inform? After I rejoined the back of the queue a few times, a security man was watching it, so I moved off and chatted to people waiting to do hands-on tests on the upper floor for a bit [...] then I moved back to leaflet the till queue again. [...] Then, I got a bit bored and started putting the leaflets into Apple’s product leaflet dispensers (in front of Apple’s - no leaflets were removed), which soon got me rumbled.”

To me, that was fine: I dislike that type of store, which tries to look like public space, but is actually privatised and controlled. My resource use was minimal (one security man escorting me out and some Saturday kid probably “cleaned” the leaflet racks) but I passed information to many Apple customers, who are one of only two groups Apple must keep happy, ultimately. Popey is a bit upset by the latest protest idea, which he describes as:-

“urging people to book slots at the Apple in-store “Genius Bar” to ask probing questions which they already know the answer to about their company policy on DRM and Free software. The ‘Genius Bar’ is an official Apple support avenue for their customers, and is a service provided inside many of their stores.”

It’s also been criticised by neuro and Pete along similar lines. I agree. It’s wasting both side’s resources. Apple don’t really need to keep their staff happy (they can hire new staff) and it offends Apple customers and shareholders. It’s not even efficient: each unit of Apple employee time wasted costs a unit of DbyD worker time and who’s got the most resources there? What are FSF thinking? It’s stupid, even without the bad press it’s had. So, this brings me in a roundabout way to answering

“I am aware of the Linux Foundation and the FSF and others, but how does SPI differentiate itself and its goals from those organizations?”

from why people don’t join SPI. In my opinion, a key difference of Software in the Public Interest is that it is managed by free software developers in a more-or-less democratic fashion. Even though I’ve failed in two elections, I still think it’s better than the alternatives. No project as unpopular as wasting Apple Store helpdesk time would ever get backed by SPI, would it?

Some interesting two days have just passed. I took the airplane from Mar del Plata to Buenos Aires on the 17th, slept one night in a hostel on a mattress that felt like stone, and went to the 'Circulo del Officieles de la Mar' (or some such, I don't know spanish) where I had to give a talk. The oral feedback I've gotten so far indicates that people did appreciate it, which is good. I had originally planned to do some sightseeing in Buenos Aires after my talk, but me being tired and not immediately finding anyone else to go with me changed those plans. Instead, I just stayed at the C.O.M., where we played some Mao. Amongst other things. About an hour before I left the place, Holger Levsen gave me a tripod, with the request to take it with me to Belgium, and keep it until FOSDEM. As that would also give me the use of an tripod, and being the photography hobbyist that I am, I was too happy to oblige. There was just one problem: the suitcase that contained my luggage already had a bulge from here to tokio, so adding it was a no-no. No problem, I thought. I'll just keep it in my carry-on luggage. There's some space left there, and I certainly didn't have 5kg of carry-on luggage yet. Problem solved. The 'security' in EZE is a joke; the security officer doesn't even look at the screen while X-raying my luggage. Well done. So I fly from EZE to MAD on the first, 11-and-a-half-hour leg of my flight. Madrid Barajas is a crazy airport, with its own private metro. Somewhere between gates R-something (where IB6844 from EZE arrived) and J40 (where the flight to BRU will depart from), we need to go through a security checkpoint. Strange, since I hadn't had to do that on my way in from BRU to EZE; but of course you have no choice, so I comply. As I've gotten used to by now, what with the mess of cabling in my laptop bag and the other metal objects there, I get a baggage check. The security officer who goes through my very messy bag asks me to open the bag containing the tripod, so he can see it. He then promptly decides that it is a "sharp" object, and that it is "not acceptible". Here the fun begins. Of course, trying to argue with a security guard isn't the best idea if you want to be allowed on any flight, ever, so I don't even try that. I ask him 'can I check that in, instead?' and he tells me how to go back, to the exit, and to the departures hall where I can have that little tripod get checked in. So I go there. The lady at the check-in desk tells me that she cannot check the tripod in, because the flight has already been closed and I'm too late. I tell her I'm from a connecting flight, and that I can't help that. She says she knows. So I ask her what my other options are. She tells me I could try to get on the next flight to Brussels, which is at 20:00 (rather than the 16:20 flight that I was booked on). I consider that an acceptable compromise, so ask her how to get that done, and she refers me to the ticket sales desk, some 10 meters to my left. Unfortunately, the 20:00 flight turns out to be completely booked, and there are no other flights on the same day. The best alternative he can give me is a flight to Amsterdam at 19:20, which would cost 'a little more'. I ask him 'how much more'. He starts looking up things in his computer, and after a few minutes tells me that he cannot refund my ticket for the 16:20 flight and that a flight to amsterdam would cost me 429. At this point I get somewhat angry, and tell him that this is not acceptable. He refers me to the customer service desk right across. The lady at the customer service desk is friendly, but firm: she cannot help me. Either I leave the tripod behind, or I pay whatever the ticket service asks me—she takes my word that it is 429, because she cannot check it, and cannot change it, either. I get more agitated, and ask to see the supervisor. She points me to a lady a little across who can 'call' the supervisor. At this point it is 15:55 As I reach that lady, apparently someone else had just asked for the same thing, and starts arguing with a guy in black uniform, who appears to be the supervisor, about the fact that he's got more than 30kg of luggage which he isn't allowed to take along with him. Dude, not a single airline will let you do that, no matter how much you argue. I let them argue for about 10 minutes, all the while nervously looking at my watch, but then I interrupt with 'please, I don't have much time left before my flight leaves'. That gets his attention. In our two-minute conversation, the guy in black backs up everything the other three people have told me, adding that '429 is reasonable; your luggage is already on the flight, and it will cost Iberia a lot of money to delay the flight and get it off'. I curse. He also seems surprised at the notion that I'm not allowed to take a tripod in my carry-on luggage. Hmm. Figuring I have nothing to loose, I curse one more time, and make a run for it. When I reach the security checkpoint, I ask to be allowed to skip the queue in front of it, on the grounds that I have only 15 minutes left before my flight leaves. Mind you, this is not the same security checkpoint as the one where I was rejected about an hour earlier; this is the security checkpoint between the departures hall and the secure area, whereas the other security checkpoint was one between two different terminals. I am granted my request, and quickly throw everything in the X-ray boxes. They need to go through my bag again. I help them, showing where everything conspicuous is, and opening my flute box so they know what's in there. He looks at everything, and signals that I can move on. Hang on, I think. This can't be right. 'Okay,' I ask? 'Yes,' he says. I run for the nearest info desk, show them my boarding pass, and tell them 'please, my flight leaves in 10 minutes, can you tell me where to go?' They are quick and efficient. Gate J40. I make a run for it again, and just make it in time. Of course, I could forget about my plans of finding a power outlet, plugging in my cell phone and calling my parents, but that's not a big deal—there's a train station at Brussels Airport. So what have we learned? Security on airports is random, arbitrary, non-standardized, and utterly stupid. Items that pose no danger at all—plastic bottles containing more than 100ml of water, small quantities of yoghurt that are over 100grammes but not over 100ml, tripods—are not allowed, while on the other hand I've occasionally gotten through security carrying knives and tools that are clearly forbidden by the rules. We've also learned that the rules very much depend on the particular person checking you. If you're refused for an item that is not a knife or a gun or something else similarly problematic, it may help to just find a different security checkpoint and try again. It's not as if they make any sense, anyway.