Search Results: "pm"

3 November 2008

Enrico Zini: rc-buggy packages sorted by popularity

rc-buggy packages sorted by popularity After doing a round of tag reviews so that we release lenny with the latest tag contributions, I still had a bit of spare time. Since people are urging to fix the last bugs I thought I could contribute a list of RC-buggy packages sorted by popularity: 
#!/usr/bin/python
import urllib2
import sys, re #, gzip
PKGRE = re.compile(r'<strong>Package: </strong><a href="[^"]+" name="([^"]+)">')
POPCONRE = re.compile(r'^(\d+)\s+(\S+)')
PKGMAPRE = re.compile(r'^([^(]+)\(([^)]+)')
COMMASPLIT = re.compile(r'\s*,\s*')
# Allow to download other package views
if len(sys.argv) > 1:
    url = sys.argv[1]
else:
    url = "http://bts.turmzimmer.net/details.php"
def read_packages(url):
    "Download the list of rc-buggy packages"
    fd = urllib2.urlopen(url)
    for line in fd:
        mo = PKGRE.search(line)
        if mo:
            yield mo.group(1)
# FIXME: this is a deprecated data source, but at the moment it is just what
#        is needed
def read_bintosrc(url = "http://qa.debian.org/data/ddpo/results/ddpo_packages"):
    "Download the binary to source package map"
    fd = urllib2.urlopen(url)
    for line in fd:
        mo = PKGMAPRE.search(line)
        if mo:
            src = mo.group(1)
            bin = COMMASPLIT.split(mo.group(2))
            for p in bin:   
                yield p, src
def read_popcon(url = "http://popcon.debian.org/by_inst"):
    "Download popcon information"
    fd = urllib2.urlopen(url)
    # It doesn't work on urllib2 objects because it wants tell()
    #fd = gzip.GzipFile(fileobj=fd, filename=url, mode="rb")
    for line in fd:
        mo = POPCONRE.match(line)
        if mo:
            yield mo.group(2), int(mo.group(1))
print "Reading mapping of binary packages to source packages..."
bin2src = dict(read_bintosrc())
print "Reading popcon scores..."
# We actually want the score of source packages, and for it we use the score of
# its binary package with the highest score
scores = dict()
for pkg, score in read_popcon():
    pkg = bin2src.get(pkg, pkg)
    if pkg not in scores:
        scores[pkg] = score
#scores = dict(read_popcon("http://popcon.debian.org/by_inst.gz"))
print "Reading list of rc-buggy packages..."
# The bug page has a mix of source and binary packages: convert all to source
# packages
packages = sorted(read_packages(url), key=lambda x:scores.get(bin2src.get(x,x), 0))
for p in packages:
    print scores.get(bin2src.get(p,p), 0), p
This can be run on people.debian.org as ~enrico/popzimmer (0 is the rank of packages that for some reason cannot be mapped to any rank): 
$ ~enrico/popzimmer 
Reading mapping of binary packages to source packages...
Reading popcon scores...
Reading list of rc-buggy packages...
0 roxen-fonts-iso8859-2
2 dpkg
11 pam
61 glibc
61 libc6
123 initramfs-tools
149 bind9
150 grub
180 portmap
213 nfs-common
279 libgl1-mesa-dri
298 libsnmp-base
298 snmpd
309 avahi
397 mysql-dfsg-5.0
408 xserver-xorg-input-wacom
409 xserver-xorg-video-vesa
428 docbook-xml
499 openoffice.org-core
567 xine-lib
600 evolution-data-server
669 eog
671 xserver-xorg-video-intel
691 epiphany-browser
691 epiphany-gecko
695 kdelibs4c2a
698 uswsusp
754 guile-1.6
959 gpm
964 cups
990 kdebase
1002 ghostscript
1069 giflib
1076 xulrunner-1.9
1177 libapache2-mod-perl2
1217 vlc-nox
1229 kernel-package
1311 kdeutils-dev
1520 libparted1.8-10
1767 wireshark
1785 selinux-policy-default
1908 dia
1943 htop
1983 boost
2140 cantlr
2145 transmission
2159 emacs22
2312 swfdec-mozilla
2449 eclipse
2449 libswt3.2-gtk-java
2728 lilo
2898 libnss-ldap
3013 java-access-bridge
3051 blender
3099 libtemplate-perl
3499 fglrx-atieventsd
3602 virtualbox-ose
3619 jfsutils
3806 anjuta
3962 kbuild
4076 libgnustep-gui0.14
4222 libwebkit-1.0-1
4332 rosegarden
4412 matplotlib
4412 python-matplotlib
4950 mail-notification-evolution
5195 nut-cgi
5273 sbcl
5538 ruby1.9
5679 kmymoney2
6076 websvn
6147 miro
6264 gnat-4.1
6431 gallery2
6478 gnome-chess
6512 joystick
6862 istanbul
6937 wordpress
7476 luatex
7635 csound
7635 python-csoundac
8257 libpam-mount
8471 wmcpuload
8631 nbsmtp
8642 oxine
8774 axiom
8949 libengine-pkcs11-openssl
9052 clamav-getfiles
9221 open-iscsi
9262 ptex-bin
9701 egroupware-core
10012 debian-cd
10127 libjogl-java
10453 libjson-xs-perl
10698 gcjwebplugin
10736 request-tracker3.6
11294 python-extclass
11518 rkward
11600 alevt
12929 otrs2
13205 bindgraph
13956 icecc-monitor
14223 netmaze
14505 cdebconf
14526 mumble
14687 fnonlinear
14724 recite
14789 jbidwatcher
15766 libnss-ldapd
15812 glpi
15905 lustre-source
15982 win32-loader
16232 ampache
17168 libghc6-missingh-dev
17311 acl2
17830 apertium
18462 emacspeak
18469 glassfish
19257 isight-firmware-tools
19719 libnagios-object-perl
20144 libwx-perl
20465 dns2tcp
21614 python-kinterbasdb-dbg
21690 hf
21902 libcobra-java
22365 tomcat6
22573 libphp-snoopy
23501 coco-java
23926 boxbackup-server
24009 libghc6-wash-dev
26323 libdesktop-notify-perl
26439 opendb
28426 adolc
28795 xorp
29986 libgdata-java
31262 libjboss-serialization-java
31472 pixelpost
31658 cournol
32939 mediamate
34365 gforge-plugin-scmcvs
34659 libgearman-client-async-perl
42233 libhamcrest-java
48076 mahara
55303 mxallowd
85881 roxen-fonts-iso8859-1
So, if you don't know from what package to start fixing bugs, "Begin at the beginning,", the King said, very gravely, "and go on till you come to the end: then stop."

Michal &#268;iha&#345;: Bad translations status

I know phpMyAdmin translators will love us for this, but I just commited translatable messages for new setup script. This leads to situation when even the best translations are at 75% (see statistics). I hope our brave translators will manage to handle these 350 new messages for upcoming 3.1 release, at least for major languages, which were always close to 100%. PS: Looking for translator to Czech!

2 November 2008

Julien Valroff: Laptop as desktop computer replacement

I am currently studying the possibility to change my current desktop computer for a laptop. This is the first time I am looking at these powerful laptops, as I now only use cheap laptops for small office use (email, web etc.) when traveling. My aim is now to use a laptop for my daily tasks: picture editing, small development, multimedia etc.

Ideal features would be: I do not care about the weight, as it would mainly stay on my desktop. I currently have a desktop computer with 2Go RAM, an Intel 4400 Dual Core CPU (@ 2Ghz) with 200Gb hard disk - though it is enough at the moment, I would prefer the laptop to be more powerful in order to be able to run smoothly with future applications which will be released in 2 or 3 years! Ideally, it would be around 1,000 (obviously, the cheaper the better, but I want something reliable for a few years if possible). Last but not least, it needs to be 100% compatible with GNU/Linux (Debian in particular) - I do not care about compiling kernel modules, but would rather prefer they are free (as in free speech). Do you have any experiences with such laptops?
What would be your recommendations? Is my idea something totally weird (I mean, should I keep my current desktop computer)?!

1 November 2008

Uwe Hermann: Google Tech Talks: coreboot (aka LinuxBIOS): The Free/Open-Source x86 Firmware

coreboot Google Tech Talk 2 Here's a nice opportunity for everyone to learn more about coreboot, a Free Software / Open Source firmware/BIOS for x86 PCs. Ron Minnich, founder of the LinuxBIOS (now called coreboot) project, Peter Stuge of Stuge Konsult, and Stefan Reinauer of coresystems GmbH have given a presentation for the Google Tech Talks series recently. The topic was (of course) coreboot, its history, goals, features and technical details, surrounding tools and libraries such as flashrom and libpayload, as well as an automated test system for running a hardware test-suite upon every checkin in the coreboot repository. coreboot Google Tech Talk 1 A video of the talk, aptly named coreboot (aka LinuxBIOS): The Free/Open-Source x86 Firmware (134 MB), is available from Youtube, get it for instance via: 
  $ apt-get install youtube-dl
  $ youtube-dl http://www.youtube.com/watch?v=X72LgcMpM9k
The talk includes various demos of coreboot and various payloads you can use with coreboot. One nice example is the TINT payload, a Tetris-like game for Linux (apt-get install tint for the curious), which has been reworked to be usable as a coreboot payload. coreboot Google Tech Talk 3 So, yes, you can now put Tetris in your BIOS ROM chip and play it from there (no hard drive required). Other demos included some cluster nodes with coreboot, and a "normal" x86 desktop board booting coreboot + Linux in a very few seconds (much room left for optimizing there though, if you really want to get into fast booting). Check out the full talk for more infos, and if you're willing to give it a try (see the list of currently supported boards), contact us on the mailing list or join the #coreboot IRC channel on Freenode.

30 October 2008

Marc 'Zugschlus' Haber: Nagios, Parent Hosts, and traceroute on the Internet

Nagios has the - very useful - feature of “parent hosts”. If it deems a host A being down, it first checks its parent host, B, and reports A only as down if B is up. This goes back recursively until a host with state “up” is found and only the first “down” host is actually reported. This keeps on-call people from being bombed with alerts in case of major network outages and makes sure that the alerts that are actually sent out do reasonably accurately describe the actual outage. As an individual who has some “external” servers in various data centers on the Internet, I would like to not be alerted multiple times that my servers at ISP C, D, and E are down if there is an outage at the ISP F hosting my Nagios installation or at one of the various exchange points temporarily rendering the servers unreachable (without me being able to do anything). The solution sounds easy but is surprisingly hard. How one could solve the issue For each host in the manual nagios configuration, do a traceroute and generate host stanzas for each of the hosts found: 
$ sudo traceroute -I icmp -n -A 213.239.240.200
traceroute to 213.239.240.200 (213.239.240.200), 64 hops max, 28 byte packets
 1  <snip>
 2  217.243.221.150 [AS3320]  1 ms  0 ms  0 ms
 3  62.154.10.106 [AS3320]  1 ms  2 ms  2 ms
 4  217.239.40.226 [AS3320]  51 ms 217.239.40.234 [AS3320]  3 ms 217.239.40.226 [AS3320]  60 ms
 5  193.159.226.2 [AS3320]  3 ms  3 ms  3 ms
 6  213.239.240.200 [AS24940]  6 ms  7 ms (TOS=238!)  8 ms
$
This sounds easy and surely can be accomplished in a few hundred lines of perl. However, this makes it necessary to parse Nagios configuration if one wants to obtain the list of configured hosts directly from there. Unfortunately, the syntax of Nagios configuration is “historically grown” which makes it quite illogical and awfully hard to parse - this would easily double the size of the traceroute-based configuration generator. Additionally, the (real) traceroute given above shows three other issues that might arise from such a setup. Dynamic Routing Network paths are not as static as a traceroute suggests. On the Internet, the ISPs use BGP to connect their networks together and to ensure connectivity. In case of network reconfiguration (which may be caused by desired technical or policy changes or actual failures), the traceroute to a given network host may change. Considerable experience is needed to judge whether a change in the traceroute is only short-term while one looks at it, or long term meant to stay. Of course, only long term changes should be reflected to Nagios configuration. The possibility of changes makes it necessary to distinguish between “manually” and “automatically” generated Nagios host entries and to have a mechanism to quickly re-generate the “automatic” entries according to the current situation found on the network. Alternative Network Paths If you look at hop 4 of the traceroute given above, you see two IP addresses. This is a common “one packet left, one packet right” setup where two identical lines are sharing the load and this one sees different interfaces of the next hop router. Which IP address should one put into the parent host declaration then? Later Nagios versions supposedly are able to handle multiple parents to a host, but the docs of course don’t cover this complex configuration and I didn’t yet find the time to find out how this mechanism works and whether it can be used to represent a network layout like that in Nagios. Unpingable Routers For some network operators, ICMP echo request packets (ping) pose a significant load on their network equipment. To remedy that load, they have disabled ICMP echo reply transmission on these machines to keep the load down. This makes the router still show up in a traceroute (as this is done by sending packets with short TTL values and making them expire in transit, resulting in an ICMP TTL exceeded message being sent back), but it cannot be pinged. Hop Number 5 on the traceroute above is such a box: It shows up fine in the traceroute, but pinging it will always return “DOWN”. Currently, I simply omit such hosts from the Nagios configuration, but this may result in extra and inaccurate alerts once the unpingable host itself fails. A different possibility would be to check the host’s availabilty by sending an appropriately TTLed packet either to the host itself or to a host that is “behind” that host. The latter introduces even more complexity since one needs to find out a host “behind” the unpingable box, but this may be necessary since in the case of Hop Number 5 above, the IP address of the hop does not even seem to be in AS3320’s internal routing and tracing to the hop itself stops well before the expected place of the host regardless of from where one traces. Conclusion This is a surprisingly hard issue if one wants to do network monitoring while generating accurate alerts. Surely, it’s a topic that needs to be addressed in network monitoring, so I am quite interested in how other people tackle this. Please comment! Thanks in advance.

Wouter Verhelst: Reading the "Metro" on my Hanlin V3

It was recently brought to my attention that the free newspaper "Metro" has an online version, where you can download the entire newspaper in PDF form; or use it in an online version, which consists of a series of images of each page, where if you click on an article, the usemap of that image is a link to the relevant article in HTML form. Of course my hanlin doesn't have a webbrowser (it has a somewhat limited HTML parser, but that does not understand even hyperlinks, let alone usemaps), so the PDF version is what I need. Unfortunately, the only option which "Metro" provides is a set of one-page PDF files. They are somewhat readable on my hanlin (someone with worse eyes than my own would probably have trouble reading the small letters, but for me it's not a problem); however, the fact that rather than just using the 'page turn' buttons on my hanlin I have to close the current file, open the next, and reconfigure the zoom all over, means that this multi-PDF thing isn't exactly great, and that I'd prefer just getting one PDF file instead. So I got to work, and tried out a few things. There doesn't appear to be anything to merge multiple PDF files into one. There is 'psmerge', but that only does PostScript files, not PDF ones. But that's not a problem, because PDF files can easily be converted to postscript and back, right? Well, no. The first pdf-to-postscript converter that I tried was 'pdf2ps', a wrapper script around ghostscript. Unfortunately, using that results in some ugliness: original PDF source, as rendered by Xpdf output of pdf2ps, as rendered by gv As you can see in the second image, using ps2pdf results in some quality loss. Ghostscript apparently doesn't understand the letter 'A' very well, and (more importantly) loses the anti-aliasing that is part of the pdf file. Converting this file back to PDF and storing it on the hanlin results in an unreadable text. But it gets worse. Once I had converted all those PDF files to postscript using pdf2ps and merged them with psmerge, the output was gibberish. Or Klingon, take your pick: psmerge output, as rendered by gv In case you were wondering, yes, that is the exact same fragment (counting lines sucks) at the exact same zoom level. Clearly this was a dead end. I found that there was a second pdf-to-ps converter, called 'pdftops', which uses the xpdf code base to do its thing. This converter produced clearly better PDF output; I could not see any difference between the Xpdf rendering of the original file, and the gv rendering of the pdftops output. Also, the psmerge output does not garble things as badly: better psmerge output, as rendered by gv Sadly, it still loses anti-aliasing, and thereby the readability of the document on my ebookreader device. Moreover, the pdftops output seems to confuse psmerge, to the extent that it hangs on some pages. Another approach I tried was to import the pdftops output into scribus, and create a multi-page PDF file with that program. Unfortunately, however, scribus does not seem to like the pdftops output. Xpdf also has a 'pdftoppm' converter. With that, I was able to create a multi-page .pdf file that was not garbled, and did still contain anti-aliasing. Unfortunately, since the PPM format is a raster image format, the PDF file that is created in this way does not scale very well, resulting in artifacts and, again, an unreadable PDF file on the hanlin device. So it appears that for now I'll be stuck with storing multiple files on my device. Sigh. I wish that wouldn't be necessary... Update: So I needed pdftk.

29 October 2008

Benjamin Mako Hill: An Invisible Handful of Stretched Metaphors

The following list is merely a small selection of scholarly articles listed in the ISI Web of Knowledge with "invisible hand" in their title: And, finally:

28 October 2008

Christian Perrier: Completing the final l10n NMU sprint

I'm now reaching the very bottom of the Debian localization radar that points out packages with pending debconf l10n bug reports. About 20 packages are currently still under work (either waiting for the maintainer's answer, or in a translation update round...or waiting for the package to go out from DELAYED). That should be completed in about 10 days. The remaining packages are those with very recent l10n bug reports and I can't decently nag maintainers for bugs sent since less than one month. The result is now that we have 3 languages that reached near 100% completeness: German, French, Portuguese. Czech is over 97%. Swedish has been dramatically revived and now over 85%. And some more translation teams such as the Basque, Russian, Italian and others have taken this last opportunity to get as many translations as possible in the game. The overall quality of several packages I NMU is not optimal. I often refrained myself from trivial fixes as this is not the policy during freezes but that's really sad. I might initiate a QA NMU campaign when we enter the squeeze development round, to fix all those trivial lintian warnings (obsolete debhelper compat level, maintainer scripts ignoring errors, debian/rules ignoring make errors, typos here and there, etc.) Anyway, once this l10n NMU stuff is done, the question becomes: what will I do now and until we release lenny? Maybe come back to the French translation of the Developer's Reference which I worked on during the trip to Debconf8, then left aside since then....

Russell Coker: Jabber

I’ve just been setting up jabber. I followed the advice from System Monitoring on setting up ejabberd [1]. I had previously tried the default jabber server but couldn’t get it working. The ejabberd is written in Erlang [2] which has it’s own daemon that it launches. It seems that Erlang is designed for concurrent and distributed programming so it has an Erlang Port Mapper Daemon (epmd) to manage communications between nodes. I’ve written SE Linux policy for epmd and for ejabberd, but I’m not sure how well it will work when there are multiple Erlang programs running in different security contexts. It seems that I might be the first person to try running a serious Jabber server on SE Linux. The policy was written a while ago and didn’t support connecting to TCP port 5269 - the standard port for Jabber inter-server communication and the port used by the Gmail jabber server. The ejabberd has a default configuration file that only requires minor changes for any reasonable configuration and a command-line utility for managing it (adding users, changing passwords, etc). It’s so easy to set up that I got it working and wrote the SE Linux policy for ejabberd in less time than I spent unsuccessfully trying to get jabber to work! It seems that Jabber clients default to using the domain part of the address to determine which server to talk to (it is possible to change this). So I setup an A record for coker.com.au pointing to my Jabber server, I’ll have the same machine run a web server to redirect http://coker.com.au to http://www.coker.com.au. For Jabber inter-server communication you need a SRV record [3] in your zone. I used the following line in my BIND configuration: _xmpp-server._tcp IN SRV 0 5 5269 coker.com.au. Also for conferencing the default is to use the hostname “conference” in the domain of your Jabber server. So I’ve created conference.coker.com.au to point to my server. This name is used both in Jabber clients and in sample directives in the ejabberd configuration file, so it seemed too difficult to try something different (and there’s nothing wrong with conference as an A record). I tried using the cabber client (a simple text-mode client), but found two nasty bugs within minutes (SEGV when a field is missing from the config file - Debian bug #503424 and not resetting the terminal mode on exit - Debian bug #503422). So I gave up on cabber as a bad idea. I am now testing kopete (the KDE IM client) and GAIM aka Pidgin. One annoying bug in Kopete is that it won’t let me paste in a password (see Debian bug #50318). My wife is using Pidgin (formerly known as GAIM) on CentOS 5.2 and finding it to work just as well as GAIM has always worked for her. One significant advantage of Pidgin is that it seems impossible to create a conference in Kopete. Kopete uses one window for each chat and by default Pidgin/GAIM uses a single window with a tab for each chat (with an option to change it). I haven’t seen an option in Kopete to change this, so if you want to have a single window for all your chats and conferences with tabs then you might want to use Pidgin/GAIM. Another annoying thing about Kopete is that it strictly has a wizard based initial install. I found it difficult to talk my mother through installing it because I couldn’t get my machine to see the same dialogs that were displayed on her machine. In retrospect I probably should have run “ssh -X test@localhost” to run it under a different account.

Share This

Ond&#345;ej &#268;ert&iacute;k: Google Mentor Summit III

On Sunday I checked out from the hotel and enjoyed the breakfast, basically the same as the first picture here. Which is cool, the best way to start a day is with a good breakfast in the morning.



At Google I finally met with Bart Massey, who works at the Portland State University where I was an intern (in 2005) and also he was sponsoring one SymPy GSoC student in 2007.


Then we started to port SymPy to Jython with Philip Jenvey. I first installed openjdk-6, which is in Debian main, then checked out Jython svn, typed "ant" and it just worked. This is so awesome, that we finally have a truly opensource and fully working java implementation in Debian. Philip fixed some bugs already, so SymPy imports just fine. Most of the tests run, but also quite a lot still fail.

We managed to isolate two bugs: issue 1158 and 1159. Then there was an annoying recursion bug, that took us several hours to dig into and I then had to leave at 6pm without a clue. But Philip kept saying, I just need to look more at the java stacktraces that Jython was generating and I'll figure it out. And he did! The next morning he filed an issue 412 in the PyPy tracker, because they seem to have the same problem. There are subtle differences in how to handle __mul__ and __rmul__ methods that are not well documented, so it works in CPython but fails in Jython/PyPy. I think we can fix that in SymPy too in the meantime, so I am very excited, because it seems SymPy will work on top of Jython soon.

There was also a git vs hg session with both hg and git developers. Since I recently switched from hg to git for all stuff where I can choose the vcs, it was very interesting for me. And I must say I am glad I switched. Mercurial works fine enough and with the recent rebase feature I think it is usable, but git just has a bigger momentum at the moment and it just doesn't get in my way.

During the day Steve sent an email to the GSoC list that Dirk forgot his jacket in the hotel room, so I said to myself, haha, that's a pity. And then I realized, oh shit, I left my jacket there as well. They didn't have it in the evening when I stopped there, but fortunately they sent me an email yesterday that they found it, so I'll stop there on my way back.

We also took couple of group pictures, here is just my lame attempt (I am sure people will soon post better ones):


On Monday morning I met with Fernando Perez, the author of ipython, he know works with Jarrod. Then I rented a car and came to visit Brian Granger and his family in San Luis Obispo.

We went to the beach in the evening:





And had a dinner together. Brian showed me his options pricing example he did with SymPy, so I made him create an issue 1175 with the code.
Then we played with the parallel stuff that Brian contributed to ipython and we tried sympy with it and to our disappointment, we found pickling bugs in sympy, it works with protocol 0, but not protocol 2, see the issue 1176 and 1177. This almost smells like a bug in Python itself, but I need to investigate that later more deeply.

Today is Tuesday and we'll work on doing atomic physics calculations with sympy. See also Brian's email to the sympy list about it.

27 October 2008

Decklin Foster: We could customer care less

(And now, a cathartic "I hate this company and I am never patronizing them again, please don't either" post. I should have done ones for at least Speakeasy/Covad and IBM's server division last year. Ah well.) I went to go register a new domain yesterday (announcement soon! Probably this weekend). I've been using Joker for a while, IIRC on the principle that the cooks were using it so it must be decent enough. While registering, I had a bunch of problems with the horrid, useless "Verified by Visa" popup [1] that you see everywhere these days, and eventually this must have looked like suspicious activity to the bank so my card was declined. I only actually found this out when I got mail saying my order was canceled. Okay, sure. I wrote back and explained what I thought was going on and that I'd like to resolve it. Another half hour later, I got this response: 
Hello,
please use
   https://joker.com/goto/support/
for all support inquiries.
Regards, your Joker.com team
This is complete and utter bullshit [2]. If a company is going to mail me, I expect to be able to mail them. I have a mail client; it manages communication the way I want it to, and runs my preferred editor to compose messages (which is, in fact, how I am writing this very blog post). I am sick of dicking around on JavaScript-requiring web forms that all work differently and typing in postage-stamp-sized little textareas. Anything I do type in them is lost, because unlike my email client, web forms don't save sent messages unless the authors feel like letting you have a Cc:. When a company forces me to do this, I take it as a sign of disrespect. I tolerate a web browser [3] for reading pages; "applications" are universally painful [4]. One of the things I used to like about Joker was the PGP mail interface. AFAIK, they have not killed it (I didn't bother to check), but with automatic "we don't want to listen to you unless you inconvenience yourself" bounces like this, what's the point? I surmise that there is, or was, some smart person there who understood how (and more to the point, why) to hook a pseudo-mailbox up to a software system, and that they have been overridden by someone in management who realized it's much easier to have dime-a-dozen webmonkeys hook a form up to the same system since 90% of users just don't care (and even people who do notice and dislike this, but are not as inflamed as I, have come to expect it because everyone else refuses mail, so, y'know, pick yr battles son, etc.). Not the sort of culture I put faith my in. The irony of it all was that I was registering this domain to run a service I had decided to create specifically because of another site refusing mail and directing me to an even lamer web form. That would take incoming mail and, you know... process it with software. Suffice it to say, I am no longer going to be their customer. In deciding who to use instead, I figured I'd do a survey of where the domains in that "Subscription" column [5] on Planet Debian were registered, but... WHOIS is basically useless. Every server just returns free text, formatted differently by (apparently) every implementation under the sun. Cheaply parsing something out from .com/.org/.net is possible, but InterNIC kindly blacklists your IP after making more than a few requests in a few minutes. I guess I'm just gonna go with Gandi (but other suggestions would be welcome). In somewhat unrelated developments, I went to nic.at to update my nameservers for Where the Bus At? last week. There was no authentication or anything on the request form, so I just filled it in and sent it off. I got some automatic mail saying I need to print out a PDF, sign it, and fax it internationally. Annoying, and hardly as secure as mailing them with PGP, but whatever. But then, also yesterday, I got another mail saying the update was complete (lo and behold, it was). I am now somewhat concerned about the security of my domain: it seems like anyone can come by and put something in the form and if I'm not around to notice the courtesy mail and ask that they stop the request, it'll eventually go through, no questions asked. I have not yet written them to figure out what the deal is, though. Kinda burned out. I guess I didn't really have high hopes for dealing directly with a ccTLD registrar (this was the first time I've done it... I can't believe I blew 60 on a cutesy domain name) rather than a reseller who competes in a market, but then, I go and google "domain registrar" and look at all the AdWords dollars spent trying to compete with GoDaddy [6] and just kind of want to put my head in my hands. On DJB's DNS pages there's this bit about setting up a domain. It doesn't say "How to (buy register whatever) a domain name". It says, "How to receive a delegation from .com". Which is of course, how it works. And what I want to buy. I don't want "parking" or even gratis nameservers. Just a delegation from .com. Please. No AdWords came up when I googled for that phrase to copy the link. Sometimes I guess markets just sink to the bottom. Anyway. I feel like there's a free-software angle here. My continuing irrational hatred of using other people's forms, web-based mailing-list substitutes, nameservers, etc. stems not so much from their suckage but from the fact that there is no longer any software there, in front of me, for the four freedoms to possibly apply to. Being able to run your mail reader for any purpose doesn't win you much if no one uses mail. I don't really know what to do about this.
[1]It was only twelve dollars! If I go to hipster market or some other place with brand-new POS systems they don't even make me sign a paper slip for less than $20 or so. But this is the sort of thing dreamed up by people who think good security is setting a cookie to denote that I've answered "what was your first dog's name" or whatever. IF YOU WANT TO BREAK INTO MY BANK ACCOUNTS: I've set the answer to every single one of these questions to "security theater". Easy to remember.
[2]Not to mention a glaringly RESTless URI.
[3]Also, as a rule I normally try not to have anything up on my desktop that I can't close and re-open at any time, thanks to screen, MPD, emacsclient, actually using my browser's bookmark function and resisting the lure of tab-bar.js, etc. Web browsers are supposed to suck at preserving state; HTTP is stateless. See also here (and for further reading on that in the Haskell world, check out Yi. I'm hoping to switch to it someday).
[4]Debbugs, for example, may not have the slickest interface, but if I want to do flags, labels, archiving, threading, or whatever, I can; I'm not at the mercy of some front-end web developer. But we've all heard this litany before.
[5]Thanks, Hpricot!
[6]Even if GoDaddy's customer service were completely hacker-friendly, I would not use them, because their president Hates America.

Colin Watson: Totem BBC plugin

A while back, the BBC approached Canonical about providing seamless access to unencumbered BBC content for all Ubuntu users (in the UK and elsewhere). We agreed to approach this by way of a plugin for our primary media player, Totem, and asked Collabora Multimedia to do the plugin development work. The results are in what will shortly be released as Ubuntu 8.10, and are looking really rather good. At the moment the content available from the BBC at present is mostly audio, but support for video is in place and the feed is expected to be fleshed out here over time. We have a genre classification scheme in place, and will see how that scales as the amount of available content grows. The code has been submitted upstream, although there are still a few issues to work out there. This is not the same thing as iPlayer; all the content available here is DRM-free. Some of it is geographically restricted to the UK, and these restrictions are handled on the server side to make sure that the client is free of encumbrances. Christian Schaller from Collabora posted about this a little while ago. Since then, the UI has been improved somewhat and some I/O issues have been fixed to the point where we felt comfortable enabling the BBC plugin (as well as the YouTube plugin) by default in Ubuntu 8.10. Here's a screenshot of the current interface. This is exciting stuff with a lot of potential. To try it out, run Applications -> Sound & Video -> Movie Player and select the "BBC" entry from the drop-down box labelled "Playlist". If you find bugs, please report them!

Miriam Ruiz: We finally have love in Debian!!

L ve bannerL ve is a 2D game development framework based in Lua and OpenGL, created to be a user-friendly engine in which simple (or complicated) games could be made without having extensive knowledge of system or graphics functions and without having to dedicate time towards developing the same engine features time and time again.After some weeks in the NEW queue, we finally have L ve in Debian.In this time frame, L ve 0.5.0 has been released. I’ll try to have the latest version packaged and uploaded as soon as possible. In any case, it’s so cool to finally have love in Debian! You can now apt-get install love in your system ;) Lior Kaplan spoiled the surprise some weeks ago, btw :P

26 October 2008

Christian Perrier: I'm already hearing the laughs...

...of my friends, workmates and colleagues, when I will have to try to explain them that, no, I have no bloody idea of when Debian will release. Mostly because: The project is currently heavily frozen. Our bug report rate during last month was 3000bugs, to be compared to the 5000 bugs/month ratio of the normal development process. So, yes, I'm already hearing the laughs about the trust that can be put in Debian. At some point, we should remember that even our technical excellence will not prevent us from people looking elsewhere for something serious to work with. Again, prove me wrong. Release the damn installer. Stop this nonsense about the so-called freeness of whatever bit in Debian. Postpone the discusion about contributors/maintainers/developers. Release the thing. By the way, thanks to Joss for the cookies contest. From what I've seen during the week-end, it was very successful. And, un surprisingly, those working on it were those not wasting their time in silly games. Ah, and I fixed an RC bug this week-end, too...

25 October 2008

Ben Armstrong: Wifi roaming on the move

Preface: legal stuff Depending on where you live, use of random open wifi networks might be considered illegal. Although I know of no law in my country against doing this, where you live, things may be different, so educate yourself before following my example. Wifi roaming on the move For the past year I have used my Eee PC on the metro transit bus, as it is the perfect size for this. When I can, I take advantage of open networks to connect back home. In this article I will show how you can actually use such access, will recommend which software works best and will explain how to set it up. So, what can I really do with the net while in motion? How about irc? No kidding! This shell one-liner does the job: 
while ! ssh -t 10.9.8.7 'screen -UDr' 2>/dev/null ; do echo -n "." ; sleep .1 ; done
We’re concerned about privacy and security on other people’s networks, so we use ssh. Inside that screen session, keep irssi running for a continuous irc presence. Every 10th of a second, the script tries to resume your session. Once a the net passes by, typing “[Enter]~.” drops you back out to retry again. The row of dots serves as a progress bar. If you want to get fancy, you could replace that with an ASCII ‘spinner’. I leave that as an exercise for the reader. :) Come on, is that really usable? Certainly. As the good folks on #eeepc at irc.freenode.net and #debian-eeepc at irc.oftc.net can attest, even a half dozen brief connections can be enough to provide tech support, rough out some development ideas, or just carry on some friendly chat. If your spouse or housemate is online too, it can even help you make some last-minute grocery-list changes. :) While the script is working, enjoy some offline things such as blogging, reading a cached Planet Debian feed or a PDF book. OK, so what else do I need to make it work? Well, you’ve seen so far we need on a server: On the Eee, you should also have: But what do I need wpasupplicant for if I only use open networks? Well, wpasupplicant is really handy for fast control of networks in Roaming Mode (as per /usr/share/doc/wpasupplicant/README.modes.gz). A GUI is just too slow. For example, you may need to filter out ‘junk’ open wifi nets: pay-per-use hotspots or nets that hide their name. You’ll not only want a “catch-all” stanza for any open network, but also a number of the most commonly found default network names, e.g. 
# Catch-all; associates with any open network:
network= 
     key_mgmt=NONE
 
# Commonly found default network names:
network= 
     ssid="default" 
     key_mgmt=NONE
 
network= 
     ssid="linksys" 
     key_mgmt=NONE
 
# etc. by watching scan_results closely you'll find several more
While the connect script is running, keep a separate tab open in gnome-terminal for wpa_cli. Use ‘disable_network 0’ to drop the catch-all and let the other stanzas kick in and ‘enable_network 0’ after filtering is no longer needed. Isn’t it really slow to get connected? Without some tweaks, yes. But here are a few things I have found by trial and error that help. One issue is that dhclient takes too long to retry, so you drop the delay between retries back to 1 second in /etc/dhcp3/dhclient.conf: 
backoff-cutoff 1;
initial-interval 1;
If the signal is very weak, you’ll want the transmit power turned up to the maximum value for your hardware. For the Eee PC 4G this is 50mW, so you set this in /etc/network/interfaces for ath0, your wireless interface: 
wireless-txpower 50mW
Finally, while you’re in motion you are constantly leaving no-longer-usable access points behind. The driver needs to discard them as quickly as possible and scan for new ones. So again in /etc/network/interfaces for ath0, you bring the default scanvalid 60 seconds down to 10 with: 
pre-up /sbin/iwpriv ath0 scanvalid 10
Conclusion The netbook has changed how we use the net. Now that we can take it everywhere, we either pay through the nose for expensive cell network access or else use the patchy wifi coverage blanketing most urban areas. A few lucky cities have such things as municipal wifi nets, but the rest of us have to scrape what access we can wherever we go. You are now equipped to try this out on the bus. If you find any other tips that can help, I’d love to hear from you.

24 October 2008

Sandro Tosi: Bits from the reportbug side of Debian

Things are evolving under the hood for reportbug, and we are preparing a version targetted for squeeze.

The version 3.99.0 has just been released to experimental; the main goals for this release are:
Other goals we'd like to archive for the 4.0 version (for which 3.99.x are the pre-releases) are:
Since this is a development version, you can experience some bugs/regressions, so please try it, stress it and "reportbug reportbug" if needed :) You're help is really appreciated! (and we have cookies ;) )

Holger Levsen: Dear DSA FTPMaster Keyring-Maint Secretary FrontDesk DPL

For one, I think it might be a good idea to pause this proposal now. (Reason: lenny)

And I definitly think you should develop it using the DEP procedure. (And not a series of mails to different lists and some blog posts, which is very hard to follow.) And most probably this "you" (delegates) should become a "we" (Debian).

Love, Holger.

P.S.: my changelog form blog engine looks cool, but fails with commata in titles. Gah.

Joerg Jaspert: Developer status

Nice how some people (on purpose?) try to misunderstand everything. And love to flame. (Yes, it was my mistake to not explain the meaning of we right at the top - or in that mail at all -, sorry.) Lets address a few of the issues: There are many more things on the mailing list, but this post is already a little to long, so lets stop for now. I might blog more later on and reply to a bit more. I might also try and reply on the mailinglist, but erm, thats a HUGE thread already… What I won’t reply to are those few people who wish me to die or suffer from something painful. Sorry, I have to disappoint you, none of this is on todays agenda. Comments: 5

23 October 2008

Joey Hess: anatomy of an atrocious announcement

Here are some tricks that you can use to ensure that your announcement to debian-devel-announce has maximal negative utility. hide who's talking Make sure that your announcement does not start off by making clear who's speaking. The announcement will have a From line with your name on it, and that's good enough. Everyone knows who you are, what groups you're a member of, and can guess what group you're speaking for ... or whether you're just speaking for yourself. Throughout the annoucement, be use to use terms like "We plan", "let us describe", "we are considering", "This is where our proposal comes in.", etc. Maybe in a footnote include a clue such as a mention of "the NM-Committee", but be sure to not sign the announcement with the name of any group or team or set of people, so that the reader is left guessing about who this "we" is all the way to the end and beyond. (And hey, "cabal" is a fun word.) weasel words Scatter throughout the announcement some vaguely worded digs at things you disagree with. The key here is to make clear what you really mean, while providing enough plausible deniability that you can deny having said it later. So you might say ...
Some time ago a few Developers thus went and pushed forward the "Debian Maintainer" status.
... Thus implying that this was a few malcontents who should be ignored. Sure, the whole project voted for this, and so you should be sure to allude to that vote somewhere else. Maybe in a different paragraph entirely, you could refer to "Debian Maintainers (DM) [GR-DM]". (It would be excessive to have that footnote refer to a bogus url like http://vote.debian.org/something.) the big lie Make sure that your announcement starts off all bright and cheery on the surface, with just a hint of deadly steel jaws underneath. Something like:
If you are an existing Debian Developer or Debian Maintainer, don't be afraid, we are not going to take anything away from you.
Natter on for many, many pages of trivilities, good ideas, bad ideas, half-baked ideas, conflicting acronyms, and useless footnotes. Then spring the trap:
In future this will be a list maintained by the NM-committee. At the time of migrating from the old to the new way, Ftpmaster will convert the existing DM-Upload-Allowed fields into that list, so there should be no interruption in your ability to upload.
And ability to upload is all that matters. Who cares about the regression back to the tyranny of unix permissions? Who cares that DDs won't be able to decide when they trust a DM to upload their package, or that DMs won't be able to sign up quickly and easily, or that DMs will have to petition a disinterested committe to change anything? Nothing will have changed, nothing will be taken away, nothing to see here, move along: The big lie.

Matthew Palmer: Porting debian-installer to the DNS-323

I've just started a new job, so I decided to take a few days holiday in between and work on something that's been on my todo list for way too long now -- making the debian-installer work on the D-Link DNS-323. After a couple of days of work, I've just finished off the install of Debian on my DNS-323, which I initiated by loading a firmware image into the web interface of the stock firmware, and then SSHing into the machine at the appropriate moment. What this means is that anyone with a DNS-323 can install Debian on it. I think that's worthy of a small, quiet "woo hoo!"... Looking back now, it seems to have been a fairly simple and straightforward operation, but I can remember that it didn't look so simple when I started a week ago. So, in the interests of perhaps short-cutting someone else's adventure into the world of d-i hacking, I'm writing everything up while it's fresh in my mind (the benefits of being on holidays -- plenty of time to write long blog posts). Alternately, if anyone has a random device they'd like to see d-i on but doesn't have the inclination to do it themselves, feel free to send me one and I'll do it for you. Before I begin, I've got a few "shout outs" (I think that's what the cool kids are calling it these days) to people who've helped me along the way. First off, Martin Michlmayr has answered innumerable dumb questions from me, and I think wrote most of the parts of d-i that I've extended -- it's fair to say that I'd have had no hope of doing this without him. Thanks also to the people in #debian-boot and #debian-arm for answering questions and putting up with my frequent bouts of self-congratulation whenever I passed the most miniscule milestone. Finally, everyone who has put work into d-i has my thanks, too -- it's not perfect, but it's a damn sight better (both to use and also to hack on) than any other installer I've ever met. So, without further ado, off we go... Work out how to get a console Unless you are a guru of unbelievable skill, you will undoubtedly not do everything quite perfectly the first time, and you'll want to see the kernel/init boot messages, and maybe in some cases try some investigation and diagnostic work. For that, you're going to have to have some sort of console access. If the device doesn't have a full screen/keyboard thing going on, the chances are you'll almost certainly end up with a serial console. A few devices have a network management thingo that will give you a console, usually over VNC or some sort of horrid Java applet, but I've only ever encountered that in rackmount servers, which have a keyboard/video interface anyway... In theory, I suppose, the kernel could provide a network console, but I've never heard of such a thing being attempted, and Google is not helping me out. I might add that to the todo list as a future hacking project... On the DNS-323, it was a serial console that was needed. Like a lot of the embedded devices I've seen, it didn't quite talk RS-232, so I needed an adapter (I got a TTL-232R-3V3 USB-to-3.3v-serial adapter from a local electronics supply place, which worked perfectly with Linux as soon as I plugged it in). I needed to solder a few wires to a connecter inside, and connect that up to the USB doohickey, but it all came good in the end. The chances are someone's done a similar job for your device of choice, so a search for things like "devicename serial console" will almost certainly pay off. Work out how to load and boot a kernel/initrd Your development work will involve booting weird kernels and initial ramdisks to make everything work; you need some way of stuffing those into the device and booting from them. Everything has a boot loader of one sort or another; working out what you've got and how to make it sing and dance is the ultimate quest here. How this is done will vary greatly from device to device; the WRT-54GL, for example, waits for a tftp connection for a couple of seconds on startup, which you can use to send a new firmware; many other devices have a boot loader that you have to explicitly control (via the console), but it will make a tftp connection out to another machine if you ask it to. Some devices you just need to load the data in via serial cable (at 115200bps it's bearable; at the 9600bps I've heard some devices work at, it would be tres painful). The DNS-323 runs uBoot as the boot loader, which will do a tftp request by default. However, D-link, or Marvell, or someone in the decision-making chain for this thing decided to knobble it and removed most of the useful commands, so sending the thing via serial port is the only option. Since writing to flash takes a pile of extra time, I found (eventually) that you can just load the data into RAM somewhere and boot from there, without needing to write it to flash (also handy to avoid wasting precious write cycles), since you can tell uBoot to boot a kernel located anywhere in the address space. Obviously you need to reload the kernel/initrd on every boot if you don't write it to flash, so if you've got a "stable" kernel or initrd that you think you'll be using for a while (when you're testing the other item) you should flash it then. One note on uBoot, that took me far too long to work out -- you can't just stuff a kernel fresh off the Makefile in and assume it'll work -- uBoot needs a special header prepended that tells it all about what you're loading (architecture, file type, compression method, etc). You "wrap" a file in this uBoot header with the "mkimage" command (in the uboot-mkimage package in Debian). Incidentally, if you need to remove the uBoot bits from a file, just strip the first 64 bytes off -- that's all the uBoot header is, so something like this will do nicely: 
 dd if=file.uboot of=file bs=64 skip=1
A final uBoot warning: it's fairly important to specify a "load address" for the files; if you get them wrong, your kernel and initrd may or may not boot (I never quite got the hang of why some wrong values worked and some didn't). If your device uses uBoot, and you have the ability to extract the uBoot-enabled kernel and initrd from a vendor-provided firmware image, you can use mkimage -l to inspect what all the values need to be for your own kernels and initrds. Just stick with them. Does Debian support your hardware? Obviously, if you want to run Debian on something, Debian has to support that hardware. The easiest way to test this out is to try an install. For this, you'll need to retrieve an appropriate kernel and initrd pair for the installer. Some arches, like i386 and amd64, only have one kernel and initrd for the installer, so you just have to choose if you're going to netboot, or do a full CD install, or whatever. Other arches, though, like arm, have subtly different images for different machines. There are two reasons they vary: firstly, they often need to split into subarches, where they all share a general CPU architecture but there are enough differences in the hardware that they're not all supportable by the same kernel. At a more fine-grained level some different machines of the same sub-architecture need differently formatted firmware images to boot. On ARM, in particular, it's pretty common to need to prepend a "device ID" to the kernel image so that the kernel knows what sort of machine it's booting on, which means that every machine typically needs it's own kernel image, regardless of any other issues with firmware formats. Choosing the sub-architecture should be relatively straightforward, if you know much about the machine you're targetting. Picking a machine type can be a bit harder, but again, if you know what your boot loader needs, it shouldn't be too hard to find a kernel/initrd from the existing machines in your subarch that you can mangle to fit. Load up your kernel/initrd into the device, boot it, and see how far it gets. You might get lucky, and the kernel inits, your hard drives are detected, your NIC is initialised, and you're off and running without further ado. Or you might not be so lucky. The worst case is that the kernel doesn't even get as far as loading the initrd -- it might not print anything at all, or the kernel might hang part way through the boot. This is likely to require a fair amount of debugging fun, as initial boot failures get a bit tricky. A better situation is where the kernel boots OK, and d-i starts, but something in the install process falls over. This can either be a kernel/driver problem, or a d-i problem. Either way, since d-i has started, you're likely to be able to get a shell and do some debugging. In order to make d-i work, you need to have somewhere to install to (disks, usually) and a source of packages to install from (CD/DVD if you're on a suitably large device, or the network otherwise). These are the two key ingredients you need; most everything else is stuff that ranges between "nice to have" and "critical" for operational purposes, but which has absolutely zero impact on your ability to do the install, so we'll ignore it for now. For the DNS-323, I had a few kernel problems (which I'd been warned about by Martin before I started) -- the NIC wasn't initialised with it's MAC address, which meant it was running around my Ethernet screaming "I am 00:00:00:00:00:00! Ph33r m3!", and the SATA controller just didn't seem to exist at all. These were kinda critical to doing an install, so it was time to get hacking to make those work. There were also a couple of parts of d-i that needed tweaking -- the preseeding code didn't support the device, and I needed to package a tool to build a "complete" uploadable firmware image and teach d-i how to use it. Apart from that, though, things were looking pretty good. Make the Debian kernel support your hardware Once you've worked out what hardware you need to make work, you need to dig into the kernel and fix it. Sometimes, if you're really lucky, a kernel version newer than the one currently in Debian has better support for something, and you can backport the fix / feature improvement. Other times, there's something similar already in the kernel for a different device and you can crib off that (I did this extensively for my little fixes). Then there's the "ask an expert" school, where you find someone who knows what they're on about and pick their brain (I did a bit of that, too). Finally, there's plain old "research and development", for which there are piles of documentation already out there. My changes for the DNS-323 came to 121 added lines and 16 deleted lines, most of which was cribbed-and-tweaked from other parts of the kernel. I might have written 10-20 lines of code "from scratch". It's not always scary-hard. Cross-compiling your own kernel One thing that I think needs mentioning in the kernel hacking department is cross-compiling a modified kernel for testing purposes. Chances are your main workstation has a lot more grunt (and build tools) than whatever you're trying to get Debian onto, so cross-compilation is the way to go. You need to first grab a copy of the source you want to build (from apt-get source linux-image-blahdiblah, or a release tarball, or Linus' git repo, it doesn't really matter) and a cross-compilation toolchain. For my cross-compilation toolchain, I just added this to my sources.list: 
 deb http://www.emdebian.org/debian/ unstable main
then I ran apt-get update, and installed these packages: 
	binutils-arm-linux-gnueabi gcc-4.1-arm-linux-gnueabi
	gcc-4.3-arm-linux-gnueabi-base libc6-dev-armel-cross
Kudos to the emdebian team for making that such an utterly painless exercise, by the way -- the days of needing to faff round with gcc sources to get a cross-compilation toolchain are apparently over. Once you've got a source tree and a cross-compiler available, you can run any of the usual kernel make commands (like make zImage or make oldconfig), as long as you've got a couple of make variables defined. In my case, I used: 
 ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-
ARCH tells the kernel what architecture to build for (useful in all make commands because not all arches have the same config options in make menuconfig, for example) and CROSS_COMPILE tells the build system what to prefix the calls to gcc, ld and such with (if you installed the same cross-compiler packages as I did, you'll find it installs files like /usr/bin/arm-linux-gnueabi-gcc, so you can ls /usr/bin/*gcc to find out what you'll need for the CROSS_COMPILE setting). Hack d-i to make things work In addition to tweaking the kernel, it's quite possible that the internals of d-i might need some tuning. It's possible that d-i might need to be "taught" to recognise the hardware in your device, or you'll need to extend some aspect of the installation process to complete the installation. In order to hack d-i, you need to know a bit about how it works and how it is structured, so let's run through that first. The entireity of d-i is based around udebs, which are packages of the same format as "real" debs, but which have been built differently (against a small libc, for instance) and have had everything that isn't absolutely necessary (docs, non-essential scripts, extra functionality, etc) stripped out of them to create the smallest possible packages. There are a variety of different udebs -- most are derived from otherwise normal, law-abiding citizens of the archive, like busybox (basic unix utilities like cp and ls), the shell that's used in the installer, parted, various libraries, and so on. Others are "special", and have no big brother in the archive. These are mostly "task" udebs, that define what actually happens in the installer. When they're "installed", they have special fields in the package metadata that define where in the install sequence they go, and they end up on the menu you see if your installation goes a bit pear-shaped. The tricky thing about udebs is that, as small as they are, there's too many of them to be able to have them all available when the system first boots, so even the "setup" of the installation gets done in several stages. First, the kernel and initrd boot. The initrd contains just enough stuff to get access to the rest of the udebs, and nothing else. Even this early stage is built from udebs -- there's lists of what goes into the initrd for various architectures and flavours of installer under installer/build/pkg-list in the d-i SVN. If you need an extra udeb in your initrd, that's the place to specify it. Once the initrd has gotten access to a source of packages (off a CD, over the network, telepathy, whatever) it can drag the rest of the packages it needs from there. Space is still an issue at this point, though -- there's no diskspace yet to put anything onto, so everything needs to fit into a ramdisk. Don't go loading anything you don't need. The packages loaded at this stage are what defines most of the steps that will be done for the rest of the install, so once all those extra udebs are loaded, the remainder of the install kicks off and does it's thing. Different architectures and install methods have different requirements, too, which complicates matters. For example, when you're installing on a regular PC with screen/keyboard, you can guide the install using that. However, if you've got no screen, how can you control the install? There's serial consoles, but requiring everyone to void the warranty (more) on their little embedded doodad by installing a serial console cable to install Debian isn't so hot. So, lacking a terminal, how do you control the install? Well, on install flavours for devices that might be in this situation, you can SSH into the installer and complete it that way. That feature is, like everything else, provided in a udeb -- called network-console, as it happens. But what IP address do you connect to, and what network settings does the device use? Again, without a console you can't configure your network, and not everyone has a DHCP server (and even fewer people have a DHCP server that will hand out an address that they'll know in advance, or be able to find out). So, there's a udeb (called oldsys-preseed) that does it's best to configure your network based on the settings already in the nvram for your device. In short, there's a lot of tricky, well-thought-out shit going on inside d-i, and you might have to deal with a little bit of it. The nice thing is that by now I expect most of the really hard problems have been solved, so with a bit of luck you'll only ever have to do minor tweaking. In general, all of the packages that are "d-i specific" live in packages in the d-i SVN. If you need to modify a udeb, you need to rebuild it and put it in installer/build/localudebs/ (creating it first if need be) and every time you change it, you need to delete installer/build/apt.udeb/cache/. To give a concrete example (from the ever-present DNS-323), I had to modify oldsys-preseed so that it would get the details from the nvram that the stock firmware uses to configure itself. This, in turn, required that the minix-modules udeb be included in the initrd, because the DNS-323's nvram is actually a small minix partition stored in flash. Painfully, this required some more hacking, because the armel d-i kernel wasn't configured to generate minix-modules. That might need some explanation... Since building the kernel is a fairly massive undertaking, CPU-wise, instead of building all the udebs from scratch there's a package called kernel-wedge to split a real linux-image package into lots of little udebs. There's a separate source package for each architecture, linux-kernel-di-$ARCH-2.6, that contains the configuration data for kernel-wedge to specify what udebs need to be built, and what goes into each of them. A small tweak to the armel package and voila! I had a minix-modules udeb to include in the initrd. With the config stuff I'd done in the d-i build itself and my changes to oldsys-preseed, we were done with d-i. Tell d-i how to build your bootable artifacts If your device needs a "different" install image from other similar devices, you need to write some makefile rules to build those files. Look under the installer/build/config/$ARCH/ directory to see the various rules files available. I had to package a new tool to create uploadable firmware images for the DNS-323, along with adding a rule to use that tool to build images. None of it was particularly difficult. Polish the diamond Getting d-i working is only part of the battle; it's quite likely that there will be other hardware in the machine that needs to be supported. This could involve kernel-level stuff to interface to the hardware, or userland tools to manage the hardware. There's several bits in the DNS-323 that still need work; I can't control the LEDs on the front panel, nor the fan, and I can't use the on-board temperature sensors yet. But, I do have Debian running on the thing, which is what I was shooting for. Examples If you're interested in exactly what I did, there are a number of bugs in the BTS that contain the patches I made:

Next.

Previous.