The Linux kernel project has seen major changes to its development and release strategy in the last few years, in particular since the first stable release of the 2.6 series in December 2003. This series was opened almost three years after the 2.4 series in January 2001, which many felt was too long. A problem that resulted from the long interval between major releases was that vendors had to back- and forward-port a lot of patches. Nowadays, major development happens on the 2.6 series and new releases are published every three to four months. This new development model has faced much controversy. While some people, in particular developers of the kernel, claim that the new model is working very well, some users are worried about the number of significant changes and lack of stability in the kernel. Andrew Morton, a lead developer, has expressed several times that he believes the kernel is getting buggier.

Version	Date	Months
1.0	1994-03-14
1.2	1995-03-07	12
2.0	1996-06-09	15
2.2	1999-01-25	31
2.4	2001-01-04	23
2.6	2003-12-17	35

Past problems

• Because of the long release cycle, many changes accumulated. It was hard to get the development stable and there were few testers.
• Features got out very slowly because of the long release cycle.
• Hardware support and crucial features had to be backported to the latest stable kernel.
• Vendors backported many features to their own releases. The code base from different vendors diverged a lot from each other and from the official development version.

Solutions

• New versions are now released every two or three months. There is a two week merge window after each release during which new features are accepted; subsequently, the focus is on stabilization.
• There is now a steady flow of code into production and many people get to test the new code.
• Features get out more quickly.
• Vendors can directly work with current releases and get their changes into official versions easily.

Outstanding problems

• There is no long-term stable version based on the 2.6 kernel. This is being addressed with Adrian Bunk's 2.6.16 long-term maintenance tree but whether it will have a big impact remains to be seen.
• Regressions between versions are introduced more frequently. Better control and tracking of regressions are needed.
• The Bugzilla bug tracker needs to be integrated better into the development process and it would be helpful to have a QA person. In January 2007, Google announced that they're looking for a kernel QA person who'd work together with Andrew Morton.

Via Russel Coker. A. Copy the list below to your own journal and Bold the actions you are already taking
Underline the actions you plan to start taking
Italicize the actions that don't apply to you B. Add one (or more) suggested action(s) of your own C. Leave a comment here, so that she can track the meme to your journal, and copy your suggested action(s) back to my master list.

1. Replace standard incandescent light bulbs with compact fluorescent light bulbs - my room has a 12 W bulb. How much watts do you need for lighting?
2. Choose energy efficient appliances
3. Wash clothes in cold(er) water
4. Turn the thermostat of your hot water tank down to 50 C (125 F)
5. Install a programmable thermostat (or turn the heat down over night and when you're out of the house) - it is often sufficient to have the heater go on at like 7 at night and 5 in the morning for an hour each
6. Register with the [Canadian Marketing Association's] Do Not Contact Service to reduce the amount of junk mail delivered to your house. - Substitute MPS.
7. Eat less meat (particularly feedlot beef) - it's a lot healthier anyway. I enjoy the occasional steak, but you actually enjoy any taste more if you have more variance
8. Walk, bike, carpool or take public transit as often as possible - I don't have a car. If I need to transport something big, I borrow one. I do my regular shopping via bicycle or public transit, which saves me looking for a parking space etc. anyway. Also note that when using public transit, you can work a bit, read a book, enjoy some music, ... and you arrive less stressed out than when driving a car.
9. Make sure you know what can be recycled in your area, and try to recycle as much household waste as possible
10. Compost using an outdoor compost bin or an indoor vermicomposter - at my parents
11. Clean or replace filters on your furnace and air conditioner
12. Buy local, organic or fair trade food where possible - tastes much better anyway
13. Reduce air travel - pity that long-distance trains suck in Australia
14. Wrap your water heater in an insulation blanket
15. Use a clothesline instead of a dryer whenever possible - don't own a clothes dryer, hanging up clothes inside near a heater works on cold days - you often can do without ironing, too, by hanging them on a clothes hanger for drying they'll often be reasonably straight. My washing mashine has a program helping at that, too. It uses 40 C and takes just about .8 kWh and is called "easy iron".
16. Plant a tree - pity my trees are dying because of the drought / climate change
17. Buy fresh foods instead of frozen - especially with meat I do really notice a better taste
18. Keep your car tuned up and your tires inflated to their optimal pressure - I don't have or need a car
19. Use biodegradable dishwashing liquid, laundry soap powder, etc. - causes less allergies
20. Drink tap water (filtered if necessary) rather than buying bottled water - our tap water here in Munich is better than bottled water anyway, and it saves you from carrying it, too.
21. Turn the tap off while brushing your teeth
22. Unplug seldom-used appliances and chargers for phones, cameras, etc., when you're not using them - consider to use a master switch for them
23. Plug air leeks and drafts around doors and windows with weatherstripping
24. Switch from disposable to reusable products: food and beverage containers, cups, plates, writing pens, razors, diapers, towels, shopping bags, etc. - especially shopping bags and beverage containers is easy
25. Consider garage sales, Freecycle, eBay, or borrowing from friends/family before buying a new tool or appliance - saves energy and money
26. Reuse bathwater, maybe to flush the loo, water the garden, etc.
27. Make sure your roof is well-insulated.
28. Always wear a jumper/sweater and socks indoors unless it's warm enough outdoors to go without both. - and turn down the heater a bit, it saves a lot of energy and money. Sweater and socks are quite comfortable...
29. Run your vehicle on biofuel/sustainable fuels
30. Set up a grey water barrel to use when clean water isn't necessary
31. Put grey water on your garden immediately without storing it.
32. Install tanks to collect rain-water from your roof for watering the garden, washing your car, etc. (at my parents; we also installed an overflow valve which will automatically redirect overflow water to plants that can always use more water)
33. Don't keep windows open when your heater is on - especially don't keep them half-open. If you want better air, turn off the heater and open the windows for a few minutes completely. This will quickly refersh the air in the room. Then close the window and turn the heater on again.
34. Turn off light that you don't need
35. Stop your engine when waiting more than 10 seconds - for example at train crossings; maybe doesn't apply to Diesel as much because it relies on a operating temperatrue, but I doubt it cools down that quickly
36. Install a solar panel on your roof for water heating - especially in summer or in sunny regions, this will allow you to completely turn off the heater during summer or even most of the year. A solar water heater will easily heat your water to 60 C (on hot days we reach 70 or 80 C), while even keeping your house a bit cooler). It can be installed yourself. At the same time, maybe add insulation to your pipes. If you minimize temperature loss over night, you'll even have warm water left in the morning.
37. Use air circulation for cooling - much cooling can be achieved by having a good air flow, wearing appropriate clothing and avoiding heavy work during the hot hours. Consider doing a siesta and work in the morning and evening
38. Ask your bank for an online-only account - some banks even offer lower fees if you don't use paper-based banking
39. Try to make more use of daylight - that's like free light, you know.
40. Walk and run more - it's healthy, saves you fitness club costs and it saves some energy if you are not using the lift. Walk to the supermarket or fitness club!
41. Use your local recreational facilities - there are probably nice parks nearby where you can go running, maybe some place to go climbing nearby, some pool, ...; this can be an interesting alternative to watching a movie, too
42. Use the refridgerator smartly - don't leave it open when not necessary, and keep the temperature only as low as you need it. Bring all your shopping into the house, sort it, and then open the refridgerator and put it all in in one go, for example.
43. Learn how to cook energy efficient - for example, if you plan ahead you can put frozen foods into the refridgerator to defrost at virtually no power cost. It's also better for the food. Use lids when boiling water; you can also use it often for cooking if you turn down the heat. For rice, I usually put on the lid, and when the water is boiling I can turn the heat off again. With the lid on, the rice will end up cooked just fine. Also pick an appropriate applicance. A real water cooker will be more power efficient for making your tea than putting a small pot on a large flame.

the other day, on ICQ:

K: “What are you doing today?” Me: “Taking some training in using a commercial spam filter appliance.” K: “It’s all Spam. Autodelete everything.”

Next thing I remember was the trainer asking why I was laughing so hysterically.

GNOME provides a complete desktop environment that is easy to use. The project experienced major problems, such as delays, during its 1.x series and in particular during the preparation of its 2.0 release. GNOME introduced time based releases after its 2.0 release and has significantly improved release management. The project has published time based releases every six months for a number of years now and is considered as the reference model for a good implementation of time based release management. They have shown that it's possible to set and meet deadlines in volunteer projects and to release on time.

Version	Date	Months
1.0	1999-03-03
1.2	2000-05-25	15
1.4	2001-04-02	10
2.0	2002-06-27	15
2.2	2003-02-06	7
2.4	2003-09-11	7
2.6	2004-03-31	7
2.8	2004-09-15	6
2.10	2005-03-09	6
2.12	2005-09-07	6
2.14	2006-03-15	6
2.16	2006-09-06	6
2.18	2007-03-14	6

Past problems

• The main goal of version 2.0 was to change internal interfaces but after more than a year of work the project felt they had to deliver more user-visible changes, therefore leading to delays.
• Developers were disappointed with delays and that their work was not available to users.
• It was not clear what was going on. Developers constantly had to ask about the release status.
• Freezes were announced and people worked towards them but then they were delayed. Freezes also often came unexpectedly.
• Vendors had deadlines but the GNOME schedule was unpredictable. Vendors did not know whether they should aim for the next version or focus on the previous version and backport fixes.
• Vendors had to backport many changes to a version that the GNOME project considered as old.

Solutions

• The project introduced a rigorous schedule promising a release every six months.
• There was a core team so few people had to agree to the introduction of a schedule.
• GNOME introduced policies to keep the development tree fairly stable.
• The project introduced the idea of reverting: if a feature was not ready on a certain cut-off date, it would be taken out again.
• The project gained credibility because releases were actually performed on time.
• The schedule allowed vendors better planning, and hence vendors could implement their features on the main development tree.

Outstanding problems

• The six month schedule has been successful in the delivery of incremental updates. There are some concerns whether this release cycle makes the project less innovative and ambitious regarding major changes that would lead to GNOME 3.0.

Theses days, I've been looking for a way to have crypto-containers in Debian. I wanted something easy to use, easy to move but still quite secure, which wouldn't require root access to the box, and as an option could use my gpg key.

I looked closely at two projects:

* cryptsetup/luks

The nice thing is that you only have one file containing all your crypted filesystem, it uses the Linux unified key management system (luks), and you are able to use a gpg key to crypt the data. I didn't tried it yet, because there are some drawbacks which I find really important. It uses multiple commands for every operations (creating the container, mounting and unmounting it).

For creating the crypto container you need to:
- create the file
- use it as a loop device (root access)
- initialize the container
- open the container
- create a filesystem on it
- mount it

Each time you want to mount it, you need to:
- create the loop device from the file
- open the container
- mount it

It surely can be wrapped, but that may be difficult to deploy. And you need root access for all operations (you can mount/umount without it using fstab, pmount or things like that, but still it's using privilegied access). And the major drawback, I think, is that the container size is fixed. You choose it when you create the initial file which you'll mount as a loop device. I don't know if there's a way to extend it after creating crypto-container, but anyway it'll need to be manual.

* encfs

encfs is fuse-based, so it doesn't require anything, beside fuse support in the kernel and your user belonging to fuse group. No need to root access nor anything. You create the container with only one command, then mount or umount it with only one command too. You can add and remove files from the container without asking yourself about disk usage or something, because the container is "expanded" (see after) when you add files, automatically.

There are two drawbacks in encfs, for me:
- crypto-container isn't a file, but a folder, with files and folders using crypted filenames but real size. (more or less, because you can use options to modify it but you won't hide a 20MB file for example). It's less secure than a one-file crypto-container where you can't determine how many files there are in it (and with crypsetup/luks, you won't be able to determine how much space is used in the container as the size is fixed anyway). It's less easy to move around, put on an usb key or transmit. You can tar it, yes, but you lose the comfort.
- you can't use a gpg key

In the end, I guess I'd like something like .dmg OSX Disk Images, which can be encrypted. They are really easy to use, as far as I could have seen. Maybe not so secure, but this is a bruteforce attack against master password (1024 bits RSA key), not against 128bits aes key of the container.

Anyway, I've setup an encfs folder here, but am still looking for something wich would improve the situation.

The design is now agreed with the weavers, but unfortunately I misunderstood the price-list, so the prices I've been quoting for hand-sewn 8-yard kilts, and machined lady's 4-yard kilt-skirts were too low -- the correct prices for those and a couple of other things being:

Hand-sewn, 8 yard kilt:	£420.00
Machined, 4-yard, skirt-kilt (lady's):	£237.50
Machined, 6-yard kilt:	£270.00
Waistcoat:	£113.00

I'm sorry about quoting prices without making absolutely sure they were correct, the reason was that I was quoting the prices for standard weaves from stock, and it turns out that the custom weaves are more expensive. That said, note that the machined kilts are £80.00 cheaper than the previously quoted price (for hand-sewn), and that these are perhaps more appropriate to the use to which many of us will be putting the kilts anyway (hand-sewn are really an evening dress item, after all) Even so, I'll probably still fork out for the full 8-yards, on the basis that my dinner jacket is falling to bits after 20 years wear, so I need something new anyway. In case you're wondering what on earth I'm talking about, this has been discussed elsewhere. Just to make sure there are no other nasty surprises, if you are of a more rotund nature, you may find that it will take some extra material to encompass your circumference -- as an example, a 7 yard machined gents kilt will cost an extra fiver to stitch, and an extra £25.00 for the yard of cloth, so will come in at £300.00 rather than the usual £270.00 Confused? Well, so was I --- The designer's sending me a fuller price list, which I will ensure than anyone on the list of currently interested people is fully conversant with before we put the order in, so don't panic that I'll spend your money without giving you a final chance to come to your senses ;-) I hope that doesn't result in everyone changing their minds, but I'll understand completely if it does. Alternatively, you could decide that this is an opportunity to save yourself £80.00 by opting for a machined kilt (which AIUI are still made to measure, but just have proportionately less work and material in them) Anyway I'll post any further developments here so as not to continue spamming the lists. So, if you're interested in getting hold of any of the above despite the new prices, and have not yet told me you need to mail me now! -- the order will probably go in at the end of the week.

If you are running a system with ext3/ext2 on top of a LVM partition, or just when you made the partition bigger, you obviously want the file system to grow, too.

Easy, ext2online is the answer:

ext2online /dev/mapper/hdapool-homefilesystem

Unless you run into an error like:

ext2online v1.1.19 - 2001/03/18 for EXT2FS 0.5b
ext2online: group 0, block 2 not reserved

ext2online: unable to resize /dev/mapper/hdapool-homefilesystem

That's when you start googling and find that ext2online is not that online and needs the file system to be unmounted you need to run first:

ext2prepare /dev/mapper/hdapool-homefilesystem 23G

Note that the size there is the desired new target for the filesystem.
So you log off, go to a console, umount the file system (unlucky you if is the root file system, in that case you'll need the rescue mode - does D-I have ext2prepare?), run the command.

Now fsck.ext3 the device, mount it and try again ext2online. Now it should work.

The GNU Compiler Collection (GCC) is a compiler suite which supports a number of programming languages, such as C and C++. It is a very important development tool and is the standard compiler among free software projects. GCC was quite stagnant in the middle of the 1990s until the EGCS project formed. EGCS took over development of the official GCC in October 1998 and instituted rigorous processes, such as high levels of peer review, and created a steering committee which has the power to appoint maintainers and make important decisions. In theory, the project follows a time based release with an interval of six months. In practice, the project has released only one new version every year in recent times.

Version	Date	Months
3.0	2001-06-18
3.1	2002-05-15	11
3.2	2002-08-14	3
3.3	2003-05-13	9
3.4.0	2004-04-18	11
4.0.0	2005-04-20	12
4.1.0	2006-02-28	10

Past problems

• The GCC project suffered from a closed development style in the past: few people could make code changes and the mailing list was by invitation only.
• There was a long time between releases and development snapshots, which contained bug fixes and features, were not available to the public.
• When development opened and picked up, significant code changes were often made which required a long stabilization phase.

Solutions

• The project moved to a more open development style and established a steering committee which has the power to appoint new maintainers and make important decisions.
• The development process was divided into three stages in order to coordinate code submissions and keep the development tree reasonable stable. Major changes that developers wish to make during stage 1 and stage 2 have to be proposed on the project's wiki. Proposals are reviewed by the release manager who also assign a sequence to proposed projects which says when they can be applied to the development tree.
• All patches are peer reviewed on the development mailing list and need approval before they can be applied. The steering committee appoints maintainers who can approve patches that touch particular areas. There are maintainers for the C front-end, various port maintainers, and maintainers for other areas.
• Each development stage lasts two months, theoretically yielding a release every six months. A regular release cycle ensures that it is not the end of the world if a feature does not make it into the following release.

Outstanding problems

• The release manager is busy and has not pushed the release forwards as much as would be possible.
• The branch criteria may need revision to make it easier to create a branch which leads to the next stable version. At the moment, the branch criteria is to have 100 or less regressions on the development tree. However, this number may include regressions that are also present in previous releases, which makes it fairly hard to reach this criteria.

... or why PHP sucks so much?

It is quite common (at least for me) finding abused web applications on shared hosts around. The most typical case is finding some kind of IRC bot running as www-data and a few unauthorized files around used for phishing, cross-site scripting or what else. That motivated me to try playing with mod_chroot in order to minimize the possibility of webapps abuse from our friendly kiddies^Wcrackers. Mod_chroot is a nice tiny Apache module whose purpose is the confinment of webapps within a limited tree, where nothing but a few files and dirs are available to try exploits and abuse badly written code.

Unfortunately, I found that mod_chroot poses one major problem (among others, see its CAVEATS doc): the sucking mail() function is not working out of the box, because PHP folks - God knows why - decided to implement that stuff by calling a local /usr/sbin/sendmail program within a shell call. The most sane option is simply ignoring the issue and living happy with a disabled local mail() function. A nice solution in that case is using a PEAR Mail module, which is able to send mails via SMTP in much more elegant way. Unfortunately, there are quite a good number of morons out there, and that could be not a viable option if you are the system administrator of a bounce of shared hosts which are exposed to those morons, who absolutely need a working mail() function.

After a few googling around, I did find that many people had the same problem, but none found (or explained) a working and neat solution for a working mail() implementation, so I'm writing some notes about that. My implementation is based on a nice tiny program (mini_sendmail) which has the great advantage of not requiring a configuration file or a spool directory to work. It also works without suid bit on, which is a good thing as well. It is statically compiled by default, so it simplifies things a lot. In order to have mail() working you need also to install a (statically compiled) shell, e.g. bash-static as sh under the /bin directory of your chroot tree (a more tiny shell would be appropriate). I commented out the silly username autodetection code in the mini_sendmail.c source, because it creates some problem within an empty chroot tree (it failed with a "can't determine username" message with or without the /etc/passwd file available).

Now, the next step is using an entry like

sendmail_path = /usr/sbin/sendmail -t -fwww-data@your.domain -s127.0.0.1

in your php.ini file. You can also use another IP address if your SMTP server is not your localhost or you used some more exotic routing setup. Having a suitable /etc/hosts is also useful. A final trick is adding a var/lib/php4 or what ever directory is required to store PHP session files. No other files should be required in order to have all working. Of course, you have also to use inet sockets for any required connection (e.g. mysql) but this is typically the default in etch. I would also suggest to add only ad hoc statically-compiled binaries within the chroot tree when required, else you will need to store all required shared libs: in that case you will need to find binaries requirement by using ldd and chroot to find loading/running errors by means of an usual trial-and-error cycle.

I added the edit comments plugin to this blog. After you leave a comment, you’ll have about 12 hours to come back and make changes. Authentication is based on IP address. It’s surprising that so few blogs implement such functionality, given how common it is to leave embarrassing typos or misspelled URLs in comments. One patch is needed for the plugin to work with the latest Wordpress release. Specifically: $location = add_query_arg('jal_edit_comments', '', $_SERVER['HTTP_REFERER']); needs to be changed to: $location = add_query_arg('jal_edit_comments', FALSE, $_SERVER['HTTP_REFERER']); I’m sure this will be added to the mainline source soon. I had to fish the fix out of the 300+ comments on the page for the plugin. In other “this blog” news, I simplified the sidebar appearance and made all the long lists collapsed by default with CSS. Hopefully it renders well on all browsers. Lately, I’ve come to doubt the utility of “lists in sidebars” at all. Perhaps a person’s latest content preferences (books, film, music) aren’t that interesting after all. Incidentally, since I made the transition to Wordpress, I’ve found it much easier to keep this blog active. Less time spent on administrative issues, and the web-based (rather than ssh–>vim) posting just makes a lot more sense these days. Comment spam is virtually nonexistent. I can get an entry up in five minutes on the train in to work without hassle. Wordpress has also greatly reduced server load (perhaps because I had an inefficient blosxom setup). I recommend it. If you are already a Wordpress user, as a public service announcement I’ll repeat the announcement from about two weeks ago: Wordpress 2.1.1 may be dangerous:

Long story short: If you downloaded WordPress 2.1.1 within the past 3-4 days, your files may include a security exploit that was added by a cracker, and you should upgrade all of your files to 2.1.2 immediately.

Using XMPP4R, I did some stats about Jabber clients usage on the Apinc Jabber server, which hosts im.apinc.org, jabber.fr, and many more using virtual hosting. I already did similar stats in March 2006 and September 2005. The poll was done by sending jabber:iq:version to online users, around 1:00 PM (french local time, most of the users are french). 1343 clients were pinged, and 1315 answered, which is better than last year (1145 answers). Of the 1145 jids that answered last year, 368 were also part of the poll this year (I don’t know if this is good, or not enough). Systems:

• GNU/Linux: 43% (2006: 38% ; 2005: 34%)
• Windows: 35% (2006: 37% ; 2005: 34%)
• Mac OS: 16% (2006: 18% ; 2005: 23%)
• Unknown: 4 (2006: 5% ; 2005: 6%)
• Others: 0% (12 clients ; 2006: 0% ; 2005: 1%)

Clients:

• Psi: 24% (2006: 28% ; 2005: 28%)
• gaim: 22% (2006: 25% ; 2005: 25%)
• Gajim: 12% (2006: 5% ; 2005: 3%)
• Kopete: 11% (2006: 7% ; 2005: 7%)
• iChatAgent: 9% (2006: 13% ; 2005: 18%)
• libgaim (Adium): 5% (2006: 4% ; 2005: 3%)
• Pandion: 4% (2006: 4% ; 2005: 2%)
• Miranda: 2% (2006: 2% ; 2005: 1%)
• BitlBee: 2%
• neos: 1%
• Unknown client: 0%
• Exodus: 0%
• Imendio Gossip: 0%
• Jabbin: 0%
• Spark IM Client: 0%
• Trillian: 0%
• JBother: 0%
• jabber.el: 0%
• JETI: 0%
• JAJC: 0%
• Class.Jabber.PHP: 0%
• Jabberwocky: 0%
• Tkabber: 0%
• Gush: 0%

(all clients with at least one reply are listed here) I also did some stats on the Linux distros. With 566 Linux users, one can consider that statistically signifiant.

• Client answers with the kernel version, not including distribution information: 62%
• Debian: 14%
• Ubuntu: 12%
• Gentoo: 2%
• Unknown distros (not provided by the client): 2%
• Fedora Core: 1%
• Arch Linux: 1%
• Mandriva: 1% (despite the fact that most users are french!)
• Slackware: 0%

(Other distros are below 5 replies) The Debian/Ubuntu situation is interesting. Debian is not dying, even on the desktop ! The server is hosting jabber.ubuntu-fr.org, so the results are biased towards Ubuntu. Update: it seems that Psi reports “Debian GNU/Linux (testing/unstable)” even when running on Ubuntu. This is the case for ~5% of linux users. To server admins: if you are running a large jabber server with jabberd 1.6, it’s easy to give me the right to get the list of online users, so contact me if your are interested in me doing your users stats. It would be interesting to see if users from different servers/countries have different behaviours. I’m not sure if it’s possible with ejabberd and other servers.

Erich Schubert comments on the issues relating to getting big changes into Debian. This is something that I had also noticed. I started work on SE Linux in Debian in 2001 and continued it actively until 2003 when I joined Red Hat. Less than a year after I joined Red Hat there was a Fedora release with SE Linux fully integrated and shortly after that there was a release with SE Linux on by default. The reason for this was that Red Hat management supported the idea of SE Linux and everyone had to accept it. There was no option for a package maintainer to refuse to support SE Linux.

Recently in a discussion on debian-devel one DD (who I won't name in this blog post) advocated removing SE Linux support from dpkg. I then asked him whether he had the same attitude towards non-executable stack
(Exec-Shield/PaX/OpenWall), Poly-Instantiated directories, and PIE executables. When he expressed interest in having those features I pointed out that one of the enemies of security in Debian is the fact that every person controls their little area and has no requirement to work towards common goals (apart from the most obvious ones of making the system work).

This means that instead of having a little cooperation from other developers anyone who wants to get a significant change included will have to fight hundreds of battles.

SE Linux is a classic example of this. Debian could have had SE Linux support long before Fedora, but instead it gets it long afterwards.

The same battles occur with regard to all the other security measures I mentioned (and some others I didn't). We could made Debian the most secure Linux distribution, there are many people who have the skills and the interest in doing so.

If you want features such as exec-shield, then you are missing out - largely because the people with the skill and time to work on them are too busy fighting trench-warfare rather than actively coding.

Now while I strongly object to most incarnations of the "you can't force a volunteer to do anything" meme that infects Debian I do agree that we can't force developers to write new code. We can however strongly discourate an antagonistic attitude towards new features. If someone proposes a feature
that you don't plan to use but which doesn't hurt you then there's no reason to attack - you can just ignore it. If someone sends in a patch that adds a feature which is requested by many people but you personally don't use, then if it has little or no down-side (linking against a couple of shared objects as is the case for many SE Linux enabled programs provides no measurable overhead) and the code is good it should be merged!

The real problem is that some DDs are more concerned about what is best for them personally (in the most short-term manner) than about what is best for the users.

The aim of Debian is to integrate software produced by other projects to create a complete operating system based on free software. In recent years, the project has faced increasingly delayed and unpredictable releases. Most notably, the release process of Debian 3.1 was characterized by major delays. Initially announced for December 1, 2003, the software was finally released in June 2005: a delay of one and a half years. Since then, the project has made a number of improvements to its release process.

Version	Date	Months
1.1	1996-06-17
1.2	1996-12-12	6
1.3	1997-06-02	6
2.0	1998-07-24	14
2.1	1999-03-09	7
2.2	2000-08-14	17
3.0	2002-07-19	23
3.1	2005-06-06	35

Past Problems

• Release management was not very organized and release updates were posted only infrequently. Because of this and the lack of a roadmap, freezes were often announced out of the blue.
• Due to the unorganized nature of the release, several new and unexpected blockers were found during the release process, leading to delays.
• The unexpected delays meant that software was frozen for a long time, in the case of Debian 3.1 for over a year. When this release was finally published, many components were already out of date and would often not meet user demands.
• The fact that Debian has experienced significant delays with several of its recent releases has led to problems with the image of the project. There is a perception that Debian is slow and cannot meet deadlines. This is also associated with frustration in the developer and user community.

Solutions

• The project has implemented better release management structures. In particular, Debian moved from a single release manager to a team during the 3.1 release cycle. The release is now handled by two release managers with the help of several release assistants.
• A release date for the next release was set well in advance and there is more planning.
• Release announcements are sent more frequently and various information sources have been implemented through which developers can stay informed about the release status.
• Release targets have been defined better and there is a distinction between blockers and goals. Only blockers hold up the release whereas goals can be postponed for a future release. For example, the XFree86 to X.org transition and the integration of AMD64 support were blockers, while LSB 3.1 compatibility and SE Linux support were only goals for Debian 4.0.
• The release team has shifted the responsibility of achieving targets to specific developers and teams. There are also better criteria now defining these responsibilities. For example, criteria have been published that architectures need to fulfil for inclusion in Debian 4.0 and there is a page indicating the status of each architecture.
• The release team is increasingly giving encouragement to developers to help out with software packages and bugs which normally do not fall into their domain.
• A staged freeze has been implemented according to which software is frozen in stages according to its importance. The toolchain and base system are frozen earlier than the majority of package. The majority of software is now frozen when most blockers have been resolved.
• The use of the experimental repository has been promoted to make sure that the main development repository is in a good shape most of the time.

Outstanding problems

• Developers still need to be convinced that targets can be met, that deadlines are real and that Debian can release on time.

[ Warning: Long post follows ]
Just today, I read that H01ger is happy he can wear just a T-shirt somewhere in Northern Germany. Well, from my point of view, much the opposite has happened. And I'm not just happy - I'm fucking proud!
About four years ago, Nadezhda went almost every weekend to hike to the mountains with the Grupo de los Cien people. I'll sidetrack a bit to speak about them: Grupo de los Cien is a group of mountain lovers (no relation with the enviromentalist group which later took the same name) that, since the 1950s, have built and maintained the high mountain shelters in Central Mexico. It is a group, by the way, with which Nadezhda fell completely in love - and now it's up to me to find out why ;-) But lets go back to the main topic.
Almost every week, she came back delighted and loaded of energy from a mountain hike. Almost every week, I told her I wished to go with her, of course, if they went to a simpler route. Of course, lugging around over 130Kg of humanity is no easy deal. And, after over a year and because of many problems that came together at once, she stopped going to the mountain as well. She always kept an eye on their friends, but so far had not been able to return to the mountain.

Ok, so last week she heard the group was organizing volunteers to go fix the Ayoloco 2 shelter. She signed up, and invited me. I was afraid, but accepted the challenge - Of course, full of fear.
Being this a historical event (at least in my life), I cannot but offer you some of the photos I took. And, yes, I've just started playing with Flickr. Lets see if it works for me ;-)
Saturday, 5 AM, we woke up. 5:50AM, we met part of the group near Daniel's house, in Condesa. 7 AM, we were having a nice breakfast (tamales and coffee, yum!) in Amecameca, still 28km from La Joya. From Amecameca, we went to Paso de Cort s, where we met the rest of the group, and from there we went by a little, bumpy and unpaved road until La Joya. Somewhat around 20 people started the hike at around 9 AM.


According to Google Earth, La Joya (just South of the Iztacc huatl - white woman or sleeping woman in N huatl) is at 3995m - And, of course, to start at that altitude is not easy. The air is very thin, and walking a little bit too fast quickly makes you feel the blood pumping, trying to get some more oxygen. I often remembered how my Bolivian friends behave in Potos , walking slowly even when in a hurry :)

We first went down a little valley, then up. The first portion of the hike was on soft, wet sand, with some grass (what we call zacate - Strong, hard grass, not what you commonly see in a garden). I soon learnt that was a blessing, as it was by far the easiest part. Then we went up a steep and quite rocky area - It started getting tricky for somebody not used to trusting his own feet. And it was steep, yes. This was the first time (of many, of course) I started thinking whether I should head back.

We reached what I foolishly thought was the summit, only to find a steeper, longer way to go. It started getting common to find little deposits of snow/ice (I don't know the exact term in English - it's called aguanieve in Spanish, literally snow-water. [[Update]: Thanks to the now-DD H01ger, I now know that aguanieve in English is called sleet]. I understand it's a bit harder than snow, but still much lighter than hail). We crossed a completely rocky section - No sand to hold the rocks to their place. That meant being extra careful (and thus, extra slow).

But just afterwards, in case it was not enough, Mother Nature answered my pleads for some sand - we crossed a section made exclusively of loose sand. And, please, if you have ever walked upwards on sand, imagine doing so at over 4000 meters. I was taking so much care of how and where I stepped that I strayed not more than 5 meters down of the path the group was following - Being able to climb back to the right path was really not easy. This was the point that not only I was thinking about heading back, but I'm sure Nadezhda thought I would abort the mission. She cheered me up, took some pictures of me as soon as I got out of the sand trap, and we were able to move on.

After the sand, some more rocks, and we started feeling the chilly wind. We entered a cloudy region - Humidity does get the coldness into your bones! We could not really see where was the rest of the group, so we proceeded the best way we could. Fortunately for Nadezhda and me, just behind us came the very experienced Mario Corsalini and led us. And, yes, we finally saw the Ayoloco 2 shelter.

I was by then exhausted, after some four hours of ascent. I ate one of my apples - The sweetest apple I have tasted so far. After resting a bit, and realizing I was almost frozen (we were at 2 Celsius under, with constant chilly, humid wind)... Then I started working with the guys, reinforcing the shelter, while Nadezhda and some other people painted the inside with. People started eating - of course, I joined them. And, after some time up there, we started going back down. Miguel ngel, a volunteer in the Izta-Popo park, walked a good portion of the descent with me, giving me tons of help and tips. Thank you, thank you, really. The descent is, of course, much easier - We were chatting most of the time, and we made only around two hours back. The weather was quite nice on us - We had some aguanieve falling every now and then (yes! Yes! many of you guys have heard/read me bitch on how I had never seen snow falling before - Ok, this is not exactly snow, but I think it qualifies as my first experience ;-) ), but aguanieve does not make you wet, as it bounces off the clothes. When we approached La Joya we started having some light rain. We hopped in the cars, and then the rain really began.
But even if this was not enough: Grupo de los Cien was celebrating the 83th birthday of one of its senior members, Poncho Rico. No, being 83 years old is not an excuse for skipping this memorable hike.

We went to Anatolio's house, very close to Amecameca, and had a delicious dinner, cake and drinks.
Nadezhda: Thank you. So, so, so very much. Thank you for being so patient with me, for dragging me up the Sleeping Woman. Thank you for getting me into this so very important and beautiful experience. I hope this will be the first of many. There is still a very long way for me to go before I'm really up to the challenge, but believe me: This will not be my last time on the mighty mountains.

(Update: Yes, Gunnar, the policy says that the debian/rules file must be a makefile, so if you do something else, you have a serious bug because of a policy break. Of course, probably there is nothing that should stop one from using, let's say bash, or python or perl, from a technical PoV.)

Erich has a wish list for Lenny.

What draw my attention was the very first "define and deploy a standard way of packaging software, that can be used for like 90% of packages", later he mentions rationale "other distributions (e.g. *BSD ports, Gentoo?) are ahead of us in this respect, having a standard way of packaging and building things and keeping track of changes."; these two quotes contain the real essence of Gentoo's success in the area of fast development.

A half a year ago I did an experiment in packaging a rpm, a deb and an ebuild for the same piece of software. What I observed back then made me think a lot about the entry level knowledge needed for a person to create a decent package for those types of package systems.

With the risk of being flamed by many people that will miss my point, I will tell you from the start that the deb would have been the most painful if had to start from scratch. Second would have been the rpm due to lack of good/updated documentation and the clear winner in lack of pain points was the ebuild. In reality places were switched between the deb and the rpm.

And before I start, please understand that what I want to point out is that the building system in debian is showing its age and we should do something about it, and is not about pointing people to documentation; it is about making things behave more sanely by default and not about having to read a whole day to understand why the package fails to install a file you just added then to find that you had to add the destination path to a debian/dirs file etc...


The deb had to be the most difficult. Not because I didn't do it before, or because it was hard, but because there are a lot of things that need to be done because the package management is not smart enough to do it by default (i.e. class inheritance does not exist, like is the case for the ebuilds or the macros for rpms).

The Debian approach is backwards, is reversed with regard to the others and the way it should be by default, to allow easier understanding of the packaging. You can, by default, make a package that is tweaked a lot. There is no concept (if you don't count cdbs which is boo-ed a lot by some people) or possibility to just say "make this package a package that is a standard linux app with the regular ./configure && make && make install recipe". And I don't count such a recipe the templates based on dh_make, that is way too verbose. A new maintainer wannabe has difficulties understanding why are those lines in there, and what they do, and most importantly, why is that order correct and no other. Another question which is legitimate is "is that the minimum or can that be trimmed"? (Follow debian-mentors and see how many people leave the .ex files just because they are afraid they will not know how to generate them again.)

Even the rpm has a concept of standard stuff to do for a package, there are some macros that implement that. And that is more logical for an untainted mind (as in not yet debianized) than what we have in the dh_make template, or worse, a clean and new debian/rules file. Why should one care to learn the ways and philosophy of make just to make a small package?


Let me tell you something that will feel like knife stab for many people in our community: the clear winner was the ebuild, not because I was smart or exciting, but because:

• it was dead simple to make a working one
  
• the documentation is really verbose and complete
• there is a default behaviour for every ebuild, even for a minimalist ebuild
• if your ebuild has a special need, the ebuilt can override the default behaviour by redefining the function (target, in debian/rules-speech), but you don't need to, in most cases
• there are some already defined classes which can be inherited to obtain a tweaked default behaviour (imagine typical cmake build system, typical autotools build system, typical kde build system - and iirc, this can include build dependencies)
• all the information about the package is centralized in one file which is human readable from the start, plus that is obvious, or one can understand fast what a field/function/variable means from the start, without digging through the documentation continuously
  

I did the ebuild in one day's work (it was less, but some will not believe me, so I'd say 1 day is reasonable enough) starting from absolutely 0 knowledge about the packaging system (if you don't count knowing to emerge package as knowledge).

What is done properly in the portage system is the fact that there are some default classes which your ebuild can inherit to behave more like a certain type of packages (base - a generic recipe, gnome, kde, games, sourceforge - they download tarballs directly, if needed - ...)

Why this is not possible for deb right now? Simple, we have it as a rule in the policy that the debian/rules file is a makefile. So even if one would implement a class-like model for deb packages, you'd still have the debian/rules file as a make file...

Gentoo implemented initially portage's default behaviour (and as a consequence, the "classes") in shell, later they migrated some parts to python and the whole portage is implemented in python now (while classes are implemented in bash). I am not saying that we should put python in essential, God forbid, probably the emdebian guys would stab me :-), I am saying that possibilities exist, alternatives should be possible.

I know, some will just say "show me the code", "making such a change over the entire archive is insane" or "the makefile is a really good way to do things like installing and building, after all, that's what it was designed for". Yes, all of these will miss the point, which is having nothing-to-do as a default policy for a package is a bad policy, it should be an empty/default package will use a standard recipe to build itself, and that recipe should be implemented in the build system.


And if you still don't believe me, just look at these few ebuilds and tell me you'd not be able to write them after a quarter after seeing one the first time and skimming over the docs or/and knowing what a regular gentoo user knows (what a keyword is, what is a slot). Also compare with the debian counter parts and don't forget the different building aids that we have.

http://sources.gentoo.org/viewcvs.py/gentoo-x86/games-arcade/amphetamine/amphetamine-0.8.10.ebuild?view=markup
http://svn.debian.org/wsvn/pkg-games/packages/trunk/amphetamine/debian/rules?op=file&rev=0&sc=0
http://svn.debian.org/wsvn/pkg-games/packages/trunk/amphetamine/debian/control?op=file&rev=0&sc=0
http://svn.debian.org/wsvn/pkg-games/packages/trunk/amphetamine/debian/postinst?op=file&rev=0&sc=0

http://sources.gentoo.org/viewcvs.py/gentoo-x86/games-arcade/holotz-castle/holotz-castle-1.3.8.ebuild?rev=1.3&view=markup
http://svn.debian.org/wsvn/pkg-games/packages/trunk/holotz-castle/debian/rules?op=file&rev=0&sc=0
http://svn.debian.org/wsvn/pkg-games/packages/trunk/holotz-castle/debian/control?op=file&rev=0&sc=0


http://sources.gentoo.org/viewcvs.py/gentoo-x86/games-arcade/late/late-0.1.0.ebuild?rev=1.10&view=markup
http://svn.debian.org/wsvn/pkg-games/packages/trunk/late/debian/rules?op=file&rev=0&sc=0
http://svn.debian.org/wsvn/pkg-games/packages/trunk/late/debian/postinst?op=file&rev=0&sc=0
http://svn.debian.org/wsvn/pkg-games/packages/trunk/late/debian/postrm?op=file&rev=0&sc=0


http://sources.gentoo.org/viewcvs.py/gentoo-x86/games-arcade/ppracer/ppracer-0.3.1.ebuild?rev=1.15&view=markup
http://svn.debian.org/wsvn/pkg-games/packages/trunk/ppracer/debian/rules?op=file&rev=0&sc=0


One more thing, the ebuilds mostly need not be updated when an upstream release occurs, they just need to be copied and given a different filename.

I was the maintainer of minit for some time, and spent some time adding some runit-like functionality to minit I then called "enitdir". It worked similar to runit (IIRC, maybe it was a different init). Basically you had a directory for each runlevel, which contained symlinks to all services supposed to be running. There is a symlink called "current" or something pointing to the current runlevel. enitdir monitored this symlink and the directories contents via dnotify, so it supposedly was both very fast and efficient at noticing changes there. Switching runlevels was as easy as "ln -snf current foobar"; starting and stopping services worked by removing the symlinks in the current runlevel or manually calling minit (for non-persistent changes, restarts, etc.) (I think I never uploaded a package with any of this, the minit packages remained really close to upstream. I intended to do a fork/rewrite named enit, but never got around to it. Many of my ideas are addressed by upstart.) However this didn't solve some integration issues. First of all, not all packages back then were using invoke-rc.d; this should be a lot better nowadays. But the major issues lie within handling multiple inits on one system. There is a hook that applications could use to prevent init scripts from running (which might be the answer to Wouters iniscripts post). It's called "policy-rc.d". However there are some things wrong with this approach: it doesn't really support the installation of multiple inits or handles the problems of switching between inits in any way. Basically, the moment we need to switch inits (and thus service startup/stopping behaviour!) is during reboot. While the old init is still running, any start/stop operations MUST still use the method appropriate for the current init system. Otherwise, major bugs can occur, especially with smarter init systems that respawn services. E.g. the user runs "apt-get install sysvinit mysql-server". Lets assume sysvinit has a debconf prompt 'make me the default init' and the user says yes. Next mysql-server is upgraded, during which it calls the sysvinit way of stopping the service (i.e. /etc/init.d/mysql-sever stop), then does some dangerous things to the database, then restarts the server. However, a smart init such as minit (when equipped with appropriate service files) will notice the mysql-server process dying and immediately restart it (notified by kernel, efficient and fast) while the upgrade script is still messing with the database files. Boom, there goes your database. I guess you got the point. So we probably need a smarter invoke-rc.d script. It should support

• init detection - figure out which init is running, e.g. by comparing the inode of /proc/1/exe to the inode of /sbin/init.* (current init doesn't need to be named /sbin/init, but note that this magic might break on upgrades. Fallback to readlink(/proc/1/exe) maybe?)
• different init behaviours. The current policy-rc.d hook is pretty much a yes/no hook. Instead it should call the appropriate init-specific invoke tool.

Such an invoke tool could then for example look like this:

if [ -e "/etc/enit/service/$1" ]; then
    enitctl start $1
else
    /etc/init.d/$1 start
fi

You get the idea, having the policy script figure handle compability for services that don't have an appropriate service description for this init system yet. (Yes, s/start/$2/ or something, above is pretty dumb, but helps bringing my idea over).

Here's an "interesting" phenomenon I noticed on my new (not-Linux-hating) 4GB portable music player:

daniel@jeeves:~$ df -h /media/IAUDIO
Filesystem Size Used Avail Use% Mounted on
/dev/sdb 3.8G 3.8G 0 100% /media/IAUDIO
daniel@jeeves:~$ du -hcs /media/IAUDIO
2.3G /media/IAUDIO
2.3G total

Just in case "df" and "du" had different notions of how large a gigabyte was, I asked them to tell me the number of bytes instead, with about the same result.

I won't comment on the fact that I paid for 4GB but got 3.8GB; that's typical (whoops, I just did). But where'd that extra 1.5GB go? FAT can't seriously be imposing a >50% overhead on file size, can it?

The end result here is that on my 4GB music player, I can store only a little more than 2GB of music. Boo. :-(

[UPDATE]: running fsck revealed that I had 1.5 GB in lost clusters on this filesystem and gave me my space back. Since this device is almost brand-new, I'm puzzled about how this happened...but oh well, it seems to be fixed easily enough.

Have been feeling rather unwell for the past couple of days, so I decided, while I feel like death warmed up, how could hacking at Valgrind possibly make it worse?

I took Tilman Sauerbeck's extended version of Dave Airlie's valgrind-mmt, and cleaned up a couple of minor bits -- changed the offset to be specified in hex, added support for repetitions (e.g. 'repeated 255 times', instead of the same line 256 times over), and also added support for the in*/out* family on AMD64, which was quite entertaining as I've not really touched either assembly or Valgrind before. Got it working in the end, despite libVEX's best efforts to frustrate me, and ended up feeling slightly better as well. Huzzah.

Anyway, 'sudo valgrind --tool=mmt --offset=0xc1000000[0] /usr/bin/Xorg :0 -ac > x.log 2>&1' will trap all MMIO accesses made by X or its VBIOS, provided you run the BIOS through x86emu instead of lrmi.

Postscript: spent a few minutes after writing this trying to figure out if there was anything I should've said, and didn't. Now, hours later, I realise there was one minor detail omitted: the URL. gitweb is just there, and the anonymous clone URL is git://people.freedesktop.org/~daniels/valgrind.

[0]: Get the base address with lspci -v. Your video card should have two PCI regions, of which one is big (your main video memory), and the other one is 32 or 64kB (the MMIO space, i.e. the bit you want).

We need to send out invitation letters for people needing a Visa to come to DebConf. And we really need to do this soon, as a Visa application process can take months. So it needs to be done real soon now.

So if you have some spare time (I don't) and care about people coming from countries needing a Visa to visit UK, please read this thread.

As I understand it (which might be wrong or inaccurate) the Debian UK or DebConf Ltd. people want more feedback from UK visa officials outside the UK. I'm sure there is better information how you can help in this thread, if not, please ask.

Just a few minutes ago reportbug-ng was accepted and should be available in unstable tomorrow (For the impatient: you can already get it from incoming). If you can spare some time please test it and send me suggestions, bugreports and stuff.