Search Results: "Herbert Xu"

17 February 2011

Raphaël Hertzog: People behind Debian: Maximilian Attems, member of the kernel team

Maximilian, along with the other members of the Debian kernel team, has the overwhelming job of maintaining the Linux kernel in Debian. It s one of the largest package and certainly one where dealing with bug reports is really difficult as most of them are hardware-specific, and thus difficult to reproduce. He s very enthusiastic and energetic, and does not fear criticizing when something doesn t please him. You ll see. My questions are in bold, the rest is by Maximilian. Who are you? My name is Maximilian Attems. I am a theoretical physicist in my last year of PhD at the Technical University of Vienna. My main research area is the early phase of a Quark-Gluon Plasma as produced in heavy ion collisions at the LHC at CERN. I am developing simulations that take weeks on the Vienna Scientific Cluster (in the TOP 500 list). The rest of the lab is much less fancy and boils down to straight intel boxes without any binary blobs or external drivers (although lately we add radeon graphics for decent free 3D). Mathematica and Maple are the rare exceptions to the many dev tools of Debian (LaTeX, editors, git, IDE s, Open MPI, ..) found at the institute, as those are unfortunately yet unmatched in Free Software for symbolic computations. The lab mostly runs a combination of Debian stable (testing starting from freeze) for desktops and oldstable/stable for servers. Debian is in use for more than 10 years. So people in the institute know some ups and downs of the project. Newcomers like my room neighbors are always surprised how functional a free Debian Desktop is. :) What s your biggest achievement within Debian? Building lots and lots of kernels together with an growing uptake of the officially released linux images. I joined the Debian kernel team shortly after Herbert Xu departed. I had been upstream Maintainer of the linux-2.6 janitor project for almost a year brewing hundreds of small cleanups with quilt in a tree named kjt for early linux-2.6. In Debian we had lots of fun in sorting out the troubles that the long 2.5 freeze had imposed: Meaning we were sitting on a huge diverging monolithic semi-good patchset. It was great fun to prepare 2.6.8 for Sarge with a huge team enthusiastic in shipping something real close to mainline (You have to imagine that back then you had no stable or longterm release nor any useful free tools like git. This involved passing patches around, hand editing them and seeing what the result does.) From the Sarge install reports a common pattern emerged that the current Debian early userspace was causing lots of boot failures. This motivated me to develop an alternative using the new upstream initramfs features. So I got involved in early userspace. Thanks to large and active development team initramfs-tools got a nice ecosystem. It still tries to be as generic and flexible as possible and thus gains many nice features. Also H. Peter Anvin (hpa) gave me the official co-maintenance of klibc. klibc saw uptake and good patches from Google in the last 2 years. I am proud that the early userspace is working out fairly well these days, meaning you can shuffle discs around and see your box boot. Later on we focused on 2.6.18 for Etch, which turned out to a be good release and picked up by several other distributions. Only very much later we would see such a sync again. With 2.6.26 for Lenny we got somehow unlucky as we just missed the new longterm release by one release. We also pushed for another update very late (during freeze) in the release cycle, which turned out to semi-work as too much things depend on linux-2.6. For Squeeze 2.6.32 got picked thanks to discussions at Portland Linux Plumbers and it turned out to be a good release picked up by many distributions and external patchsets. The long-term support is going very well. Greg KH is doing a great job in collecting various needed fixes for it. Somehow we had hoped that the Squeeze freeze would start sooner and that the freeze duration would be shorter, since we were ready for a release starting from the actual freeze on. The only real big bastard on the cool 2.6.32 sync is Red Hat. Red Hat Enterprise 6.0 is shipping the linux-2.6 2.6.32 in obfuscated form. They released their linux-2.6 as one big tarball clashing with the spirit of the GPL. One can only mildly guess from the changelog which patches get applied. This is in sharp contrast to any previous Red Hat release and has not yet generated the sharp and snide comments in press it deserves. Red Hat should really step back and not make such stupid management moves. Next to them even the semi-maintained Oracle Unbreakable 2.6.32 branch looks better: It is git fetchable. What are your plans and those of the kernel team for Debian Wheezy? Since 2.6.32 many of the used patches landed upstream or are on the way (speakup, Kbuild Debian specific targets, ..). The proper vfs based unionfs is something we d be looking forward. We haven t yet picked the next upstream release we will base Wheezy on, so currently we can happily jump to the most recent ones. There are plans for better interaction with Debian Installer thanks to generating our udebs properly in linux-2.6 source itself. Also we are looking forward to using git as tool of maintenance. We d hope that this will also allow for even better cross distribution collaboration. Concerning early userspace I plan to release an initramfs-tools with more generic userspace for the default case and finally also a klibc only for embedded or tuning cases. What do you like most in Debian? For one thing I do like the 2 year release cycle. It is not too long to have completely outdated software and on the other hand it gives enough time to really see huge progress from release to release. Also at my institute the software is is recent enough without too much admin overhead. For servers the three years support are a bit short, but on the manageable side. I do enjoy a lot the testing distribution. For my personal use it is very stable and thus I mainly run testing on my desktop and work boxes. (Occasionally mixing in things from sid for unbreaking transition or newer security fixes). Debian is independent and not a commercial entity. I think this is its main force and even more important these days. I enjoy using the Debian platform a lot at work thus in return this motivates me to contribute to Debian itself. I also like the fact that we strive for technical correctness. Is there some recurrent problem that hinders the progress of Debian? The New Maintainer process is a strange way to discourage people to contribute to Debian. It is particularly bureaucratic and a huge waste of time both for the applicant and his manager. It should be completely thrown overboard. One needs a more scalable approach for trust and credibility that also enhances the technical knowledge for coding and packaging of the applicant. NM is currently set in stone as any outside critics is automatically rejected. Young and energetic people are crucial for Debian and the long-term viability of the project, this is the reason why I d consider the New Maintainer process as Debian s biggest problem. Note from Rapha l Hertzog: I must say I do not share this point of view on the New Maintainer process, I have witnessed lots of improvements lately thanks to the addition of the Debian Maintainer status, and to the fact that a good history of contribution can easily subsume the annoying Tasks & Skills questionnaire. Another thing I miss is professional graphics input both for the desktop theme and the website. I know that effort has been done there lately and it is good to see movement there, but the end result is still lacking. Another trouble of Debian is its marketing capabilities. It should learn to better sell itself. It is the distribution users want to run and use not the rebranded copies of itself with lock-in sugar . Debian is about choice and it offers plenty of it: it is a great default Desktop. Linus Torvalds doesn t find Debian (and/or Ubuntu) a good platform to hack on the kernel. Do you know why and what can we do about this? The Fedora linux-2.6 receives contributions from several Red Hat employed upstream sub-Maintainers. Thus it typically carries huge patches which are not yet upstream. As a consequence eventual userland troubles get revealed quite quickly and are often seen there first. The cutting edge nature of Fedora rawhide is appealing for many developers. The usual Debian package division of library development files and the library itself is traditionally an entry barrier for dev on Debian. Debian got pretty easily usable these days, although we could and should again improve a lot more in this sector. Personally I think that Linus hasn t tried Debian for years. I have the feeling that the implication of the Debian Kernel team in LKML has been on the rise. Is that true and how do you explain this? Ben Hutchings is the Nr.1 contributor for 2.6.33. He also is top listed as author of patches on stable 2.6.32. Debian is not listed as organization as many send their linux-2.6 patches from their corporate or personal email address and thus it won t be attributed to Debian. There is currently no means to see how many patches get forwarded for the stable tree, but I certainly forwarded more then fifty patches. I was very happy when Greg KH personally thanked me in the 2.6.32.12 release. In the Squeeze kernel, the firmwares have been stripped and moved into separate packages in the non-free section. What should a user do to ensure his system keeps working? There is a debconf warning on linux-2.6 installation. It is quite clear that the free linux-2.6 can t depend on the firmware of the non-free archive (also there is no strict dependency there technically). On the terminal you d also see warnings by update-initramfs on the initramfs generation for drivers included in the initramfs. The debconf warning lists the filename(s) of the missing firmware(s). One can then apt-cache search for the firmware package name and install it via the non-free repository. The check runs against the current loaded modules. The match is not 100% accurate for special cases as the one where the device might be handled well by this driver without firmware, but is accurate enough to warrant the warning. The set of virtualization technologies that the official Debian kernel supports seems to change regularly. Which of the currently available options would you recommend to users who want to build on something that will last? KVM has been a smooth ride from day zero. It almost got included instantly upstream. The uptake it has is great as it sees both dev from Intel and AMD. Together with libvirt it s management is easy. Also the performance of virtio is very good. The linux containers are the thing we are looking forward for enhanced chroots in the Wheezy schedule. They are also manageable by libvirt. Xen being the bad outside boy has an incredible shrinking patchset, thus is fair to expect to see it for Wheezy and beyond. For many it may come a bit late, but for old hardware without relevant CPU it is there. Many tend to overstate the importance of the virtualization tech. I d be much more looking forward to the better Desktop support in newer linux-2.6. The Desktop is important for linux and something that is in heavy use. The much better graphics support of the radeon and nouveau drivers: The performance optimizations thanks to dcache scalability work and the neat automatic task-grouping for the CPU scheduler are very promising features for the usability of the linux desktop. Another nice to have feature is the online defrag of ext4 and its faster mkfs. Even cooler would be better scalability in ext4 (This side seems to have seen not enough effort lately). Is there someone in Debian that you admire for their contributions? Hans Peter Anvin and Ted Tso are a huge source of deep linux-2.6 knowledge and personal wisdom. I do enjoy all sorts of interactions with them. Christoph Hellwig with Matthew Wilcox and also William Irwin for setting up the Debian kernel Team. Several Debian leaders including the previous and the current one for their engagement, which very often happens behind the scene. The Debian Gnome Team work is great, also the interactions have always been always easy and a pleasure. Martin Michlmayr and previously Thiemo Seufer do an incredible job in porting Debian on funny and interesting ARM and MIPS boxes. Debian has a lot of upcoming potential on this area. I m looking forward to other young enthusiastic people in that area. Colin Watson is bridging Debian and Ubuntu, which is an immense task. Michael Prokop bases on Debian an excellent recovery boot CD: http://www.grml.org. I d be happy if any Debian Developer would work as carefully coding and working.
Thank you to Maximilian for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

One comment Liked this article? Click here. My blog is Flattr-enabled.

7 April 2009

Manoj Srivastava: Manoj: Not your father's kernel-package

A few hours ago, a new version of kernel-package was uploaded to experimental. This is a major change,and I would appreciate it if folks took it out for a spin, kicked the tires, and provide feedback about where this version is lacking. This is only part of the way along in this development cycle. I would like to add a debug-info separation, either in a different directory than / in the image packages, or a separate package by itself. I would also like to create an overlay directory for /usr/share/kernel-package/, so people can inject code or override the defaults for kernel-package easily. I am also willing to make any changes to standardize the handling of hook scripts for kernel packages in Debian.
Table of Contents

./debian/ is ephemeral
make-kpkg removes and re-creates ./debian on every invocation. This started as an exercise to protect ourselves from the upstream builddep functionality, that randomly cleaned out ./debian whether or not it had created it, effectively making it impossible to run dpkg-buildpackage easily (which is ok, if all you care about is the image package) This does make the kernel-package far more nimble; we now offer less surprise to users who did not expect stampts that the kernel-packagge used to not do duplicate work. Now, if you edit a couple of files in the kernel source, and run make-kpkg, the kernel will build as expected. There are no more version mismatch errors, and the kernel version can be modified using localconfig as one desires. With this, kernel-package can routinely be used to build kernels out of the git tree. The con is that we no longer cater to official kernels, or to anyone who expected content in ./debian to persist. At some point, there are plans to implement an overlay directory that will shadow /usr/share/kernel-package/ruleset, but that is not yet implemented. In any case, the kernel team in Debian regards kernel-package to be broken, and have been bad mouthing it and deprecating it for a few years now, so this will not be a loss for them.

Get rid of the facility to patch kernel sources
The patch the kernel facility was adding complexity, and failing to provide the flexibility required for a generic patching facility. It used to be useful at one point, but in the modern parlance, witht he widespread use of distribute version control systems, and various facilities to manage source and patch them, the built in version was clunky. This means the --added-patches option of make-kpkg is gone, the work-around is to prepare the kernel sources before calling make-kpkg.

Remove special case code for official kernels
For the longest tine (well, ever since Herbert Xu too over building kernel images from me), kernel-package has carried specal case code for official images. This has caused some problems, recently, since the need to preserve ./debian has caused no end of problems when the version changed out from under ./debian, or when people wanted to edit a file and expected kernel-package to do a minimal recompile. However, sometime in the Etch release cycle, the kernel team deprecated kernel-package as the means of building official kernels. They have recently started saying they think kernel-package is broken, and have their own recommendation for how to build kernel packages. Therefore, a full release cycle later, we can get rid of the special case rules used for official packages. Also, this allows us to drop ./debian at the drop of a hat, and recreate it with an version that reflects the current state of the kernel sources.

Header package no longer create symbolic links in /usr/src
Instead, ship an example shell script that replicates the old behaviour. This script can then be deployed on the target machines, and could be a part of a locally created kernel configuration package, if one needs to deploy the same behavior across a cluster of machines.

The postinst no longer manipulates symlinks
This is a shift from previous behaviour. Any symbolic link manipulation must now be done with hook scripts in /etc/kernel/*.d directories. Firstly, modern boot loaders scan the boot directory for kernel images, and the user no longer has to code in the path to the symbolic links that the kernel image package used to manipulate. Secondly, hardcoding the behaviour into the postinst made for a very rigid policy; and user wanted more flexibility than that. There is an example shipped with the package that shows a more flexible scheme that kept two symbolic links for version 2.4 kernels, and two symbolic links for 2.6 kernels; it can be easily modified to keep two links for 2.9 kernels and two links for 2.8 kernels, or one of each, or whatever the user wants.

Image postinst no longer runs a boot loader
Please note that this was already the case for grub, one of the more popular boot loaders. Now that we have a mechanism for running arbitrary scripts when the image packages are manipulated, we can stop embedding the boot loader actions in the package itself. This means that lilo, elilo, etc will no longer be run directly by the post install script, and all the code related to detecting the boot loader, managing the configuration, and adding bits about bootloader documentation is all removed from the postinst. This allows the image package to be more flexible, since the end user is no longer restricted to the actions encoded in the image package. This is a fairly large change. It also opens the door for the user to easily use non-standard bootloaders, if they so desire.

The image postinst no longer creates an initramfs
Instead, there are example scripts provided that will perform the task. These scripts will work for official kernel images as well. The initramfs scripts provided work with the make-kpkg images as well as the official images, and are thus better than the script shipped with initramfs-tools themselves, as they offer a super set of functionality. This also demonstrates how the posts install script communicates with the initramfs creation scripts so that no initramfs is generated in case you do not want it.

17 April 2008

Adrian von Bidder: State of the Nation, sort of thingy

Readin Sam's announcement, I thought “finally!”. I am now, however, deeply disturbed by Joerg's reaction to his delegation. This way, I have a very strange impression on what's happening. Didn't Joerg know that Sam was going to delegate him? Of course, this is not legally relevant, but I do not feel that this is how it should happen. (Mark the difference: this is not about somebody being forced into a team that doesn't necessarily welcome the new member, but about the proposed new member being forced into a decision in this way.) On balance: please do it. The situation reminds me of the Debian kernel team when it changed from Herbert Xu to group maintainership, which now works quite nicely afaict. But obviously, there is no guarantee that things will work out nicely, and there might be serious friction as well. OTOH the new DPL publicly stating that he supports this decision should be a Good Thing™

24 January 2007

Wouter Verhelst: Debian motivation

Steve asks a pretty good question: what is the motivation to work on Free Software in general, and Debian specifically? There are actual scientific studies on that subject as academics try to wrap their head around this strange thing where people do not get paid, but still put in a hell of a lot of work. What motivates me to put in hundreds of, sometimes boring, hours working on this stuff? I honestly don't know. I started doing working on Debian with a desire to make some name and fame for myself; but frankly, if name and fame is what I'm after, I should have stopped doing this a long time ago and should have tried to go work for TV or something. Or I should have been working on some stuff which is central to Debian (either the OS or the Project). So while "name and fame" might've been the reason in the beginning, it's not my motivation today. It could be that I want to help shape the Debian operating system into something useful; that I want to contribute so that the system is something I like to use. But I don't think that's the case; if that was my motivation to work on Debian, I would have more interest in doing some infrastructure work, which I'm not doing: the m68k port is not exactly the most important port of the Debian project, and my most popular package, nbd-server, is installed by 101 popcon submitters, or 0.38% of them all. Why the server seems more popular than the client is a mystery to me, but well. It could be that the Debian project is this warm and fuzzy place which allows me to meet people from the comfort of my bed room, but I don't think that is the case. If I want to meet people, I'll go to a bar, or go sing in the choir, or go play my flute in the ensemble. That's a lot more direct, a lot easier than trying to set up some trip to Scotland, and a lot cheaper, too. And if I want a warm and fuzzy place, then with all due respect, Debian is not what I'm looking for. It could be that by working on Debian, I want to challenge my knowledge and in that way increase it; this would look good on my CV, and it could perhaps satisfy a desire to learn new stuff, all the time. But that's not entirely true. I don't need to be a Debian developer to challenge myself or to learn new stuff; and if I would really want to have something that looks good on my CV, I would be much better off by joinging a project that produces insane huge gobs of code, rather than a project which is perceived to do nothing more than 'just putting stuff in packages'. I think my real motivation is a bit of a mixture of all of the above. I still feel proud when people talk about things I've done, or when people praise me for the code I've written; and I still feel sad when people mock the m68k port, even if 'only jokingly' (because we've pulled off a lot). I often try to file good bug reports when I find issues with Debian, and purposely run unstable on my laptops to help find bugs quickly, in an effort to help turn Debian into something useful. There is a lot of flaming on Debian mailinglists, but there are also a lot of friends—who, contrary to those from the choir and those in the ensemble, actually know what I'm talking about if I say "I'm trying to port Debian to those ColdFire boards, which needs changes in the compiler, the linker, and the C library. Possibly also the assembler". And finally, while Debian work hardly challenges me anymore, it does still allow me to learn new stuff every day—which is quite useful in my day job. So, there you have it. I don't have a specific reason why I'm still a Debian Developer after five years; it's more of a general feeling that being part of Debian is the right thing to do. I guess you could also say I have this feeling that I can't abandon "them"—whoever these "them" might be. Obviously that's incorrect (I've seen much more important people, such as e.g. Herbert Xu, being replaced without much of a hitch), but the feeling still is there, nagging on me. So there you go. What's your motivation?

18 February 2006

Wouter Verhelst: Herbert Xu,

You are my hero. Thanks.