Search Results: "nate"

12 June 2006

Robert Pickel: some updates

What I did this week:

I wrote code that contains many "bashisms" but runs, this is the code that detects boot paramaters from the user for a few boot options. I would like to combine the boot params into on parser that sorts through all the user entered boot options instead of looping through them xxxxx times during the execution of casper-bottom scripts.

I had a rough time testing the changes I made to casper-ng, so I started to work on a tool that will be used to easily unpack, pack, and modify images created by live-package.
This did divert my attention for a while, but not too long.
  • Its called live-helper and can be found in the subversion repo.

Worked on documentation, explaining the theory of operation behind casper-ng.

  • I am currently in the process of finishing up the casper-ng documentation, you should be able to grab this in a few hours using subversion.

Cleared things up Marco, he will work on the initramfs hooks and I work on the casper-bottom scripts. I finished up removing some code that modifies the root files system at boot time and moved it to postinstall. However, I did go a little nuts with this, and was asked by Marco to keep this to a min.

  • I will work on casper-bottom scripts, he will work on the mkinitramfs hook (image detection and so on)

My plans for this week:

  • Adjusting casper-ng documentation after a peer review by the debian-live group - this will be ongoing for the rest of SoC.
  • Experiment with some hardware detection, creating some images and testing on as many machines as possible (I am limited to mostly i386) - plan on using discover
  • I think it will be possible to use yaboot to boot debian-live on ppc & ppc64 unfortunatly the only box I can test on right now is an experimental blade server that IBM donated to a research project I work on at school (not ideal)... I have to find some people who are willing to test ISOs for me or a good emulator.
  • Start examining the scripts executed once casper hands everything over to init on the rootfs - I get some weird errors somtimes, prob because there are some saved setting from the box that created the iso. Modules that are installed at boot time etc.
*POST AS MUCH AS POSSIBLE: my mentor is starting to freak out because I did a poor job of keeping him posted on my work.

11 June 2006

Florian Ragwitz: Class::DBI is dead

David, you wrote about Class::DBI performance. Even without benchmarking you could have found out, that Class::DBI is quite slow. Fortunately there are some alternative object-relational mappers available in the perl universe. The best ones I found so far are Rose::DB::Object and DBIx::Class. I took a closer look at both and would like to share my experience. As this shows, RDBO is faster than DBIC in most of the case. The generated SQL doesn't differ too much and therefor it must be the perl side of things that makes the difference. Matt S Trout <dbix-class@trout.me.uk> says:
  However, RDBO achieves its perl speed by aggressive inlining of stuff
  etc. - for example the main object retrieval function in RDBO's manager
  class is >3000 lines in a single sub. DBIC values extensibility over a
  few extra sub calls, so methods are much more broken out and there are
  many more ways to hook into the DBIC execution process to extend.
Also its idea of resultsets is something I really love. Here's a small example to illustrate that:
  my $user_rs = $schema->resultset('User')->search(  registered => 1  );
  $user_rs    = $user_rs->search(
		    comment.title => 'Foo'  ,
		   
		  	join     =>   'article' => 'comment'  ,
			order_by => 'user.name',
		   ,
  );
  # no sql executed yet.
  # now you can use your resultset as an iterator or query a list of User
  # objects from it or ..
  while (my $user = $user_rs->next)  
	  ...
   
  # or
  my $count = $user_rs->count;
Using these resultset makes it extremely easy to built up queries piece by piece and to work together well with, for example, a templating system. You don't need to fetch all row-objects and give them to your template. You can just pass the iterator to the template library. There's a lot more to say about this two object-relational mappers (for example RDBO supports prefetching of multiple one-to-many at once, which DBIC doesn't), but maybe you just should take a look yourself. I personally prefer DBIx::Class for its vast extensibility.

Ian Murdock

Charles DiBona: “We think that Google poses a threat to Microsoft long-term by offering an alternate platform for application development. But we also believe that Microsoft recognizes this threat and is investing appropriately in its own operations to counter it.”

Wouter Verhelst: Sexism.

I had long thought that Sexism was a thing of the past; that women would have the ability to do what they wanted, and would mostly not be discriminated against. And while I was not so naive to think that there was no discrimination at all, I was under the impression that any such discrimination would mostly be something the discriminator did not think of consciously. That things like people being rejected outright for a job because they were women, and, well, "women can't work as well as men", were stories of the past. At least if I did ever discriminate against women, it never was intentional. Little did I realize how far away from the truth that was. As some troll's recent mails show, there are actually people out there who do thinkn that the influx of Women into Debian should be stopped, and that Debian should be a men only club. Our coward is too afraid to post under his real name, instead relying on tor and throwaway yahoo accounts to mask his identity. So let's call him 'Anonymous coward'. Occasionally, this is the main reason why I don't like tor; full anonymity should not be allowed, one should be able to have people stand up for what they say, and face the consequences. Anyway, that's a different story, for another time. What puzzles me is how anyone would think that there could possibly be a correlation between orphaned packages and the existance of Debian-Women. Personally, I've been happy that there is a Debian-Women which strives to make our community more complete; the lack of women (or any considerable part of the world population, for that matter) is a serious disadvantage for any community that tries to do something for the "greater good". Personally, I'm disgusted and offended by the mails this moron is sending; and even though I realize that there probably still is some (unconscious) discrimination against women, I can hope to safely say that Debian is tolerant regarding people that represent a minority amongst its community members. Even Jonathan Walther, whom we kicked out for (amongst other reasons) trolling on the debian-women mailinglist, never (to the best of my knowledge) said anything which implied that he thought Debian-Women should be disbanded (although he, err, had a "different" idea of what they should be doing). I guess all this only means we don't live in a utopia, and that Debian is not free of the plagues that will hit any sufficiently large community. But that doesn't negate the fact that I'm still offended by this lunatic.

David Pashley: Class::DBI performance

Class::DBI is a very nice database abstraction layer for perl. It allows you to define your tables and columns and it magically provides you with classes with accessors/mutators for those columns. With something like Class::DBI::Pg, you don't even need to tell it your columns; it asls the database on startup. It's all very cool mojo and massively decreases the development time on anything database related in perl. Unfortunately, as far as I can tell, it has a massive performance problem in its design. One of the features of Class::DBI is lazy population of data. It won't fetch data from the database until you try to use one of the accessors. This isn't normally a problem, except with retrieve_all(). Basically this function returns a list of objects for every row in your table. Unfortunately, due to the lazy loading of data, retrieve_all() calls SELECT id FROM table; and then every time you use an object it calls SELECT * FROM table WHERE id = n;. For a small table, this isn't too bad, but for a large table, it's a killer. I did a little benchmark today to see just how much slower it is over plain DBI. I wrote two functions which iterate over a table, assigning one value to a function (forcing Class::DBI to fetch the data). The table in question contains 635 rows. The code I used was:
use strict;
use warnings;
use Benchmark qw(:all) ;
use Foo;
use DBI;
sub class_dbi  
   for my $foo (Foo->retrieve_all())  
      my $bar = $foo->bar;
    
 
sub dbi  
   my $dbh = DBI->connect("dbi:Pg:dbname=$db;host=$host",$user, $passwd);
   my $sth = $dbh->prepare("SELECT * FROM foos;");
   $sth->execute();
   while(my $row = $sth->fetchrow_hashref())  
      my $bar = $row-> bar ;
    
 
cmpthese(100,  
      'Class::DBI' => 'class_dbi();',
      'DBI' => 'dbi();',
    );
The results:
brick david% perl benchmark.pl
           s/iter Class::DBI        DBI
Class::DBI   10.3         --       -97%
DBI         0.351      2845%         --
Class::DBI is more than 28 times slower than using DBI directly. I'm hoping that someone will now tell me "Oh you just do blah", otherwise I'm going to have to rewrite some of my code. One thing to learn from this is that reduction in development time can often cost you more in other areas, and it's often runtime performance. Update: It appears that the bug is that Class::DBI::Pg does't set the Essential list of columns, so Class::DBI uses the primary column. you can fix this by adding the following to your database modules:
__PACKAGE__->columns(Essential => __PACKAGE__->columns);
Remember you'll need to do that for each of your modules; you won't be able to do it in your superclass, as you won't have discovered your columns yet. This has increased performance, but not massively. New timings (with the addition of using Class::DBI through an iterator):
              s/iter Class::DBI it    Class::DBI           DBI
Class::DBI it   6.35            --           -2%          -94%
Class::DBI      6.23            2%            --          -94%
DBI            0.350         1714%         1680%            --
Update 2: It appears that further speedgains can be made by not using Class::DBI::Plugin::DateTime::Pg to convert the three timestamp columns in my table into DateTime objects.:
              s/iter Class::DBI it    Class::DBI           DBI
Class::DBI it   1.26            --          -11%          -72%
Class::DBI      1.12           12%            --          -69%
DBI            0.350          260%          220%            --

9 June 2006

Lars Wirzenius: Travel report: Debconf6, plus rest and relaxation in Los Angeles

This May I spent four consecutive weeks abroad, visiting first Debconf6 (including the Debcamp that preceded it), and then Los Angeles afterwards. I won't write a detailed report of everything that happened; nobody likes to read pages after pages of descriptions of meals and such. Instead, I'll jot down a few memories of moments. It's still long, sorry about that. May 6, about 3 AM, at home. Waiting outside for a taxi. Pretty tired already: spent the night frantically weighing all my clothes to optimize my packing. It is warm enough to wait outside without a jacket. The night is calm and beautiful. Mankind is so much nicer when it sleeps. May 6, about 4 AM, the Helsinki airport. I bump into Tuukka and Tulitar, who are on the same plane. Kaol also joins us, but that I knew beforehand. I'm suddenly much less apprehensive about the long flight. May 6, Mexico City airport. We meet more Debian people. The first taxi driver we're assigned seems to be suspect somehow. The other taxi drivers rebel and our luggage is put into another car, with another driver. I have no idea what is actually going on, but that develops into a theme for the Mexican part of the trip. May 6, Oaxtepec. We're finally here, but it seems very difficult for us to get rooms. While we wait, we and some of the people who arrived earlier go to a restaurant. The Germans want to order the same thing for everyone, but fail to agree on what, so eventually we order all the steaks on the menu. The food is good, but the restaurant staff would probably have preferred to go home at closing time. We eventually get rooms and I collapse into bed. Debcamp week, Oaxtepec. We have network. We don't have network. We have network. We don't have network. We have network. We don't have network. We're frustrated all the time. I was supposed to do billable work (related to Debian, even), and can't. Should've prepared for this possiblity. I'm too dependent on a good Internet connection, need to figure out to reduce this dependency. To have something else to do, I start the Mugshots project: take a photo of everyone together with a paper with their name, or IRC nick, or anything else they want to say about themselves. Debconf, late one evening. Everyone is partying one way or another, drinking, dancing, discussing. Lots of people around already. Nice people. Still I withdraw into myself. I'm physically present, but not interacting. Never know what to say; exhausted by having to remember who everyone is. I'm introverted, and deal better with smaller gatherings. My life's ambition is to become a multi-millionaire recluse. Debconf, late another evening, Oaxtepec. Everyone else is getting drunk, and it affects me. I make the silly bet about Debian releasing on time: if we do, I'll get a Debian tattoo. Not to worry, I win either way. Debconf, some evening or another. I participate in a game of Mao. It is an unpleasant game, for me. I suspected this beforehand, but now I've verified it. It is best for everyone if I never play again, now that I know enough to at least try to reach my personal, alternative goals in the game, which make the game less fun for everyone else. (Better for me too, since I make fewer enemies.) Debconf, day after formal dinner. I learn that Jonathan (Ted) Walther is gone. Good riddance. I suprise myself with the amount of glee I feel when removing him from the Mugshots gallery. (That, incidentally, was something I decided all by myself. It's my gallery, after all.) Debconf. The sponsored food is quite acceptable, but eventually becomes boring and bland. Mass-kitchen food can't but help to be like that, I guess, unless it gets expensive. I don't want to see another chicken leg ever. Debconf, early in the week. Droidy (Leena) and Burger (Ville) arrive. Very nice. Helps massively with my withdrawnness to have real life friends around. I still go early to bed, though, instead of participating in the partying. Debconf, late in the week. I get Ville and Frans Pop to talk and Ville agrees to have look at the installation manual. He's an actual (former) professional tech writer, so hopefully will be able to improve it a lot. May 22, Oaxtepec. Last day. I spend all morning packing, checking out, and waiting for a taxi. Strangely relaxing. Debconf is finally over, and although I did enjoy parts of it, and don't regret coming, I'm not sad it's over. May 22, evening, Los Angeles. Julie and Kristian meet me at the airport. Hugging happens. Happiness happens. I get my first taste of LA traffic, which seems to mostly consist of people complaining about how slow it is while driving very fast. May 23, the Promenade, Los Angeles. A three block long walking street of shops, with three (count them: three) bookstores. The two I have time for even smelled nice. I am in hog heaven. Colossal credit card cringing. May 24, Getty Center. A very nice, very impressive place architecturally. The art is good too. As usual, the impressionists make the strongest impression on me, but there's lots of other good stuff, too. Not everything, though: Robert Adams's photographs leave me unimpressed, possibly I lack the cultural reference points necessary to appreciate them. He has many photos of shabby temporary shelter-like houses that in reality seem to be nice south-western US houses from the 60s and 70s. My intuition just screams that the houses and their inhabitants won't survive their first winter. But of course, they don't have a cold winter there. The rest of the exhibition (or what I have time to see) has way too many portraits for one session. Portraits get boring after the first dozen or so, I'm afraid, unless there's something extraordinary about them. They are several rooms rebuilt from 18th century French aristocratic houses. Very nice, very pretty, but I would not like to live that uncomfortably. I have a small epiphany about nudity in art in one of the rooms with statues. I've always known nudity could be portrayed erotically, such as having two lovers embrace passionately; or that it could be portrayed more or less abstractly, using the human body, or parts of it, just because it can be a very beautiful subject, just like a flower, but without giving it a greater significance. It's decoration, nothing more. There are two statues in the Getty collection that show me a third way. They are exquisitely sensual, evoking a sense of form and texture that make me think of human skin and flesh, a sense that if one were to touch the statue, it would feel like touching a live human. Yet none of this is erotic at all. I'm not sure I can explain this, one may have to experience it for oneself. Unfortunately, I was too overwhelmed to write down the names of the statues. May 23, Los Angeles. The food takes me by surprise. Julie has always told me that food in LA is good, much better than in Finland, but she hadn't prepared me for an all-out assault on my palate. Dine well in LA and die happy. (When I go back to Finland, I will have to learn again how to cook, dammit.) May 25, Los Angeles. Jesus reminds me via SMS about the Towel Day. I participate. My hosts think I'm a bit weird, but in a funny way. Many happy days in May, Los Angeles. I visit the Los Angeles County Museum of Art (exhibiting a weird Italian designer, some boring Klimt paintings, and a very nice general collection); the Natural history museum of LA (dinosaurs! real skeletons! big ones! T.Rex! for the first time in my life! they're huge!); the Museum of Jurassic Technology (weird); had dinner with some local Debian people, and visited the local Google office afterwards; the Hollywood Boulevard at night (funny, for a while). And other places I've forgotten already. LA is not small. I visit a political theater play that is brilliant. I also visit my first American BBQ, and have my first experience with the LA gay community at the same time. I'm later told that these two are separable. I enjoy them both a lot. I visit more bookstores. My hosts keep finding new ones they want to recommend to me. Parking would be a pain in LA, except that it means that people do actually walk a lot, since the nearest parking space is always three blocks away, or five levels up without an elevator. May 28, Los Angeles. I innocently mention that LA is such a nice place, with nice people and gorgeous food, that I wouldn't mind living there. Julie's brother hears this and immediately connects me with someone he knows whose company is looking for new people. During the next three days, we meet and talk and things seem favorable. Even if I decide not to go (it may be too huge a change for me now), it's nice to feel appreciated. June 1, Los Angeles. Leaving for home. Not a sad thing, but I'm definitely feeling wistful. I get hugged about fourteen thousand times. Parting is such sweet sorrow. I will definitely have to go back to LA, one day.

7 June 2006

Erich Schubert: Mystery subscription

My last.fm status is "subscriber", though I never subscribed. Interesting. I thought maybe it was a test subscription of one month for my birthday or so (which was less then one month ago), but why didn't I get any mail to tell me of this test subscription then? Weird. So if anyone "secretly donated" this subscription to me, please, don't. But who would do so without telling me? I'm happy with their free service... So if you insist on wasting some money on me, consider my amazon wishlist instead (though I don't really care much about that stuff either...). Well, maybe the "ad" message was just delivered to the spam folder. After all, there is still a "subscribe" button on the last.fm page.

5 June 2006

Matt Brown: The importance of truly Open Standards

I think the current brouhaha surrounding PDF functionality in Office 2007 is an excellent object lesson in why software patents shouldn’t be allowed anywhere near an Open Standard (and software in general!). I’m not sure if a patent is the tool of choice that Adobe is wielding in this particular example, but even if it’s not Patent’s present a large risk to Open Standards and should be resisted. It’s the same as the distinction between a Shared Source program and Open Source or Free Software. You may have the source code for a shared source program and be able to see how it is implemented and how it operates, but there are restrictions preventing you from doing anything with that information. Likewise a standard shouldn’t be called Open simply because you have details on how it is implemented (although that is undoubtably a critical part of openness). An Open Standard should be one which is openly specified and completely free of any other encumberances (such as patents). Unfortunately I don’t see any of the standards bodies making this a priority. My understanding of the IETF RFC process is that you have to file a statement stating any IP claims but you’re not prevented from standardising somethign that you hold a patent on. W3C and the IEEE seem slightly less prone to this problem as they tend to have committee oriented standards process that require more interaction, but I’m still not aware of any direct efforts to prevent parts of the resulting standard from becoming patent encumbered. If the lack of truly Open Standards can bite Microsoft it can bit any of us. It’s not an issue we should be ignoring.

4 June 2006

Enrico Zini: Dapper on XEN (part 1/1, unfinished)

I need to do some work on a Dapper system, so it's time to try out xen:
apt-get install linux-image-2.6-xen-686 xen-hypervisor-3.0-i386 xen-utils-3.0 xen-tools
xen-tools tries to recommend xen, which is old and has been requested to be removed from the archive. The recommends needs to be ignored, which is not that trivial to do with aptitude. The linux-image doesn't happen to have an initrd. The bug has already been reported. One can recreate one using:
mkinitrd -o/boot/initrd-2.6.16-2-xen-686.img  2.6.16-2-xen-686
Then one adds this to /boot/grub/menu.lst:
title Xen
root (hd0,0)
kernel /boot/xen-3.0-i386.gz
module /boot/vmlinuz-2.6.16-2-xen-686 root=/dev/hda1
module /boot/initrd.img-2.6.16-2-xen-smp
And it boots lovely:
# xm list
Name                              ID Mem(MiB) VCPUs State  Time(s)
Domain-0                           0      939     2 r-----  4677.9
Now xen-tools provides xen-create-image. I'll try to create the Dapper image (deboostrap in my system has already been changed to be able to install dapper):
xen-create-image --size=4G --swap=128M --dhcp --volume=marvin \
                 --hostname=cavazza --dist=dapper --fs=xfs
But I get:
[...]
Creating swapfile : /dev/marvin/cavazza-swap
Done
Creating disk image: /dev/marvin/cavazza-root
Done
Creating xfs filesystem
Done
Installing the base system.   This will take a while!
Copying files from host to image.
Finished
Something went wrong with the debootstrap installation
Aborting
I love detailed error messages. This one is not. So I'll have to do by hand. The documentation says:
Before you can start an additional domain, you must create a configuration file. We provide two example files which you can use as a starting point:
  • /etc/xen/xmexample1 is a simple template configuration file for describing a single VM.
  • /etc/xen/xmexample2 file is a template description that is intended to be reused for multiple virtual machines. Setting the value of the vmid variable on the xm command line fills in parts of this template.
But in Debian there's no trace of the two example files. I've reported to Guido Trotter. Then I googled for the two files and found them. Now, on to adapt Dapper to run under Xen. The good Ubuntu wiki has an HOWTO which crudely makes you install the Tarballed Pre-built installations of Xen 3.0. It's sad that xen-enabled kernels didn't make it into Dapper roper. I can make this:
kernel = "/boot/vmlinuz-xen-domu"
memory = 128
name = "Dapper"
disk = [ 'phy:marvin/dapper,hda1,w' ]
dhcp="dhcp"
root = "/dev/hda1 ro"
I used the pre-built kernel as a domU, but:
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(3,1)
Sigh. I'll try with the Debian Xen-enabled kernels. Installing yaird and linux-image-2.6.16-2-xen-686_2.6.16-14_i386.deb and linux-modules-2.6.16-2-xen-686. It WORKS! Now, that's not Dapper: that's a debootstrap install of it. No user, no root password, no desktop, no shiny Ubuntu custom config. How to finish the installation? No idea. In the past, I could run base-config, but now it doesn't exist anymore. It seems that the best bet is not to use debootstrap, but to install the CD in the LVM partition using QEMU and then chroot into it and then install the debian xen kernel to boot it. A QEMU install will take ages, and I expect some trouble, like getting partman to see LVM. Maybe I better install on a disk image file and then use tar and netcat to bring the ubuntu installation out of the disk image and into the LVM image. For today, I didn't make it. This job slips another day. Frustration. I hope that at least this description of my efforts so far can be helpful to someone trying similar things. One suggestion I got was to boot the Dapper CD and install from there. I can't: this computer has no bootable CD. Update: I found out http://xensource.com/summerofcode.html and it says:
Project Idea 3: Xen Desktop Outside of Domain 0 This project would deliver a Xen desktop to the user of a client system that presents an abstraction of a virtual desktop to the user in which multiple guests share the virtual desktop, each with a subset of the desktop resources (pixels, etc). Smart management of the sound drivers would allow mixing of sound from multiple guests to the single device used for output. Microphone input could be broadcast to all guests. Technologies such as ALSA already emulate playback settings by downsampling for the hardware etc. so its not hard to imagine a xen-snd-front device munging the data to a common format used for the internal sound card. Issues: synchronization of playback.
So, what I'm trying to do has been proposed as a SoC project and (I guess) is something that can't be done overnight. Although I was planning to run a diskless gdm accessed with XDMCMP from domain0's X server, and to just hand out the audio card to the domU using pciback, so my task would have been easier. Thus I give up using Xen for this one. I'll work around the lack-of-bootable-cd limitation of this computer by installing Ubuntu using QEMU. Which I found out requires using the "Alternate install" CD. The desktop install CD is a live CD installing with Ubuntu Expresso, and the live CD doesn't seem to work in QEMU. Or, I'll install in a LVM partition in the laptop and then move it around using the network. This one's probably faster. Could this experience be an interesting use case for Edgy Eft?

Michael Janssen: Pet Peeve #2: Microsoft Excel and Unicode

Recently I was working on an application for ICRA2006 and I needed to output some data to be read by Excel. This would seem like an easy problem, just output in CSV and Excel can open the text just fine. Unfortunately Excel has one of the most backwards implementations of international character support that I have ever worked with. Being nieve to this, I simply tried UTF-8. I was greeted with multiple strange characters in the place of my normal umlauted and tilde-d letters. A quick search turned up that when Excel reads a CSV, it assumes that it is in ASCII or . However, you can import the CSV as a text file and select it as unicode. This is where my real pet peeve is: You can't select the character set of a CSV in Excel. You have to rename the file as a .TXT file and go through a wizard, basically spelling out that yes, it is comma separated and yes, this is the data I want to import. There's one drop down box which specifies the character set. Of course Unicode is supported there.

In the search for a solution to this problem which didn't involve a 10-step process for the customer, I discovered the Unicode marker which is supported by Wordpad, but amazingly, does nothing but add a bit of garbage in Excel. The sad part is that I can be almost certain that the next version of Office will continue to have this problem. My only hope is that somehow the XML format will be open enough to just create a native Excel file.

3 June 2006

Erich Schubert: SELinux support is a release goal for etch

SELinux is an official release goal for etch. However, we're currently far from that. So far, very few developers are actually using SELinux (if any), or even testing it. I won't be doing anything on SELinux for the next 6 weeks at least, more likely 6 months. So currently I don't see how we can manage this goal, unless more DDs (and of course users; you don't need to be allowed to upload packages) help getting SELinux properly integrated. (You can run SELinux on etch without having to recompile any software I think, but you'll have to do quite a bit of customizing yourself right now.) For a coordinated effort to get SELinux into shape for etch, there are Debian SELinux mailing lists, but so far they're basically unused. [Update: yes, one option is to remove SELinux from the etch release goals. I was in fact rather surprised to see SELinux listed there.] [Update #2: It was pointed out, that the first occurence of "release goals" in that mail is prefixed by "pet". Don't ask me what the difference between a release goal and a pet release goal is, though. I guess something like being in an official wishlist...]

2 June 2006

Martin F. Krafft: Keeping an eye on package involvement

I find my developer QA page to be unappropriately long. At the same time, I have completely lost track of which packages I maintain (well, sort of), (am supposed to) co-maintain, and to which packages I've subscribed via the PTS. A script to the rescue! Please excuse that it's bash (which sucks), but there just isn't another more rapid development tool for such petty tasks out there. Without further ado, you can download checkpkgs and give it a whirl:
  # use $DEBEMAIL
$ checkpkgs
  # use explicit email
$ checkpkgs dude@thisrocksmysocks.org
  # use separate PTS subscription email address
$ checkpkgs dude@thisrocksmysocks.org dude-pts@thisrocksmysocks.org
  # use separate PTS subscription email address and $DEBEMAIL
$ DEBPTSEMAIL=dude-pts@thisrocksmysocks.org checkpkgs
The output is quite simply a table of the following form:
hibernate;cameron@patrick.wattle.id.au;0;1;madduck-pts@debian.org
unifdef;bob@proulx.com;0;0;madduck-pts@debian.org
thinkpad;pkg-tpctl-devel@lists.alioth.debian.org;0;1;
The columns are:
  • package name
  • maintainer
  • whether I am the maintainer
  • whether I am a co-maintainer
  • the address with which I am subscribed to the PTS (if at all) (this is either ($1 or $2) or ($DEBEMAIL or $DEBPTSEMAIL))
So from the above output, I know that:
  • I am a co-maintainer of hibernate and I am properly subscribed
  • I am subscribed to unifdef but I am not a maintainer. Since I sponsored the package, I better keep track of it though.
  • I am a co-maintainer of thinkpad, but I have not yet subscribed to the PTS, so I better do this right now.
The script's only a quick hack, but maybe people come to like it. PS: I really need to set up version control again.

1 June 2006

David Pashley: Tidying Up

A few hours ago I got stressed about the lack of leg room under my desk and ended up spending the next few tidying and moving all of my computers to under the next desk. I also made the mistake of starting to remove keys from my keyboard to clean something sticky and found myself surrounded by keys and a keyless keyboard. It's now nice and shiny, which is more than can be said for the rest of the flat, which is now overrun with all the crap that was around my desk. Another thing that could do with a tidy up is Eddie, my Java liberal feed parsing library. After the initial coding sprint, I've had time to sit back and look at the design of the library and clean up any thing that sticks out. As mentioned in a previous entry, one of the things that has bothered me is that when ever you need to call an object method, you need to be certain that the object is not null. The means you end up with code like:
if (string != null && strong.equals("string"))  
This quickly becomes tiresome and the test for null distracts from the meaning of the code. Fortunately I was reminded of an improvement for string objects. Ideally, we should all be writing comparison conditionals like rvalue == lvalue. (an rvalue mostly is an expresion you can't assign to). The most common rvalue is a literal value like a string constant. The advantage of getting into the habit of writing code like this is that you'll discover at compile time when you accidentally write = rather than ==. Because you can't assign to an rvalue, the compiler will complain. What makes this interesting from a java string point of view is that you can call methods on string literals. Comparing a variable to a string literal, rather than calling .equals() on a variable is that the string literal is not going to be null, so you can remove the test for null and simplify the code:
if("string".equals(string))  
I know it's not everyone's cup of tea, but I prefer it to testing for null every time I look at a string. The other thing is that I've been reading Hardcore Java by Robert Simmons at work. Considering I've only got a few pages in so far. I've received a surprisingly large number of ideas to improve my code. The one that sticks in my head is using assert for doing post and pre conditions on your functions. Using asserts have number of advantages over throwing exceptions, including the fact they get optimised away when you do a production release. In Eddie, during a <feed> element I determine the version of Atom that we are parsing. This had a number of nested if/else if/else blocks. At the end of the function, I wanted to make sure I had set the version string to something, so had the following code:
if (!this.feed.has("format"))  
   throw new SAXParseException("Failed to detect Atom format", this.locator);
 
However, using assertions I can write this as
assert(this.feed.has("format")) : "Failed to detect Atom format";
I highly recommend the Hardcore java book if you want to improve your java programming. It includes sections on the new features of Java 1.5 and using collections. I've made a couple of other cleanups including going through member variable access specifiers to make sure they are right and making several public methods and variables and making them priavte. I also have a couple of ideas about refactoring some of the code to clean it up. Redesigning and refactoring code is almost more fun than writing it in the first place. You get to be in competition with yourself, challenging yourself to write better code and end up with cleaner code in the process. A couple of things I want to do in the near future is use a profiler and code coverage tools. If anyone has recommendations for either of these tools that integrates nicely with eclipse, I'd love to know.

31 May 2006

David Pashley: Init(1) Causing zombies

Had an interesting problem today with one of our servers at work. First thing I noticed was yesterday that an upgrade of Apache2 didn't complete properly because /etc/init.d/apache2 stop didn't return. Killing it and starting apache allowed the upgrade to finish. I noticed there was a zombie process but didn't think too much off it. Then this morning got an email from the MD saying that various internal services websites were down (webmail, wiki etc). My manager noticed that it was due to logrotate hanging, again because restarting apache had hung. Looking at the server I noticed a few more zombie processes. One thing I'd noticed was that all these processes had reparented themselves under init and a quick web search later confirmed that init(1) should be reaping these processes. I thought maybe restarting init would clear the zombies. I tried running telinit q to reread the config file, but that returned an error message about timing out on /dev/initctl named pipe. I checked that file existed and everything looked fine. The next thing I checked was the other end of the named pipe by running lsof -p 1. This showed that init had /dev/console rather than /dev/initctl as fd 0. I tried running kill -SIGHUP 1, but that didn't do anything. Then I tried kill -SIGUSR1 1, but that didn't do anything either. I checked the console, but there wasn't enough scrollback to see the system booting and decided to schedule a reboot for this evening. Rebooting the server presented me with an interesting challenge. Normally the shutdown command signals to init to change to runlevel 0 or 6 to shutdown or reboot using /dev/initctl. Of course init wasn't listening on this file, so that was out. Sending it an SIGINT signal (the same signal init gets on ctrl-alt-delete) had no response. Obviously telinit 0 wasn't going to work either. I decided to start shutting services down manually with the help of Brett Parker. The idea was to stop all non-essential services, unexporting nfs exports, remounting disks read-only and then either using sysrq or a hardware reset. Unfortunately someone accidentally ran /etc/init.d/halt stop, hanging the server, but he is suffering from a bad cold today so I forgive him. The server restarted without a hitch (thank god for ext3) and running lsof -p 1 showed init having /dev/initctl open. I don't know what happened to init the last reboot on Monday, but a reboot seemed to fix it. Odd bug, but thankfully it was a nice simple fix. I could have spent the evening debugged init. :)

30 May 2006

Erich Schubert: IEEE1394 drives... damned prolific...

Since the prolific-based 3.5" firewire+usb external HDD box I have is total crap (it often stops and restarts the disk, going readonly and losing data), I've been looking for a different drive. Unfortunately, I havn't been successful so far. The cheap computer stores around here all seem to only have drives with (probabliy newer revisions of) this chipset... and I'd really like to avoid buying another prolific crap. I updated yesterday to the latest firmware for the PL3507, and I also checked that the drive I put in the box doesn't have any errors. Didn't help. The drive still is very unreliable, in both firewire and USB mode. Unfortunately, my laptop only has USB 1, otherwise I'd buy an USB-only box which supposedly uses a different chipset.

29 May 2006

Norbert Tretkowski: Remaining photos uploaded

As promised before, I uploaded the medium and high quality photos from DebCamp, DebConf, the daytrips to Teotihuacan and Xochicalco, and LUG Camp into my gallery. Have fun. Update: Unfortunately I forgot to adjust the date on my digicam from CDT back to CET, so the time in the exif tags from the photos I made at LUG Camp was 7 hours behind. This problem can be fixed using the -ta option of jhead, thanks Colin for the hint.

28 May 2006

Ian Murdock: Picasa: Why not a native Linux port?

Robert Love: “In Google’s calculation, the cost of a native port outweighed the benefit of a native [port] v s-a-v s a WINE-fueled Picasa. Is that really unexpected? Our development platform is no shining star. We have two toolkits, poor binary compatibility, and unclear direction. To be sure, vendors such as Novell are devoting increasing effort toward improving the Linux ISV and platform story. But the community must make it a priority as well, from the kernel up through the highest levels of the graphical desktop.” That’s part of it (and certainly an accurate assessment of the kind of thinking we need as a community to make Linux a better platform). I think there’s another part of it, though, and that’s that operating systems in general are fundamentally uninteresting to Google as platforms in themselves, as well they should be—after all, the web is Google’s platform, and they dominate there, so why should they promote an alternative where they are much weaker? In my view, the (fat) client bits merely exist to ease the transition from the current, predominantly desktop centric world to a new web centric world where Google naturally thrives. After all, such paradigm shifts are always gradual. There’s the offline question too—if all your data is in the cloud, how do you get to it when you’re not online?—though I suspect over time the browser will provide offline capabilities of some sort, so this is largely a temporary problem too. So, using Wine in ports not only minimizes the incremental cost of building out a foothold on the client (which, again, is a necessity for Google, given that Google’s biggest competitor is between them and 90% of their customers), but it raises the fat client platform up a level too. In other words, it builds a fat client platform with a consistent look and feel across desktop environments; and with this, the operating system underneath rapidly becomes irrelevant. The real question is when Google will hook Picasa into a photo sharing service of its own. I already keep my photos in the cloud, and I’d love to organize and edit them in Picasa (Organizr is nice, but nowhere near as nice as a desktop application, and it doesn’t have some basic functionality, like import from camera and red eye repair). Of course, this is just one example of Google having some platform thinking issues of its own, though that’ll have to be a rant for another day.

27 May 2006

Erich Schubert: Smack your laptop to switch desktops

This page uses the motion sensors present in some new harddrives in an interesting way. Check out the flash/youtube video. It's too funny. My laptops harddrive doesn't have motion sensors - but it has two very nice keys I use for desktop switching, right next to the cursor keys. Of all key combinations I've used so far for desktop switching this is the one I love most... Unfortunately, this is a very unusual keyboard layout, and I think even IBM/Lenovo doesn't have these keys on newer models any more. A shame.

26 May 2006

Erich Schubert: Time to pptcast?

Since many "office" people love to send "funny" powerpoints around via email (which is stupid, because email is ~40% slower than others) we need to provide them with an application to do this much easier. Time to create the first pptcast. Or is ist pptfeed? pptlog? Fortunately I'm not interested in reading them.

Rapha&#235;l Hertzog: Improving Debian as a whole

I have to agree with Joey, I have the feeling that we’re not doing many “transversal” improvements and that we’re busy enough simply trying to keep up with new upstream version of software. That’s not completely true but I also wouldn’t want Debian to evolve in that direction. I think that less people are interested in doing large scale changes because the coordination with 1000 maintainers and 10000 packages is simply too complicated and taking too much time. Luckily we had significant improvements recently that should ease that coordination work. I’m thinking mainly of the usertags that allow us to use the BTS as big TODO list. But usertags are obscure to many people and not well documented yet. So there’s room for improvements: that’s why I proposed a project for Google’s Summer of code called “Distribution-wide tracker tools” (see list of accepted projects here). Check the project proposal of Arnaud Fontaine to have a more precise idea of what it could give. I want this infrastructure because I think it may help Utnubu to effectively coordinate the integration of Ubuntu improvements and because it may help Debian be more on the leading edge (instead of trying to catchup with derivatives). And last but not least, I believe many more people (with different level of skills) will be able to effectively work on some projects once the work to do has been clearly identified and registered in such a system. The project is about to start, so all ideas/comments are welcome of course!

Next.

Previous.