Search Results: "pm"

21 August 2006

Gustavo Franco: love conflicts with development

<rant>You sit in your desk for 8 hours with tons of hardware around you. You've enough time and tools to prepare, test stuff, do commits, uploads and what do you do? Almost nothing! Why? I'll let you figure it out by yourself.</rant>

20 August 2006

Lucas Nussbaum: Who is using Debian s and Ubuntu s development versions ?

Scott J. Remnant claimed that it s the development community that use Debian testing or unstable. This doesn’t match my personal experience (I know many people running Debian testing/unstable who aren’t interested in Debian development), however I needed a better way to check this than just “my personal experience”. So I tried to answer the following question : Are normal users using Ubuntu edgy and Debian testing/unstable, or do they stick with Ubuntu dapper and Debian stable ? Simplified summary: (read below for details)
Ubuntu users:
…Ubuntu dapper (stable): 96%
…Ubuntu edgy (development): 4%
Debian users:
…Debian sarge (stable): 24%
…Debian testing/unstable (development): 76%
Detailed process: First, I did a poll on #ubuntu@irc.freenode.net using a CTCP VERSION query, and looked at people using Xchat and Ubuntu (Ubuntu is displayed in CTCP VERSION replies). I got 288 replies from Xchat users, 197 of them were using the Xchat package from Ubuntu. 189 (96%) of them are using version 2.6.1 (the version in Dapper), 8 (4%) of them are using versions 2.6.4 or 2.6.6. Then, I did the same poll on #debian@irc.oftc.net. I got 85 replies from Xchat users, and filtered out 15 of them because they weren’t from Debian users for obvious reasons. 17 users (24%) are using versions 2.4.0 or 2.4.1 (Debian sarge), 53 users (76%) are using versions 2.4.3, 2.4.4, 2.4.5, 2.6.1 and 2.6.4 (Debian testing/unstable - v.2.6.4, the current version in testing/unstable, has 41 users on its own). Statistically speaking, this isn’t very good, so I compared those results with another source of information : Debian popularity contest. 3002 submitters are using v.1.28 of the popularity-contest package (version in sarge) while 9682 submitters are using v.1.33 (version currently in Debian testing/unstable). That accounts for respectively 24% and 76% ! Same as above ! (OK, this is not so good, because I haven’t considered the intermediate versions here, while I did above)
Of course, those numbers are huge approximations, and are subject to a lot of hypotheses and questions, amongst them: I didn’t run the same poll on #debian-devel and #ubuntu-devel because I received some unpleasant comments already, and don’t want to run into angry Debian or Ubuntu developers (I’m still waiting for DAM approval after all :-)

Martin Michlmayr: Status update of Debian on Linksys NSLU2

Beta 3 of debian-installer has been released recently and some people are wondering about the status of Debian on Linksys NSLU2. This ARM based device includes an Ethernet chip for which Intel provides a driver (based on some code which only recently became free software and which still requires some microcode which is only distributable together with a click-through license). During the beta 3 development process, I tested the installation on NSLU2 with my USB Ethernet gadget to make sure it works. When beta 3 came out, I wanted to integrate Intel's module and make unofficial images available. Marcus Better did some great work in the last few weeks putting the sources into a Debian packages from which the modules can easily be created. Unfortunately, they don't seem to work. My kernel oopses immediately when I insert them. I spent more than a day compiling the modules myself, trying various things, but to no avail. I have now given up and sent a call for help. What this means is that the unofficial images with the Ethernet driver are unsupported right now, even though Debian on NSLU2 works fine if you have a USB Ethernet adapter. In the long run, I hope to be able to support Ethernet using a driver which is currently being developed. The author has made good progress already and is aiming for mainline inclusion. I hope it will be usable on the NSLU2 in the next few weeks but it's hard to tell. There has also been some development that users may not directly see but which is important maintenance work. The firmware of the NSLU2 has a 1 MB limit for the kernel. So far, we have provided a special kernel for the NSLU2 but it would be better if we could use one generic kernel for all IXP4xx based devices. Together with Marc Singer, the author of APEX, we're now integrating his boot loader as a 2nd stage loader to work around this limit. In summary, NSLU2 development work is still being done, even though the in-built Ethernet is currently not supported. This problem will be addressed in the long run with a proper, mainline driver which will hopfully be usable in the next few weeks. Help with NSLU2 support is welcome because even though I'm still working on it, I now have a number of more interesting devices I'm currently working on (more on this later).

19 August 2006

Uwe Hermann: OSI Fights Free Software License Proliferation

OSI's License Proliferation Committee has published a first draft report which categorizes Free Software (or Open Source, if you want) licenses. The categories are The charter of the committee states "[t]he purpose of the Committee is to identify and lessen or remove issues caused by license proliferation", which is a good thing, IMHO. I'm fed up with the recent trend that every company or organization invents their own "open source" license (often I would not even consider such licenses remotely open of free, but that's another issue). In total, the report categorizes 59 licenses into the above categories. I'm sure most people would agree that such a huge amount of licenses is totally useless and only creates confusion and problems. Please, if you have to decide on a Free Software license one of these days, choose a major, well-known one. Or even better, if you currently use a non-standard license in one of your projects, please consider switching to a more streamline license which fits your needs. Thanks! (via Heise)

17 August 2006

Evan Prodromou: 30 Thermidor CCXIV

I can't believe it's the end of wp:Thermidor already. The summer's going so fast; Fructidor is the last month of the year in the French Revolutionary Calendar, and then it's the fall equinox and things change. Speaking of the fall equinox, the Br leurs de Montr al Burners, the local Burning Man regional organization in Montreal, is planning a decompression regional burn campout event for the weekend of 22-24 September 2006. Folks interested in Burning Man and near Montreal are encouraged (by me) to participate on the Br leurs wiki or join the bruleurs mailing list. And, yes, the event starts on La F te de la R volution on the Republican calendar. Neat. tags:

And you smell like one too As has been mentioned in many other places, yesterday, 16 Aug, was the 13th anniversary of the launch of Debian. It's definitely something to celebrate -- the project continues to be a huge success and a bulwark of Free Software. Robin Millette and the Atelier du Libre will be hosting a Debian birthday party here in Montreal on Saturday the 19th at 8655 rue St Denis, at the Cr mazie metro stop. The event is from 1:30PM to 5:30PM and is open to all. More info at http://rym.waglo.com/wordpress/2006/08/17/party/ . tags:

16 August 2006

Martin F. Krafft: Happy Birthday, Debian

We are 13 years old today. If we believe the Chinese, where "thirteen" sounds similar to " ", which means "must be alive" (thanks, Wikipedia), then all I can add is: We surely are! Alive and Kickin'! (and we didn't need to resort to Chinese for that) Thanks, Ian!

15 August 2006

Martin F. Krafft: Test-driven development

Lars, I strongly suggest test-driven development. It seems to preemptively do the most relevant of what the debugging rules you quote instruct you to do later. The principles:
  1. design your interface
  2. write tests that fail
  3. write the code to make the tests stop failing
  4. refactor
  5. lather, rinse, repeat
It's a philosophy that anticipates bugs and errors, yet it uses that very fact to guard against them. Test-driven development has changed the way I (and many others, I learnt it while becoming a Plone developer) plan, design, think about, and implement code. And yet, it's really not far away from how you're already doing things. Really. For C++ coders, I recommend libtut, which is elegant and non-intrusive. PS: The definitive (and quite good) book on test-driven development by Kent Beck is only 200-something pages, so it's also in the domain of shorter books. Update: sorry, liw, for making it sound like I assumed you had spent the last years in a cocoon or cave or curled up in a corner. I didn't intend to doubt that you know of TDD, just wanted to get that off my mind what your post put there. And of course, I agree with you: TDD will not eliminate debugging. But it will make your debugging sessions a lot less frequent, even though the really long and painful ones are exactly those that it won't prevent...

Marc 'Zugschlus' Haber: upstream version numbers and uscan

I have just experienced a nightmare with a watch file. The phpmyadmin upstream dumps all released files onto a single web page, and they use version numbering a.b.c-foon, which foo being beta, rc, or pl, and of couse a.b.c for a release version.This has resulted in the following (probably incorrect) watch file version=3 opts= uversionmangle=s/0\.tar/0-5.tar/;s/beta/0./;s/rc/1./;s/pl/9./,dversionmangle=s/0\.tar/0-5.tar/;s/beta/0./;s/rc/1./;s/pl/9./ http://prdownloads.sourceforge.net/phpmyadmin/ (?:.*/)?phpMyAdmin-([-0-9\.a-z]+)\.tar.gz$ 2.7.0-pl2 uupdateAs Germans say: Gesundheit

14 August 2006

Jaldhar Vyas: Debian Linux Adopts SeaMonkey

Well it's in NEW anyway. It should arrive in unstable in a few days once the ftpmasters process it. For those who don't know SeaMonkey is the latest incarnation of the erstwhile Mozilla Internet suite. Apart from a web browser, it includes an email and Usenet reader, an HTML editor, and IRC client. It's a big complex piece of software and I would like to get a packaging team going to look after it. My current work is available in SVN. There are a huge number of lintian errors most of which are probably bogus but still need to to be looked at. I'm sure there are other improvements that could be made too. If you would like to help, please contact me In light of some of the FUD that's going around I should mention that this package was made possible by support from the good people of Linspire Update: Ow the package was rejected. Apparently you're supposed to fix the huge list of packaging bugs before it gets distributed to millions of people. Back to the drawing board. Update 2: SeaMonkey SeaMonkey SeaMonkey

13 August 2006

Biella Coleman: STS Wiki, again

Bryan Pfaffenberger initiated and helps to maintain what has become the number one wiki and, thus, source on the academic field of Science, Technology, and Society. Since he first put the wiki up, it has balloned nicely but there is of course more room for expansion and development. He recently sent out a nice letter to the STS-grad list remidning folks to participate. And indeed, if there is something to put up, please do!

Edd Dumbill: Ten things you need to know about me

To keep you reading all the way through, the best one's at the end.
1. I'm striking back out on my own againAfter a couple of years working full time in an office, I've decided it's the right time to take my own business, the Useful Information Company, further on. As the rest of this post illustrates, we'll be involved in web technology conferences, our own products, and Rails-based software development. I'm always looking for collaborators, comment and customers, so get in touch if any of this lot grabs you.
2. I'm organising an amazing web developer event in London, late OctoberUseful's first venture is XTech WebDev London. This is the first of a new type of event for XTech: a focused one-day training event for web developers. The idea is to focus on the best practice in web development that emerges from the cutting-edge work demonstrated in the regular XTech conference.This year, that means: Ajax, web standards, open data, REST and Ruby on Rails. Any web developer wanting inspiration and a view of the future should come along. London, UK. October 31, 2006. You can reserve your place now. (We're right-priced for your training budget, and start late enough that folks could make it from all over the UK and beyond without staying a night.)
3. The pace is hotting up for XTech 2007 in ParisMeanwhile, I'm busy putting together the call for participation and programme committee for XTech 2007, in Paris, France. Our theme is "The Ubiquitous Web." As keynotes, I'm pleased to welcome Adam Greenfield, Matt Webb and Jack Schulze.
4. I'll be off travelling shortly and reconnect with old and new friends Though a little daunted by the new face of air travel, I'm heading out to O'Reilly in Sebastopol at the end of this month to take part in FooCamp. One of my main aims is to seek out the ingredients required for a successful professional online community. In September I'll be putting in some time at EuroOSCON. Both visits will let me hook up with friends, many of whom I've not seen for a couple of years.
5. I'm on the programme committee for O'Reilly's Emerging Technologies 2007 conferenceThis one has me interested and engaged. Rael Dornfest is due to unveil the call for participation in a couple of weeks' time, and I'll give my take on the conference then. The show itself is March 26-29, 2007.
6. I have an exciting startup under the covers, in the area of conference and event managementOne of the things I've done a lot of over the last half-decade is organise, coax and bully people into presenting at conferences. I'm pouring what I've learned, and more besides, into a great product. You'll soon be bored of me talking about it.
7. I'm involved in taking Pharmalicensing into an exciting new phasePharmalicensing is an online exchange for intellectual property in the biotech and pharmaceutical world. I was part of its foundation in 1998, and am still involved. I will be part of a newly energized team of people pushing it forward into 2007. 8. I'm having a bit of a clean-out of projectsAlas, with so much going on, I need to cut out a few existing commitments. One of these things will be Monopod, my Mono-based podcast client for Linux. If I can't find anybody interested in its maintenance, I'll just hang the source code out there for anyone to pick up.
9. I want to get more energy put into DOAPOne of my long term projects I definitely don't want to cut out is DOAP, my project to describe software projects in RDF. It's had a slow but sure take-up over the last few years, and is finding its way into some significant projects. However, I do need a bit more energy for it, and one or two collaborators would certainly help. 10. Around Christmas-time, I'm going to become a father to twinsI've still not recovered from the shock. Rachael and I are going to be the proud, stunned, parents of a couple of bright shiny bundles of Human 2.0. Completely awesome, and I've absolutely no idea what to do! For now, we're just concentrating on throwing enough junk out to make room for them.

12 August 2006

Lars Wirzenius: Debian Inquirer: BTS almost out of bug numbers, sources reveal

Debian development expected to halt! In a shocking revelation our reporter exposes a conspiracy to cover up the biggest risk to Debian development, ever! #debian-devel, the secret hangout of Debian movers and shakers, today casually discussed the possibility of the Debian bug tracking system running out of new bug numbers. This was the first hint of the impeding doom. When word leaked to our investigative reporter, he quickly dug into the case. Despite public protestations and secret threats, the truth was soon outed! Debian users will soon be unable to report new bugs. The bug tracking system uses a limited, fixed size integer for doing calculations on bug numbers, and the project is soon reaching its limits. With no new bugs to fix, Debian development can't continue. "We're already at six digits, there's only three more to go", says Lars Wirzenius, the Inquirer's resident technological expert. One wonders where the official annoucement is. Once again the Debian Inquirer delivers the scoop!

11 August 2006

Scott James Remnant: Newest Software

I’m confused by your blog, Lucas There seems to be no point to it; you’re comparing the stable releases of one distribution against the unstable archive of another one with a completely different release schedule. Of course the software is going to be out of date! In theory, there’s only one time when both Ubuntu and Debian are in sync (for main, at least); that’s Ubuntu’s Upstream Version Freeze. You won’t compare edgy because “not many people use Ubuntu development versions”, I would argue that this is not true and that the development community at least have an edgy machine around by now. And I would argue that the same is true of Debian, it’s the development community that use Debian testing or unstable. A sensible comparison can only be performed between the stable releases, and then you find it’s quite boring. There’s a linear map between the versions of software and the date of release (e.g. hoary has later software than sarge which has later software than warty).

Lucas Nussbaum: Ubuntu or Debian, which one has the newest software ?

It is a common belief that Ubuntu provides newer software than Debian. This is of course true when Ubuntu is compared to Debian stable. This is also true for specific software packages, like Gnome. But how does it compare in general ? At the time of the Dapper release, here is how Ubuntu Dapper and Debian Unstable compared wrt source package versions :
In Ubuntu, but not in Debian: 1795
In Debian, but not in Ubuntu: 588
Exact same version: 4361
Same upstream version: 2963
… and same Debian version, but Ubuntu has local changes: 731
… “Debian version” newer in Debian: 2197
… “Debian version” newer in Ubuntu: 35
Different upstream version: 2166
… newer in Debian: 1955
… newer in Ubuntu: 211
So Debian easily wins. But this comparison is unfair, because Debian unstable is a moving target : of course, it’s newer than Ubuntu ! What about Debian testing ?
In Ubuntu, but not in Debian: 2137
In Debian, but not in Ubuntu: 514
Exact same version: 4529
Same upstream version: 2652
… and same Debian version, but Ubuntu has local changes: 775
… newer in Debian: 1816
… newer in Ubuntu: 61
Different upstream version: 1967
… newer in Debian: 1666
… newer in Ubuntu: 301
Yes, Debian still wins. Okay, but Ubuntu Dapper had a particularly long release cycle. What about Ubuntu Breezy, and Debian Testing at the time of the Breezy release ?
In Ubuntu, but not in Debian: 2462
In Debian, but not in Ubuntu: 384
Exact same version: 4586
Same upstream version: 2253
… and same Debian version, but Ubuntu has local changes: 981
… newer in Debian: 1172
… newer in Ubuntu: 100
Different upstream version: 1375
… newer in Debian: 1020
… newer in Ubuntu: 355
Debian testing wins again. Notes: Update: Yes, Scott, I am comparing stable releases on one side, with development versions on the other side. However, I think that it is widely known that Debian testing is usable outside of the Debian development community. I know many people using Debian testing who aren’t Debian developers. And many Ubuntu users, if they weren’t using Ubuntu, but Debian, would probably use Debian testing, not stable. Regarding Ubuntu Edgy, its use is currently strictly limited to the Ubuntu development community, and I don’t know anybody not interested in Ubuntu development running Ubuntu Edgy (that’s also because Ubuntu is released every 6 months, not 18 months, of course). That’s why it’s irrelevant to compare with it.
Also, after a quick poll on IRC, I discovered that many Ubuntu users thought that Ubuntu stable releases had newer software than Debian Testing (that was my exact question). The only point of this blog entry is to prove them wrong. Update 2 : I’ll be on holidays for the next few days, so comments are put in the moderation queue until I come back.

John Goerzen: Whose Distributed VCS Is The Most Distributed?

Lately I have been trying out a number of distributed version control systems (VCS or SCM).

One of my tests was a real problem: I wanted to track the Linux 2.6.16.x kernel tree, apply the Xen patches to it, and pull only specific patches (for the qla2xxx driver) from 2.6.17.x into this local branch. I wanted also to be able to upgrade to 2.6.17.x later (once Xen supports it) and have the version control system properly track which patches I already have.

But before going on, let's establish what it means to be an ideal distributed VCS:


There are also some things that we would generally want:



Evaluation

Let's look at some common VCSs against these criteria. I'll talk about Arch (tla, baz, etc), bzr (bazaar-ng), Darcs, Git, Mercurial (hg), and Subversion (svn) for reference.

1. The fundamental method of collaboration must be a branch
All of the tools pass this test except for svn.

2. Branching should be cheap
Everyone except svn generally does this reasonably well.

The tla interface for Arch had a pretty terrible interface for this, so it took awhile simply due to all the typing involved. That's better these days.

Darcs supports hardlinking of history to other local repositories and will do this automatically by default. Git also supports that, but defaults to not doing it, or you can store a path to look in for changesets that aren't in the current repo. I believe Mercurial also can use hardlinks, though I didn't personally verify that. bzr appears to have some features in this area, but not hardlinks, and the features were too complex (or poorly documented) to learn about quickly.

svn does not support branching across repositories, so doesn't really pass this test. Branches within a repository are not directly supported either, but are conventionally simulated by doing a low-cost copy into a specially-named area in the repository.

3. Merging between branches is intelligent
Arch was one of the early ones to work on this problem. It works reasonably well in most situations, but breaks in spectacular and unintelligble ways in some other situations.

When asked to merge one branch to another, Darcs will simply merge in any patches from the source branch onto the destination which the destination doesn't already have. This goes farther than any of the other systems, which generally store a "head" pointer for each branch that shows how far you've gone. (Arch is closer to darcs here, though ironically bzr is more like the other systems)

Merging between branches in svn is really poor, and has no support for recognizing changesets that have been applied both places, resulting in conflicts in many development models.

4. Inividial changesets should be mergeable without bringing across the whole history
Darcs is really the only one that can do this right. I was really surprised that nobody else could, since it is such a useful and vital feature for me.

Both bzr and git have a cherry-pick mode that simulates this, but really these commands just get a diff from the specific changeset requested, then apply the diff as with patch. So you really get a different changeset committed, which can really complicate your history later -- AND lead to potential conflicts in future merges. bzr works around some of the conflict problems because on a merge, it will silently ignore patches that attempt to perform an operation that has already occured. But that leads to even more confusing results, as the merge of the patch is recorded for a commit that didn't actually merge it. (That could even be a commit that doesn't modify the source.) Sounds like a nightmare for later.

Arch has some support for it, but in my experience, actually using this support tends to get it really confused when you do merges later.

Neither Mercurial nor svn have any support for this at all.

5. Branching preserves full history
git, darcs, and Mercurial get this right. Making a branch from one of these repos will give you full history, including individual diffs and commit logs for each changeset.

Arch and bzr preserve commit logs but not the individual changesets on a new branch. I was particularly surprised at this shortcoming with bzr, but sure enough, a standard bzr merge from a remote branch commited three original changesets into one and did not preserve the individual history on the one commit.

svn doesn't support cross-repo branching at all.

6. Merging preserves full history
Again, darcs, git, and Mercurial get this right (I haven't tested this in Mercurial, so I'm not 100% sure).

Arch and bzr have the same problem of preserving commit logs, but not individual changesets. A merge from one branch to another in Arch or bzr simply commits one big changeset on the target that represents all the changesets pulled in from the source. So you lose the distinctness of each individual changeset. This can result in the uncomfortable situation of being unable to re-create full history without access to dozens of repositories on the 'net.

Subversion has no support for merging across repositories, and its support for merging across simulated local branches isn't all that great, either.

7. It is possible to commit, branch, merge, and work with history offline
Everyone except Subversion does a good job of this.

8. The program is fast enough for general-purpose use
All tools here are probably fast enough for most people's projects. Subversion can be annoying at times because many more svn commands hit the network than those from others.

In my experience, Arch was the slowest. Though it was still fine for most work, it really bogged down with the Linux kernel. bzr was next, somewhere between arch and darcs. bzr commands "felt" sluggish, but I haven't used it enough to really see how it scales.

Darcs is the next. It used to be pretty slow, but has been improving rapidly since 1.0.0 was released. It now scales up to a kernel-sized project very well, and is quite usable and reasonably responsive for such a thing. The two main things that slow it down are very large files (10MB or above) and conflicts during a merge.

Mercurial and git appear to be fastest and pretty similar in performance.

All of these tools perform best with periodic manual (or scheduled cron jobs) intervention -- once a month to once a year, depending on your project's size. Arch users have typically created a new repo each year. Darcs users periodically tag things (if things are tagged as part of normal work, no extra work is needed here) and can create checkpoints to speed checkouts over the net. git and Mercurial also use a form of checkpoints. (not sure about bzr)

Subversion works so differently from the others that it's hard to compare. (For one, a checkout doesn't bring down any history.)

Conclusions
I was surprised by a few things.

First, that only one system actually got #4 (merging individual changesets) right. Second, that if you had to pick losers among VCSs, it seems to be Arch and bzr -- the lack of history in branching and merging is a really big issue, and they don't seem to have any compelling features that git, darcs, or Mercurial lack. #4 was a unique feature to Darcs a few years ago, but I figured it surely would have been cloned by all the other new VCS projects that have popped up since. It seems that people have realized it is important, and have added token workaround support for it, but not real working support.

On the other hand, it was interesting to see how VCS projects have copied from each other. Everyone (except tla) seems to use a command-line syntax similar to CVS. The influence of tla Arch is, of course, plainly visible in baz and bzr, but you can also see pieces of it in all the other projects. I was also interested to see the Darcs notion of patch dependencies was visible (albeit in a more limited fashion) in bzr, git, and Mercurial.

So, I will be staying with Darcs. It seems to really take the idea of distributed VCS and run with it. Nobody else seems to have quite gotten the merging thing right yet -- and if you are going to make it difficult to do anything but merge everything up to point x from someone's branch, I just don't see how your tool is as useful as Darcs. But I am glad to see ideas from different projects percolating across and getting reused -- this is certainly good for the community.

Updates / Corrections
I got an e-mail explaining how to get the individual patch diffs out of bzr. This will work only for "regular", non-cherry-picked merges, and requires some manual effort.

You'll need to run bzr log, and find the patch IDs (these are the long hex numbers on the "merged:" line) of the changeset you're interested in, plus the changeset immediately before it on the same branch (which may not be on the same patch and may not be obvious at all on busy projects.) Then, run bzr diff -r revid:old-revid-string..new-revid-string.

I think this procedure really stinks, though, since it requires people to manually find previous commits from the same branch in the log.

10 August 2006

Martin F. Krafft: Through with XFS

I am through with XFS, once and for all. Well, at least for laptops. I still think it's a good filesystem when you can ensure that the power never goes, and your hardware is reliable, but it's just not adequate for laptops or even desktops. I ran into some serious problems a while ago, but managed to recover. Two nights ago, however, three XFS filesystems on my laptop decided to blow up and left my system thoroughly broken. I guess as the hibernate maintainer, I should really start doing my tests somewhere else than my main system... It all started out with a dist-upgrade and this output:
dpkg: error processing /var/cache/apt/archives/dpkg_1.13.22_i386.deb
(--unpack): unable to make backup link of
'./usr/share/man/man1/dpkg-deb.1.gz' before installing new version: Unknown
error 990
Looking at /usr/share/man/man1, I started to anticipate the apocalypse:
# ls -l /usr/share/man/man1
total 7956
?????????? ? ? ?    ? ? ? 7zr.1.gz
?????????? ? ? ?    ? ? ? 822-date.1.gz
?????????? ? ? ?    ? ? ? CA.pl.1ssl.gz
?????????? ? ? ?    ? ? ? Defoma::Common.1.gz
So I look at the log, and amidst kernel oops notices, there's this lovely cookie:
Filesystem "hda6": Corruption of in-memory data detected.  Shutting down
filesystem: hda6
Filesystem hda6 is /usr, so at that time I figured "it could have been worse", booted to single user, and remade the filesystem with the intention to simply reinstall all packages... when I found /var/lib/dpkg/info to be in similar condition. The rest of /var seemed fine, but I resolved then that there was no hope in reviving this system. Fortunately I brought an external drive that had just enough free space to hold my /home and some other stuff, but since USB is really slow when it comes to shifting large amounts of data, I decided to do something productive in the mean time and to answer some outstanding mails. It wasn't difficult to get SSH back up, so I started to work on a remote machine and used the time efficiently. Some time later, though, I got confused in the mist of screen sessions and was browsing my home directory on the laptop, thinking I was elsewhere (my home directories are mostly synchronised), when I noticed a directory in similar condition as the above. Oh shit. Imagine my pain and fear as I first thought my remote machine was also dying, imagine the sigh when I found out I was on the local filesystem, and imagine the shock when I realised that /home was also affected by the XFS breakage... A glance around /var confirmed that the XFS breakage was actually spreading and had now affected three filesystems on this machine. Fortunately, by that time, I had copied everything to the external drive, and decided to put my laptop and myself to sleep. I woke the next morning to the task of reinstalling the thing and decided to be optimistic about it. After all, a reinstall would mean I could finally try partman-crypto and encrypt my laptop's data to protect against leaking sensitive stuff in the case of loss or theft of my laptop. The installation was not as painless as I had hoped, but that was mainly because I ran into a known problem with the graphical installer and partman-crypto, which does not allow to set up volumes with random encryption keys (e.g. swap; see the forthcoming announcement for the beta3 release of the installer), and a bunch of smaller bugs. I had to restart the installation with the traditional frontend to get what I wanted, but other than that, I was very impressed with what our installer development team has accomplished! And a special round of gratitude to Frans Pop for not losing his patience while helping me on several occasions throughout the process. Now, 24 hours after the incident, I am back to normal with a fresh laptop and no data lost (except for one directory which I pulled from a mirrored remote machine; it had no local modifications (so why did XFS screw it up anyway?)). The fonts are all jaggy, so there's something I have to figure out. All things considered, I am sad to have lost 24 hours, but I can also relax more now, without fear of further XFS breakage or loss of private data. Update: Oh, and despite this, I did choose ext3 for all my laptop's filesystems. JFS was really cumbersome and slow last time I tried it, and I surely would not touch RazorFS after experiencing serious data loss on numerous occasions. Update: Two responses so far. Full ack for Julien (except for him laughing at me), Ingo's post warrants a reply though. First, ext3 is also journaled, and if you're about to say "yeah, but it's a hack on top of ext2, well... ext2 is damn mature, and journaling isn't really rocket science, so that "hack" isn't going to be too complicated. In fact, I like the idea of journaling being an option rather than a built-in feature. Second, of course you're supposed to keep backups. But since you keep backups, my top requirement of a filesystem is not "how to get the data back", but "how to ensure it does not break. If it breaks, I can reinstall and restore from backup, but that's a certain amount of time lost. If it doesn't break, well, that's like stealing a little something back from death then, isn't it? Third, I do follow the linux-xfs mailing list, but so what? I did not have write cache enabled, and I was running the 2.6.17.7 kernel at the time of the mishap. Lastly, you point to "excellent tools" to recover the filesystem. I am not sure how excellent xfs_repair really is when it reports "bad magic number 0x0 on dir inode 4696727" during the run, claims to have fixed it, I mount the filesystem, unmount it, run xfs_repair again and get the same message. No filesystem is perfect, and as we know from Biella's problems (among many others, ext3 is no exception. But we did get her data off! So then it's really an open field again, crap filesystem against crap filesystem. I guess at this point it helps to know that ext3 actually follows VFS semantics, while on XFS, a completed sync() syscall does not actually mean it has written the data to disk (see e.g. #317479). And then there are bugs like #239111... ext3 it is for now. If that let's me down, I'll try JFS. If that fails and noone has actually implemented a proper filesystem, I might have a go myself. Haha. Update: Alceste Scalas adds:
Ingo is right when he says that every filesystem has bugs --- but bugs apart, the design of Ext3 (i.e. its physical-block journaling) makes it a far more reliable choice for desktop and laptop PCs, expecially for people without an UPS. An Ext3 filesystem could only crash because of a bug or an hardware failure, while an XFS filesystem can be trashed even without bugs or hardware failures, due to the unavoidable consequences of a power loss on PC-class hardware.
He also alerted me to this mailing list post, which compares data=ordered journaling of ext3 (which almost noone does for performance reasons) with XFS and RazorFS. Update: You may also be interested in this post. Update: Otavio Salvador points me to this FAQ entry, which SGI must have added very lately. It explains how to deal with the directory corruption that was part of my problem. I guess I would have liked to know earlier, but I consider the outcome with dm-crypt + ext3 a win anyhow.

Bernhard R. Link: Graphic Libraries

Wouter Verhelst asked why simple games are so slow nowadays.

I think the problems are in the libs. All this modern stuff tries to become more and more modern, and get more and more stuff out of all those new render extensions, direct graphics and hardware accelerations. There simply is no way to decide which way is faster, so libraries have to guess. So it is no suprise things go wrong. And the place they go wrong are of course not the fast computers, but the older stuff, that does not has those nifty accelerations and no fast CPU to cancel it out.

Another disadvantage are all those "portable" libaries. SDL for example needs three connections to the X server before it does anything. Three times etablishing a connection, checking of security cookies, and so on. Its API looks like living with windows, or never intended to be use for anything other than full-screen mode. (You want to find out how large the window is? Why should you be able to when you said the window cannot be resized?)

QT likes to use extensions, too. I don't know if it is its fault or newer X servers, but the newer your installation gets, the slower 2D games using QT can become. (Note the can, if you have the right graphics card, lucky you, if you have the wrong one, bad luck). To be fair QT is not supposed or designed for 2D games. On the other hand I don't know what it is supposed to do other than being a C++ compiler benchmark measured in hours.

GTK was such a promising design. Object orientated (widget classes are one of those very few things were object orientated can be used with more advantages than disadvantages) but still plain C, small and looking like designed for X. To be fair, I do not even know how well it performs, as the ever increasing library cancer drives me away. From the "users should not be able to change their homedir, that would be far too much the Unix way" glib, over all this miriad of differt little libraries, all moving all the time, spewing their headers in so many different directories a compiler invocation folds three time around your terminal.

Well, enough ranted. My next graphical program will use Athena Widgets. I only have to hope all this reanimated X development in the last time will not pull xlib away from under out feets in the future...

9 August 2006

Sergio Talens-Oliag: Moved to ikiwiki

After some time thinking about it I've moved my blog from PyBloxsom to ikiwiki; I'm still playing with it and the site needs some work, but it seems to work ok and the default look and feel is good enough for me. This post is written using Markdown to give it a try, but I wrote a simple plugin to add (partial) support for rst to be able to keep all my old entries without additional editing. I've sent the file to joeyh and I hope it gets into the upstream code (improved, if possible), as that will remove the need to keep my local changes in sync with the debian package. I have not enabled the ikiwiki.cgi because the machine I'm running it is quite slow and anyway I only need it for comments, and I usually don't have many of them. Instead of enabling comments I've added a footer asking people to send comments to echo -n klgoaca@@@eapafwl.fwl tr -s @a-z@ @i-za-h@ instead. To avoid flooding planet I've used the ikiwiki flood prevention mechanism, previous posts can be reached through the categories and/or using the full Blog index. Update I've been told that there is a Perl library to parse rst. I knew someone was working on it, but have not seen code until now; the package is called Text-Restructured and it is available from the URL http://search.cpan.org/dist/Text-Restructured/. Probably my hack could be replaced by code that uses this module instead of calling python to parse the files... any volunteers?

8 August 2006

Aigars Mahinovs: SBackup revitalising - I18N

The development of SBackup was very stale for most of this year, mostly because many of the bug/feature that user were requesting required a significant rewrite of the codebase and at the same time I saw how messy the code has become over time. A few people came up and offered some help, but out of that only new icons and a command-line parameter parser were fully developed - other developers just took a task or a an idea and disappeared. However this week my trust in Open Source development model was reinforced by Jonh Wendell. He approached me yesterday about localising SBackup to Portuguese and was also willing to add the i18n support that was needed for that. I gave him all the details he needed and forgot about it. But today he came back with working code and a translation. That really inspired me to resume working on improving SBackup one little it at a time. Expect a new release version 0.10 soon with i18n and a few long awaited bugfixes. Also, if you want to translate SBackup to your language - use this file as your template. There are only 75 strings to translate. P.S. One happy user of SBackup donated some of his time cleaning up SBackup wiki from spam. Thank you for that! Now I only need a way to prevent the spam from coming back there :(

Martin Zobel-Helas: mostly done

The moderation queue of proposed-updates is now down to four packages, with one waiting to be accepted by ftpmaster. As soon as the Debian Installer is now build on the still missing architectures (alpha, arm and m68k), we should be ready to release Sarge r3 realy soon. Update: two packages are now waiting for ftpmasters, thanks to dannf, who’s first accept this was.

Next.

Previous.