Search Results: "will"

10 October 2007

Russell Coker: Dreamhost and the DMCA

Dreamhost have refused my request (under the DMCA) to be correctly identified as the author of content copied from my blog. I am publishing this so that anyone else who deals with them will know what to expect. Also if someone wishes to sue Dreamhost in regard to content that they host this may help demonstrate a pattern of behaviour. The situation is quite obviously the result of a broken script used by a splogger that doesn’t correctly match author names with articles. The fact that the official Dreamhost policy is to disregard the requirement that the author(s) of copyright material be correctly identified is reprehensible. It also seems likely to open them to the risk of legal action. If you know how to contact a director of Dreamhost then please give them a link to this post and explain the risks to them. For anyone who wants the detail the messages are below.
(more…)

Martin F. Krafft: Packaging with Git

Introduction I gave a joint presentation with Manoj at Debconf7 about using distributed version control for Debian packaging, and I volunteered to do an on-line workshop about using Git for the task, so it's about time that I should know how to use Git for Debian packaging, but it turns out that I don't. Or well, didn't. After I made a pretty good mess out of the mdadm packaging repository (which is not a big problem as it's just ugly history up to the point when I start to get it right), I decided to get down with the topic and figure it out once and for all. I am writing this post as I put the pieces together. It's been cooking for a week, simply so I could gather enough feedback. I am aware that Git is not exactly a showcase of usability, so I took some extra care to not add to the confusion. It may be the first post in a series, because this time, I am just covering the case of mdadm, for which upstream also uses Git and where I am the only maintainer, and I shall pretend that I am importing mdadm to version control for the first time, so there won't be any history juggling. Future posts could well include tracking Subversion repositories with git-svn, and importing packages previously tracked therewith, but this ain't no promise! (well, that last post is already being drafted, but far from finished; you have been warned!) I realise that git-buildpackage exists, but imposes a rather strict branch layout and tagging scheme, which I don't want to adhere to. And gitpkg (Romain blogged about it recently), deserves another look since, according to its author, it does not impose anything on its user. But in any case, before using such tools (and possibly extending them to allow for other layouts), I'd really rather have done it by hand a couple of times to get the hang of it and find out where the culprits lie. Now, enough of the talking, just one last thing: I expect this blog post to change quite a bit as I get feedback. Changes shall be highlighted in bold typeface.
Setting up the infrastructure First, we prepare a shared repository on git.debian.org for later use (using collab-maint for illustration purposes), download the Debian source package we want to import (version 2.6.3+200709292116+4450e59-3 at time of writing, but I pretend it's -2 because we shall create -3 further down ), set up a local repository, and link it to the remote repository. Note that there are other ways to set up the infrastructure, but this happens to be the one I prefer, even though it's slightly more complicated:
$ ssh alioth
$ cd /git/collab-maint
$ ./setup-repository pkg-mdadm mdadm Debian packaging
$ exit
$ apt-get source --download-only mdadm
$ mkdir mdadm && cd mdadm
$ git init
$ git remote add origin ssh://git.debian.org/git/collab-maint/pkg-mdadm
$ git config branch.master.merge refs/heads/master
Now we can use git-pull and git-push, except the remote repository is empty and we can't pull from there yet. We'll save that for later. Instead, we tell the repository about upstream's Git repository. I am giving you the git.debian.org URL though, simply because I don't want upstream repository (which lives on an ADSL line) hammered in response to this blog post:
$ git remote add upstream-repo git://git.debian.org/git/pkg-mdadm/mdadm
Since we're using the upstream branch of the pkg-mdadm repository as source (and don't want all the other mess I created in that repository), we'll first limit the set of branches to be fetched (I could have used the -t option in the above git-remote command, but I prefer to make it explicit that we're doing things slightly differently to protect upstream's ADSL line).
$ git config remote.upstream-repo.fetch \
    +refs/heads/upstream:refs/remotes/upstream-repo/upstream
And now we can pull down upstream's history and create a local branch off it. The "no common commits" warning can be safely ignored since we don't have any commits at all at that point (so there can't be any in common between the local and remote repository), but we know what we're doing, even to the point that we can forcefully give birth to a branch, which is because we do not have a HEAD commit yet (our repository is still empty):
$ git fetch upstream-repo
warning: no common commits
[ ]
  # in the real world, we'd be branching off upstream-repo/master
$ git checkout -b upstream upstream-repo/upstream
warning: You appear to be on a branch yet to be born.
warning: Forcing checkout of upstream-repo/upstream.
Branch upstream set up to track remote branch
  refs/remotes/upstream-repo/upstream.
$ git branch
* upstream
$ ls   wc -l
77
Importing the Debian package Now it's time to import Debian's diff.gz remember how I pretend to use version control for package maintenance for the first time. Oh, and sorry about the messy file names, but I decided it's best to stick with real data in case you are playing along: Since we're applying the diff against version 2.6.3+200709292116+4450e59, we ought to make sure to have the repository at the same state. Upstream never "released" that version, but I encoded the commit ID of the tip when I snapshotted it: 4450e59, so we branch off there. Since we are actually tracking the git.debian.org pkg-mdadm repository instead of upstream, you can use the tag I made. Otherwise you could consider tagging yourself:
$ #git tag -s mdadm-2.6.3+200709292116+4450e59 4450e59
$ git checkout -b master mdadm-2.6.3+200709292116+4450e59
$ zcat ../mdadm_2.6.3+200709292116+4450e59-2.diff.gz   git apply
The local tree is now "debianised", but Git does not know about the new and changed files, which you can verify with git-status. We will split the changes made by Debian's diff.gz across several branches.
The idea of feature branches We could just create a debian branch, commit all changes made by the diff.gz there, and be done with it. However, we might want to keep certain aspects of Debianisation separate, and the way to do that is with feature branches (also known as "topic" branches). For the sake of this demonstration, let's create the following four branches in addition to the master branch, which holds the standard Debian files, such as debian/changelog, debian/control, and debian/rules:
  • upstream-patches will includes patches against the upstream code, which I submit for upstream inclusion.
  • deb/conffile-location makes /etc/mdadm/mdadm.conf the default over /etc/mdadm.conf and is Debian-specific (thus the deb/ prefix).
  • deb/initramfs includes the initramfs hook and script, which I want to treat separately but not submit upstream.
  • deb/docs similarly includes Debian-only documentation I add to the package as a service to Debian users.
If you're importing a Debian package using dpatch, you might want to convert every dpatch into a single branch, or at least collect logical units into separate branches. Up to you. For now, our simple example suffices. Keep in mind that it's easy to merge two branch and less trivial to split one into two. Why? Well, good question. As you will see further down, the separation between master and deb/initramfs actually makes things more complicated when you are working on an issue spanning across both. However, feature branches also bring a whole lot of flexibility. For instance, with the above separation, I could easily create mdadm packages without initramfs integration (see #434934), a disk-space-conscious distribution like grml might prefer to leave out the extra documentation, and maybe another derivative doesn't like the fact that the configuration file is in a different place from upstream. With feature branches, all these issues could be easily addressed by leaving out unwanted branches from the merge into the integration/build branch (see further down). Whether you use feature branches, and how many, or whether you'd like to only separate upstream and Debian stuff is entirely up to you. For the purpose of demonstration, I'll go the more complicated way.
Setting up feature branches So let's commit the individual files to the branches. The output of the git-checkout command shows modified files that have not been committed yet (which I trim after the first example); Git keeps these across checkouts/branch changes. Note that the ./debian/ directory does not show up as Git does not know about it yet (git-status will tell you that it's untracked, or rather: contains untracked files since Git does not track directories at all):
$ git checkout -b upstream-patches mdadm-2.6.3+200709292116+4450e59
M Makefile
M ReadMe.c
M mdadm.8
M mdadm.conf.5
M mdassemble.8
M super1.c
$ git add super1.c     #444682
$ git commit -s
  # i now branch off master, but that's the same as 4450e59 actually
  # i just do it so i can make this point 
$ git checkout -b deb/conffile-location master
$ git add Makefile ReadMe.c mdadm.8 mdadm.conf.5 mdassemble.8
$ git commit -s
$ git checkout -b deb/initramfs master
$ git add debian/initramfs/*
$ git commit -s
$ git checkout -b deb/docs master
$ git add RAID5_versus_RAID10.txt md.txt rootraiddoc.97.html
$ git commit -s
  # and finally, the ./debian/ directory:
$ git checkout master
$ chmod +x debian/rules
$ git add debian
$ git commit -s
$ git branch
  deb/conffile-location
  deb/docs
* master
  upstream
  upstream-patches
At this time, we push our work so it won't get lost if, at this moment, aliens land on the house, or any other completely plausible event of apocalypse descends upon you. We'll push our work to git.debian.org (the origin, which is the default destination and thus needs not be specified) by using git-push --all, which conveniently pushes all local branches, thus including the upstream code; you may not want to push the upstream code, but I prefer it since it makes it easier to work with the repository, and since most of the objects are needed for the other branches anyway after all, we branched off the upstream branch. Specifying --tags instead of --all pushes tags instead of heads (branches); you couldn't have guessed that! See this thread if you (rightfully) think that one should be able to do this in a single command (which is not git push refs/heads/* refs/tags/*)
$ git push --all
$ git push --tags
Done. Well, almost
Building the package (theory) Let's build the package. There seem to be two (sensible) ways we could do this, considering that we have to integrate (merge) the branches we just created, before we fire off the building scripts:
  1. by using a temporary (or "throw-away") branch off upstream, where we integrate all the branches we have just created, build the package, tag our master branch (it contains debian/changelog), and remove the temporary branch. When a new package needs to be built, we repeat the process.
  2. by using a long-living integration branch off upstream, into which we merge all our branches, tag the branch, and build the package off the tag. When a new package comes around, we re-merge our branches, tag, and build.
Both approaches have a certain appeal to me, but I settled for the second, for two reasons, the first of which leads to the second:
  1. When I upload a package to the Debian archive, I want to create a tag which captures the exact state of the tree from which the package was built, for posterity (I will return to this point later). Since the throw-away branches are not designed to persist and are not uploaded to the archive, tagging the merging commit makes no sense. Thus, the only way to properly identify a source tree across all involved branches would be to run git-tag $branch/$tagname $branch for each branch, which is purely semantic and will get messy sooner or later.
  2. As a result of the above: when Debian makes a new stable release, I would like to create a branch corresponding to the package in the stable archive at the time, for security and other proposed updates. I could rename my throw-away branch, if it still existed, or I could create a new branch and merge all other branches, using the (semantic) tags, but that seems rather unfavourable.
So instead, I use a long-living integration branch, notoriously tag the merge commits which produced the tree from which I built the package I uploaded, and when a certain version ends up in a stable Debian release, I create a maintenance branch off the one, single tag which corresponds to the very version of the package distributed as part of the Debian release. So much for the theory. Let's build, already!
Building the package (practise) So we need a long-living integration branch, and that's easier done than said:
$ git checkout -b build mdadm-2.6.3+200709292116+4450e59
Now we're ready to build, and the following procedure should really be automated. I thus write it like a script, called poor-mans-gitbuild, which takes as optional argument the name of the (upstream) tag to use, defaulting to upstream (the tip):
#!/bin/sh
set -eu
git checkout master
debver=$(dpkg-parsechangelog   sed -ne 's,Version: ,,p')
git checkout build
git merge $ 1:-upstream 
git merge upstream-patches
git merge master
for b in $(git for-each-ref --format='%(refname)' refs/heads/deb/*); do
  git merge -- $b
done
git tag -s debian/$debver
debuild   # will ignore .git automatically
git checkout master
Note how we are merging each branch in turn, instead of using the octopus merge strategy (which would create a commit with more than two parents) for reasons outlined in this post. An octopus-merge would actually work in our situation, but it will not always work, so better safe than sorry (although you could still achieve the same result). If you discover during the build that you forgot something, or the build script failed to run, just remove the tag, undo the merges, checkout the branch to which you need to commit to fix the issue, and then repeat the above build process:
$ git tag -d debian/$debver
$ git checkout build
$ git reset --hard upstream
$ git checkout master
$ editor debian/rules    # or whatever
$ git add debian/rules
$ git commit -s
$ poor-mans-gitbuild
Before you upload, it's a good idea to invoke gitk --all and verify that all goes according to plan:
screenshot of gitk after the above steps
When you're done and the package has been uploaded, push your work to git.debian.org, as before. Instead of using --all and --tags, I now specify exactly which refs to push. This is probably a good habit to get into to prevent publishing unwanted refs:
$ git push origin build tag debian/2.6.3+200709292116+4450e59-3
Now take your dog for a walk, or play outside, or do something else not involving a computer or entertainment device.
Uploading a new Debian version If you are as lucky as I am, the package you uploaded still has a bug in the upstream code and someone else fixes it before upstream releases a new version, then you might be in the position to release a new Debian version. Or maybe you just need to make some Debian-specific changes against the same upstream version. I'll let the commands speak for themselves:
$ git checkout upstream-patches
$ git-apply < patch-from-lunar.diff   #444682 again
$ git commit --author 'J r my Bobbio <lunar@debian.org>' -s
  # this should also be automated, see below
$ git checkout master
$ dch -i
$ dpkg-parsechangelog   sed -ne 's,Version: ,,p'
2.6.3+200709292116+4450e59-3
$ git commit -s debian/changelog
$ poor-mans-gitbuild
$ git push
$ git push origin tag debian/2.6.3+200709292116+4450e59-3
That first git-push may require a short explanation: without any arguments, git-push updates only the intersection of local and remote branches, so it would never push a new local branch (such as build above), but it updates all existing ones; thus, you cannot inadvertedly publish a local branch. Tags still need to be published explicitly.
Hacking on the software Imagine: on a rainy Saturday afternoon you get bored and decide to implement a better way to tell mdadm when to start which array. Since you're a genius, it'll take you only a day, but you do make mistakes here and there, so what could be better than to use version control? However, rather than having a branch that will live forever, you are just creating a local branch, which you will not publish. When you are done, you'll feed your work back into the existing branches. Git makes branching really easy and as you may have spotted, the poor-mans-gitbuild script reserves an entire branch namespace for people like you:
$ git checkout -b tmp/start-arrays-rework master
Unfortunately (or fortunately), fixing this issue will require work on two branches, since the initramfs script and hook are maintained in a separate branch. There are (again) two ways in which we can (sensibly) approach this:
  • create two separate, temporary branches, and switch between them as you work.
  • merge both into the temporary branch and later cherry-pick the commits into the appropriate branches.
I am undecided on this, but maybe the best would be a combination: merge both into a temporary branch and later cherry-pick the commits into two additional, temporary branches until you got it right, and then fast-forward the official branches to their tips:
$ git merge master deb/initramfs
$ editor debian/mdadm-raid                     #  
$ git commit -s debian/mdadm-raid
$ editor debian/initramfs/script.local-top     #  
$ git commit -s debian/initramfs/script.local-top
[many hours of iteration pass ]
[  until you are done]
$ git checkout -b tmp/start-arrays-rework-init master
  # for each commit $c in tmp/start-arrays-rework
  # applicable to the master branch:
$ git cherry-pick $c
$ git checkout -b tmp/start-arrays-rework-initramfs deb/initramfs
  # for each commit $c in tmp/start-arrays-rework
  # applicable to the deb/initramfs branch:
$ git cherry-pick $c
This is assuming that all your commits are logical units. If you find several commits which would better be bundled together into a single commit, this is the time to do it:
$ git cherry-pick --no-commit <commit7>
$ git cherry-pick --no-commit <commit4>
$ git cherry-pick --no-commit <commit5>
$ git commit -s
Before we now merge this into the official branches, let me briefly intervene and introduce the concept of a fast-forward. Git will "fast-forward" a branch to a new tip if it decides that no merge is needed. In the above example, we branched a temporary branch (T) off the tip of an official branch (O) and then worked on the temporary one. If we now merge the temporary one into the official one, Git determines that it can actually squash the ancestry into a single line and push the official branch tip to the same ref as the temporary branch tip. In cheap (poor man's), ASCII notation:
- - - O             >> merge T >>     - - - = - - OT
         - - T      >>  into O >>
This works because no new commits have been made on top of O (if there would be any, we might be able to rebase, but let's not go there quite yet; rebasing is how you shoot yourself in the foot with Git). Thus we can simply do the following:
$ git checkout deb/initramfs
$ git merge tmp/start-arrays-rework-initramfs
$ git checkout master
$ git merge tmp/start-arrays-rework-init
and test/build/push the result. Or well, since you are not an mdadm maintainer (We^W I have open job positions! Applications welcome!), you'll want to submit your work as patches via email:
$ git format-patch -s -M origin/master
This will create a number of files in the current directory, one corresponding for each commit you made since origin/master. Assuming each commit is a logical unit, you can now submit these to an email address. The --compose option lets you write an introductory message, which is optional:
$ git send-email --compose --to your@email.address <file1> <file2> < >
Once you've verified that everything is alright, swap your email address for the bug number (or the pkg-mdadm-devel list address). Thanks (in advance) for your contribution! Of course, you may also be working on a feature that you want to go upstream, in which case you'd probably branch off upstream-patches (if it depends on a patch not yet in upstream's repository), or upstream (if it does not):
$ git checkout -b tmp/cool-feature upstream
[ ]
when a new upstream version comes around After a while, upstream may have integrated your patches, in addition to various other changes, to give birth to mdadm-2.6.4. We thus first fetch all the new refs and merge them into our upstream branch:
$ git fetch upstream-repo
$ git checkout upstream
$ git merge upstream-repo/master
we could just as well have executed git-pull, which with the default configuration would have done the same; however, I prefer to separate the process into fetching and merging. Now comes the point when many Git people think about rebasing. And in fact, rebasing is exactly what you should be doing, iff you're still working on an unpublished branch, such as the previous tmp/cool-feature off upstream. By rebasing your branch onto the updated upstream branch, you are making sure that your patch will apply cleanly when upstream tries it, because potential merge conflicts would be handled by you as part of the rebase, rather than by upstream:
$ git checkout tmp/cool-feature
$ git rebase upstream
What rebasing does is quite simple actually: it takes every commit you made since you branched off the parent branch and records the diff and commit message. Then, for each diff/commit_message pair, it creates a new commit on top of the new parent branch tip, thus rewrites history, and orphans all your original commits. Thus, you should only do this if your branch has never been published or else you would leave people who cloned from your published branch with orphans.
If this still does not make sense, try it out: create a (source) repository, make a commit (with a meaningful commit message), branch B off the tip, make a commit on top of B (with a meaningful message), clone that repository and return to the source repository. There, checkout the master, make a commit (with a ), checkout B, rebase it onto the tip of master, make a commit (with a ), and now git-pull from the clone; use gitk to figure out what's going on.
So you should almost never rebase a published branch, and since all your branches outside of the tmp/* namespace are published on git.debian.org, you should not rebase those. But then again, Pierre actually rebases a published branch in his workflow, and he does so with reason: his patches branch is just a collection of branches to go upstream, from which upstream cherry-picks or which upstream merges, but which no one tracks (or should be tracking). But we can't (or at least will not at this point) do this for our feature branches (though we could treat upstream-patches that way), so we have to merge. At first, it suffices to merge the new upstream into the long-living build branch, and to call poor-mans-gitbuild, but if you run into merge conflicts or find that upstream's changes affect the functionality contained in your feature branches, you need to actually fix those. For instance, let's say that upstream started providing md.txt (which I previously provided in the deb/docs branch), then I need to fix that branch:
$ git checkout deb/docs
$ git rm md.txt
$ git commit -s
That was easy, since I could evade the conflict. But what if upstream made a change to Makefile, which got in the way with my configuration file location change? Then I'd have to merge upstream into deb/conffile-location, resolve the conflicts, and commit the change:
$ git checkout deb/conffile-location
$ git merge upstream
CONFLICT!
$ git-mergetool
$ git commit -s
When all conflicts have been resolved, I can prepare a new release, as before:
$ git checkout master
$ dch -i
$ dpkg-parsechangelog   sed -ne 's,Version: ,,p'
2.6.3+200709292116+4450e59-3
# git commit -s debian/changelog
$ poor-mans-gitbuild
# git push
$ git push origin tag debian/2.6.3+200709292116+4450e59-3
Note that Git often appears smart about commits that percolated upstream: since upstream included the two commits in upstream-patches in his 2.6.4 release, my upstream-patches branch got effectively annihilated, and Git was smart enough to figure that out without a conflict. But before you rejoice, let it be told that this does not always work.
Creating and using a maintenance branch Let's say Debian "lenny" is released with mdadm 2.7.6-1, then:
$ git checkout -b maint/lenny debian/2.7.6-1
You might do this to celebrate the release, or you may wait until the need arises. We've already left the domain of reality ("lenny" is not yet released), so the following is just theory. Now, assume that a security bug is found in mdadm 2.7.6 after "lenny" was released. Upstream is already on mdadm 2.7.8 and commits deadbeef and c0ffee fix the security issue, then you'd cherry-pick them into the maint/lenny branch:
$ git checkout upstream
$ git pull
$ git checkout maint/lenny
$ git cherry-pick deadbeef
$ git cherry-pick c0ffee
If there are no merge conflicts (which you'd resolve with git-mergetool), we can just go ahead to prepare the new package:
$ dch -i
$ dpkg-parsechangelog   sed -ne 's,Version: ,,p'
2.7.6-1lenny1
$ git commit -s debian/changelog
$ poor-mans-gitbuild
$ git push origin maint/lenny
$ git push origin tag debian/2.7.6-1lenny1
Future directions It should be trivial to create the Debian source package directly from the repository, and in fact, in response to a recent blog post of mine on the dispensability of pristine upstream tarballs, two people showed me their scripts to do it. My post also caused Joey Hess to clarify his position on pristine tarballs, before he went out to implement dpkg-source v3. This looks very promising. Yet, as Romain argues, there are benefits with simple patch management systems. Exciting times ahead! In addition to creating source packages from version control, a couple of other ideas have been around for a while:
  • create debian/changelog from commit log summaries when you merge into the build branch.
  • integrate version control with the BTS, bidirectionally:
    • given a bug report, create a temporary branch and apply any patches found in the bug report.
    • upon merging the temporary branch back into the feature branch it modifies, generate a patch, send it to the BTS and tag the bug report + pending patch.
And I am sure there are more. If you have any, I'd be interested to hear about them!
Wrapping up I hope this post was useful. Thank you for reading to the end, this was probably my longest blog post ever. I want to thank Pierre Habouzit, Johannes Schindelin, and all the others on the #git/freenode IRC channel for their tutelage. Thanks also to Manoj Srivastava, whose pioneering work on packaging with GNU arch got me started on most of the concepts I use in the above. And of course, the members of the the vcs-pkg mailing list for the various discussions on this subject, especially those who participated in the thread leading up to this post. Finally, thanks to Linus and Junio for Git and the continuously outstanding high level of support they give. If you are interested in the topic of using version control for distro packaging, I invite you to join the vcs-pkg mailing list and/or the #vcs-pkg/irc.oftc.net IRC channel. NP: Aphex Twin: Selected Ambient Works, Volume 2 (at least when I started writing )

MJ Ray: Current Issues / Walking and Cycling in WsM

I'm going to the Weston-super-Mare walking and cycling strategy consultation event this evening.

What messages should I take with me?  What topics should I watch out for?  I'll check for replies in the forum and my email and comment form before I leave (5.20pm, I guess).

For cycling, I think the key points will be: no confusing facilities, better route signs and fewer junctions.  I think walking is probably mostly about keeping paths clear, open and safe.  Have I missed something obvious?

Adam Rosi-Kessel: MBTA Responds

To follow up on yesterday’s MBTA philippic, I received a response from the T’s customer service regarding my complaints. Although it doesn’t give me hope that things will improve, it’s nice that they have at least hired someone to write personalized, polished replies. At least they can do that right:
Dear Mr. Kessel, Thank you for your email concerning service on the Needham Line. The past couple of months have been difficult for our passengers, and I want you to know we are very sorry for the inconveniences this has been causing. The causes of these delays have ranged from signal and mechanical issues to long stops boarding passengers at stations, and even track work on other lines. I know that it may seem odd that trains on other lines can impact your trains, but I assure you that is true. The Franklin, Needham, Providence, and Stoughton lines, as well as Amtrak all share the rail between Hyde Park and Boston. All it takes is one delayed train in this area to cause a cascading effect that delays other lines. There’s also the fact that trains may come in from one line to go out as another. For example, a train on the Worcester Line comes to Boston, and either the train set, or the crew (sometimes both) head out as an outbound Needham train. If the inbound is late from Worcester, then the outbound Needham will be late, which also means that its “turn,” the next inbound, will be delayed. In response to your comment about the doors not all being opened, even with a full compliment of conductors, crews are not able to staff every door, and especially at ground level stations we want to ensure that all customers are able to board and detrain safely. This is why conductors do not open all doors to the train. I contacted our Mechanical Department, asking that the PA system on train 625 be checked out to make sure everything is working properly. Please know that delays in service are not something that we take lightly, and all departments are committed to working together to ensure that our customers receive timely, safe, and reliable service. Once again, I am sorry for the inconvenience this has caused you. Thank you for writing. Sincerely, Linda Dillon
MBCR Customer Service Manager

Clint Adams: I need an admin assistant to glue papers to other papers

buxy is on the wrong track. DSA is a fundamentally-flawed organization; all of the self-selecting committees in Debian are: you cannot have a functional and respectable subgroup if it maintains autonomy like that. The solution to the current set of problems that have been plaguing us for years is not to have backroom discussions and negotiations. It is not to add MORE team members, because that will, at best, achieve minor performance improvements. At worst, that will bolster its structural problems. To fix DSA cleanly and healthily, the honorable members need to resign, and Debian needs to fire the rest. Attempt anything else and you will likely become part of the problem.

Russell Coker: Execmem and SE Linux

Eddy writes about problems getting the game oolite to run under SE Linux [1]. Strangely after I fixed the shared object issue with libffcall1 (as described in my previous post [2]) it appeared to work for me. Eddy asked how to allow one application to create write and executable memory regions without allowing such access for all programs. This can be done, in the targeted policy the type unconfined_execmem_exec_t triggers a transtion to the domain unconfined_execmem_t which permits the execstack and execmem operations. For example if the program /usr/bin/foo needs such access then the command chcon -t unconfined_execmem_exec_t /usr/bin/foo will change the type and the next time the program is launched from an unconfined session (a user login session, cron job, or daemon which is not constrained) then the change will apply.

Cyril Brulebois: Linking on hppa

Problem present in openscenegraph 1.9.x and 2.x series: bug #425837. Looking at the build logs of unstable or experimental for the hppa architecture, there's a linking problem around. Shorten error message:
/usr/bin/g++ -fPIC -shared -Wl,-soname,libosgUtil.so.6 
    -o ../../lib/libosgUtil.so.6.0.0 "many *.o"
    -L/build/buildd/openscenegraph-2.2/build/osg/lib  -losg
    -lOpenThreads -lGLU -lGL -lSM -lICE -lX11 -lXext -lm -lpthread
    -Wl,-rpath,/build/buildd/openscenegraph-2.2/build/osg/lib 
/usr/bin/ld: CMakeFiles/osgUtil.dir/Optimizer.o(.text._ZNSt8_Rb_tree 
     IjSt4pairIKjSt3setIPN3osg9Texture2DESt4lessIS5_ESaIS5_EEESt10_S 
     elect1stISA_ES6_IjESaISA_EE9_M_insertEPSt18_Rb_tree_node_baseSH 
     _RKSA_("unmangled name"): cannot reach 0000393f__ZNSt10_Select1 
     stISt4pairIKjSt3setIPN3osg9Texture2DESt4lessIS5_ESaIS5_EEEEC1Ev+0,
    recompile with -ffunction-sections
/usr/bin/ld: final link failed: Bad value
First, just in case demangling is needed (that's not the case, but imagine), there's a nice tool to get the signature back (output skipped, it is very huge, that C++!). That's c++filt from the binutils package:
c++filt _ZNSt8_Rb_treeIjSt4pairIKjSt3setIPN3osg9Texture2DESt4lessIS\
5_ESaIS5_EEESt10_Select1stISA_ES6_IjESaISA_EE9_M_insertEPSt18_Rb_tr\
ee_node_baseSH_RKSA_
Then, one tends to trust the compiler. Added the following to the cmake command in debian/rules:
-D CMAKE_CXX_CFLAGS='-ffunction-sections'
But not much better. Oh wait, there's a quite interesting string in the build log now: cannot handle R_PARISC_PCREL17F, which looks like quite arch-dependant and perfect to be searched through the internets. Some old bugreports came out: bug #334959 and bug #416496 and the solution: use -mlong-calls. From http://www.projectblackdog.com/3rdParty/gcc/HPPA-Options.html, the relevant parts: Generate code that uses long call sequences. This ensures that a call is always able to reach linker generated stubs. [ ] It is normally not desirable to use this option as it will degrade performance. However, it may be useful in large applications, particularly when partial linking is used to build the application. It looks like it might do the job. And indeed it does. Conditionally added for the hppa architecture. And -ffunction-sections isn't even needed. Thanks to the http://www.parisc-linux.org/ folks for the access to their cluster, which really helped finding out a fix very quickly! And it looks like the number of blockers for the transition to testing has been reduced. \o/

Benjamin Mako Hill: Open Minds

I'm going to be giving the opening keynote presentation at the K-12 Open Minds conference tomorrow. I was supposed to be talking on Thursday but will be filling in for David Thornburg who will be a day late. Apparently, he's been described as one of the top 21 speakers in the United States. Yikes! I'll be talking about the free software movement and hacker culture. I've heard that the whole thing will be streamed.

9 October 2007

Russell Coker: How SE Linux Prevents Local Root Exploits

In a comment on my previous post about SE Linux and worms/trojans [1] a user enquired about which methods of gaining local root are prevented by SE Linux. A local exploit is one that can not be run remotely. An attack via TCP or UDP is generally considered a remote exploit - even though in some cases you might configure a daemon to only bind to localhost (in which case the TCP or UDP attack would only work locally). When compromising a machine it’s not uncommon for a local exploit to be used after a remote exploit or social engineering has been used to gain non-root privileges. The two most common types of local root exploit seem to be those which attack the kernel and those which attack a SETUID process. For a non-SE Linux system it usually doesn’t matter much how the local exploit is run. But a SE Linux system in a default configuration will be running the Targeted policy that has almost no restrictions on a user shell session. So an attacker who wants to escalate their privileges from local user to local root on a typical SE Linux system has a significant benefit in starting from a user account instead of starting from a web server or other system process. In the SE Linux model access is granted to a domain, and the domain which is used for a process is determined by the policy based on the domain of the parent process and the labelling of the executable. Some domains are not permitted to transition to any other domains, such as the domain dhcpd_t (used for a DHCP server). Other domains are only permitted to transition to a small set of domains, for example the domain httpd_t (used for a web server) can only transition to a small number of domains none of which has any significant privileges. On a machine running without SE Linux a compromise of a DHCP server is game-over as the server runs as root. A compromise of a daemon such as Apache on a machine without SE Linux gives unrestricted access to run applications on the systems - if a SETUID-root program has a security flaw then you lose. The same bug in a SETUID program on a machine running SE Linux is not fatal because SE Linux will prevent the program from doing anything that it’s parent could not do - even if an attacker made Apache run a buggy SETUID program the broken program in question could do nothing other than what Apache is normally permitted to do. A security flaw in a SETUID-root program on a SE Linux system can still be exploited by a local user (someone who has logged in) when running the Targeted policy. When running the Strict or MLS policies many such vulnerabilities will not be exploitable by local users (for example exploiting PING would only permit network access). As a rule of thumb you should consider that a kernel security flaw will make it possible to bypass all other security features. However there are some situations where SE Linux can prevent local exploits. One example is a bug discovered in July 2006 which allowed the creation of SETUID files under /proc [2], the targeted policy of SE Linux prevented this from working. Another example is CVE-2003-0127 [3] (a kernel security flaw that was exploited by triggering a module load and then exploiting a race condition in the kernel module load process), the commonly used exploit for this did not work on a SE Linux system because the user process was not permitted the socket access used to trigger the module load (it is believed that an attacker could have written a version of the exploit to work on a SE Linux system - but AFAIK no-one ever did so).

Chris Lamb: Top 10 interactive shell anti-patterns

In general, if you’re doing a repetitive task by hitting lots of keys, you’re doing it wrong. Not only are you wasting time, you’re wearing out your keyboard and damaging your wrists and fingers. Here are the ten most common anti-patterns (actually amelioration patterns) that I commonly see in other people’s line editing:
  1. Repeating a command by retyping it
    Press Ctrl-p and Enter, or “!!“. Zsh users get “r” as well. You can even get the command n times before, use “!-1“. Using the arrow keys is obviously flawed - find out how to unmap them in your shell.
  2. Manually changing back to your previous directory
    Give “cd -” a whirl.
  3. Typing a long command (or sequence of commands) excessively often
    Obviously wrong, but I see it so often it’s worth mentioning. Try using an alias and put it in your shell’s startup file. If the command is parameterised, you could construct a simple function. If you are writing software, look into the build systems available for your language - the basics of GNU Make can be mastered within 20 minutes or so, and have many other benefits too.
  4. Opening another shell when the current process blocks
    (e.g. You realise the command you started is going to take a lot longer than you thought) Pause the current process with Ctrl+z and then put the job in the background with bg. fg will bring it to the foreground again. Fewer shells mean you stay focused on what you were trying to achieve.
  5. Searching history by grep’ing ~/.bash_history, or Up Up Up … etc.
    Use CTRL+r (or “/” in vi-mode) for search-as-you type. You can immediately run the command by pressing Enter.
  6. Adding a prefix to a command with Up Home $command Space
    (e.g. You ran a command that really needed sudo). Try: $prefix !!
  7. Changing a command with Home Right Right Right Right … BkSpace BkSpace … etc.
    (e.g you accidentally ran chown instead of chmod) Try: $new_command !*
  8. Retyping significant parts of the previous line
    $ command arg1 /long/path/to/file
    $ command2 arg2 /long/path/to/file
    $ command3 arg1 arg3 /long/path/to/file
    Try using !$ to get the right-most argument on the previous line (you can also use !:2 to get the second argument, etc.):
    $ command arg1 /long/path/to/file
    $ command2 arg2 !$
    $ command3 arg1 arg3 !$
  9. Using Home/End to move to the beginning/end of the line
    This is how you completely ruin your wrists. Use Ctrl+a and Ctrl+e.
  10. Hitting tens of keys to make a substitition
    (e.g. you made a trivial mistake in the middle of your previous command). Use ^foo^bar. If you forget the other nine, try this one - stop punishing yourself for making mistakes! :)

Ross Burton: Experiments with GUPnP

Now that through black magic, voodoo, and a working network driver I have Avahi working on my NAS and UPnP working on my SoundBridge, I can get back to the task of learning a bit about GUPnP, a GObject-based library for UPnP. My first hack is a simple tool that pops up notification bubbles when the currently playing track changes, and it was surprisingly easy. First, wecreate a Control Point for the service we want to control. This object will emit signals when devices on the network are discovered which provide this service, so by connecting to that signal all of the discovery is handled for us.
GUPnPContext *context = gupnp_context_new (NULL, NULL, 0, &error);
GUPnPControlPoint *cp = gupnp_control_point_new (context, "urn:schemas-upnp-org:service:AVTransport:1");
g_signal_connect (cp, "service-proxy-available",
                  G_CALLBACK (service_proxy_available_cb), NULL);
gssdp_resource_browser_set_active (GSSDP_RESOURCE_BROWSER (cp), TRUE);
Now in service_proxy_available_cb we are passed a Service Proxy object to which represents a device providing the service we asked for. Service proxies can have actions invoked on them, or we can be notified when a State Variable change. There is a state variable in AVTransport called CurrentTrackMetaData but don't let that fool you: for reasons too boring to detail that doesn't cause notifications. The only interesting state variable in AVTransport is LastChange, which is basically a meta-variable which contains the name and value of the last variable which changed. Madness, I agree. This is where we come to the problem with GUPnP: it makes the protocol seem so clean and simple that when warts like this become obvious, they stand out. Anyway, we want to listen for changes to the LastChange state variable. This involves adding a notify and then subscribing to the proxy. Luckily state variables emit their current value when we subscribe to them, so we never need to fetch the current value.
static void
service_proxy_available_cb (GUPnPControlPoint *cp, GUPnPServiceProxy *proxy)
 
    gupnp_service_proxy_add_notify (proxy, "LastChange",
                                    G_TYPE_STRING, notify_cb, NULL);
    gupnp_service_proxy_set_subscribed (proxy, TRUE);
 
Believe it or not, we're actually nearly finished. Now to implement notify_cb. This is passed the variable name and a GValue containing the new value. A basic implementation is pretty simple.
static void
notify_cb (GUPnPServiceProxy *proxy,
           const char        *variable,
           GValue            *value,
           gpointer           user_data)
 
    g_print ("%s changed to %s\n", variable, g_value_get_string (value));
 
And we're done! When the program starts it discovers any devices on the network, creates proxies and watches the variables, printing them as they change. Devices can come and go, GUPnP handles that automatically. Easy as pie. Well, sort of. The string is an XML document which could contain any variable, so we need to parse it looking for a CurrentTrackMetaData node. On that node the val attribute contains an escaped XML document describing the metadata in the (and I kid you not) DIDL-Lite format. The saving grace here is that at least its partially Dublin Core, but that needs to be parsed for the title and artist information. This is all standard XML mojo, and quite boring. The end result is that my first hack application is 150 lines long, the first 100 of which are entirely devoted to XML parsing. The next step is to write a convenience library around DIDL to avoid having to parse it manually. It's not exactly a complicated task, but quite tiresome. For the curious, the full source is in this Bazaar branch. NP: Directions EP, Variou (from Acroplane I think)

Russell Coker: A Long Laptop Lifetime

Paul Russell writes about his 3-yearly laptop replacement at IBM [1]. It probably makes some sense to replace laptops periodically for a large company, but if you are buying for personal use then it makes sense to try and get a longer life out of an expensive machine. I think that aiming for 6 years is quite reasonable with today’s hardware - you should be able to buy a new machine now and have it last 6 years or buy a 3yo second-hand machine and hope to have it last 3 years (most second-hand laptops on sale in every place other than Ebay were trophies for managers and never had any serious use). If you are going to buy a second-hand laptop the first thing to consider is PAE support. If you get a laptop without PAE support (I think that means all Pentium-M CPUs) then you will not have proper Xen support (it seems that all distributions have abandoned Xen support for PAE for the moment). This may not be a big deal if you don’t want Xen, but if you are a programmer then you probably do want Xen (even if you don’t realise it yet). The next issue is support for the AMD64 instruction set. 32bit laptops are going cheap at the moment but if you buy one you will be significantly limited as to what software you can run at some future time (my 32bit laptop is doing well at the moment apart from the lack of PAE support). If you are buying a new laptop then the first thing to consider when planning a long life is the warranty. In my experience most computer gear does not need a long warranty, if it survives 3 months then it’ll probably last until it’s well obsolete. Laptops however periodically wear out if used seriously, I average one warranty replacement of a Thinkpad keyboard every two years and I have had a few motherboard replacements (the lighter Thinkpads flex and they eventually break inside if you use them on enough trains, trams, buses, planes, etc). On one of the Lenovo T series Thinkpads that I saw advertised (one that I would consider if I wanted a new laptop now) there was an offer to spend an extra $350AU to get extend the warrantee from 1 year to 5 years (according to my understanding of the confusing text on the web site) on a laptop that cost $3050. An increase in the purchase price of 12% for the extra warranty is a bargain (I know that for my use they would lose money on the deal). Repair of a laptop is generally very expensive, any serious damage to a laptop that is more than 18 months old will generally mean that the replacement cost is less than the repair cost. The next thing to consider is the screen resolution. After purchasing a laptop you can upgrade the RAM and the hard drive, the CPU power of all modern machines is great enough that for most typical use it’s difficult to imagine any need to upgrade. But screen resolution is something that can never be good enough and can never be improved after purchase. Lenovo is offering T series Thinkpads with 1920×1200 resolution for $4000AU and 1680×1050 resolution for $3050AU. That’s 31% more pixels for 31% more money and seems like a good deal to me. I believe that a larger display can significantly increase productivity [2] so it seems that the extra expense would be a good investment if you plan to earn money from work you do on your laptop. As a point of reference a desktop monitor from Dell (who seems to be the cheapest supplier for such gear) with resolution of 1920×1200 will cost at least $1000AU. The hard drive capacity should not be an issue, it seems that 100G is about the minimum size. The 60G drive in my current Thinkpad is adequate for my development work (including several Xen instances and some ISO files for a couple of distributions) so unless you plan to collect MPEG4 files of TV series and store them on your laptop I can’t imagine 100G being much of a limit. Also external storage is getting quite cheap, 2GB USB flash devices are now in the bargain bin of my local electronics store and USB attached hard drives with capacities of 40G or more are getting cheap. Also with a Thinkpad replacing a hard drive is really easy and does not risk damage to the drive or the rest of the laptop (I don’t know how well other brands rate in this regard). For RAM you can buy a model with a large memory module in socket 0 (or attached to the motherboard). Adding new RAM later is easy to do. Just try and avoid purchasing a memory capacity that involves having all sockets filled with modules that are not of the maximum size - it’s annoying if you have to try and sell modules on Ebay after you buy a memory upgrade. Finally one mistake I made in the past was to not get all the options for the motherboard. Make sure that every option for Ethernet ports and 802.11 type protocols is selected. It might sound like a good idea to save $100 or so on not getting one of those options, but if you end up repeatedly plugging a CardBus or USB device for many years you will regret it. Also external devices tend to break or get lost. Rusty documents his laptop replacement as a time for spring-cleaning. I use LVM for the root filesytem on my Thinkpad so that I can easily install a new distribution (or a new version of a distribution) at any time. I’ve been through that spring-cleaning a couple of times on my current Thinkpad without needing new hardware. From a quick view of the Lenovo site it seems that an ideal new Thinkpad that would last me 6 years would cost about $4500 while one that would last me 2 years would cost $1600 (and have a significantly lower screen resolution). A Thinkpad that would last 6 years and not be so great (but still better than the cheap option) would cost about $3500. Update: One significant issue is the life expectancy of laptop batteries. If you use a laptop for mobile use (as opposed to just moving between desks occasionally) then you are probably familiar with the problem of laptop batteries that discharge after 10 minutes. Last time I checked the warranty on Thinkpad batteries was 1 year or 300 charges (whichever comes first). My experience is that after 300 full cycles a Thinkpad battery will only last for a small fraction of the original charge time. When buying a laptop I suggest getting a spare battery at the time of purchase. The spare battery may last longer than the battery that is shipped with the laptop and two batteries means that you have twice the number of charge cycles before they are both useless. Batteries apparently don’t last long if completely discharged, so charge them up before storing them and periodically charge them if they have been left unused for any length of time (maybe every second or third month). With a Thinkpad it seems quite safe to change the battery while the machine is plugged in to mains power and running (I expect that Lenovo doesn’t recommend this though). You should probably plan to have a battery die every three years of use (or sooner if you do a lot of travelling). So one spare battery may last you 6 years of use but you will need two spare batteries if you travel a lot.

David Welton: Languages: worst case vs average

This is a pretty good take on Tim Bray vs Erlang. However, I'd like to focus on something said at the end:
The fact is, all languages optimize for particular problem domains and I don't think that a universal programming language exists that would perform well on all tasks. Bray rapidly found out that Erlang isn't optimized for doing line-oriented I/O and it's regex library sucks. So what? While those problems could be eliminated from Erlang, the fact that Ericsson has deployed large telecom gear without having to fix those issues means that Erlang is ideally suited to its original programming domain.
The problem is that learning new languages is difficult for many people. They're not going to bother starting from 0 with a new language if what they've got is only 10/20% worse than the other guys. For companies, developing expertise (and trying to maintain it) in lots of languages is probably less productive than picking a few (Google only has four: Java, Python, Javascript and C++, and they're not exactly lacking in brainpower or cash). This is why "the right tool for the right job" works for things like saws and hammers, but not for complicated, idiomatic things like programming languages. Therefore, it's important that languages that aim to be widely used do lots of things pretty well, and try to avoid doing anything really badly. This is something I've always admired about Python, even if I don't use it anymore. It's a language that you can do a very wide variety of tasks with, and you can be sure there's probably some code out there already, and that Python will be ok for your task, even if it's not the best thing. Performance is a weak point compared to compiled languages, but even there, get the libraries right and things aren't *so* bad. In any case, my point isn't to bash Erlang (which I like), or put Python on a pillar (I prefer Ruby and Tcl), but to say that a widely used programming language doesn't need to be the best at everything, but it should aim to make people comfortable using it, safe in the knowledge that they can probably do most things *ok* if needs be. That's a far better situation than being able to do one thing realy well, and perform poorly at many other tasks, forcing its users to look elsewhere for solutions to those tasks. Personally, I think that for concurrency, Erlang is in a bit of a race right now - they're pretty far ahead in terms of concurrency (and also fault tolerance, but fewer people care about that), but behind in other areas (less library code). Other languages don't have as good a tools for concurrency, which isn't an easy problem. So, what will happen first - Erlang catches up elsewhere, other languages get "good enough" concurrency solutions, or a new language (like Scala) comes along and takes the lead?

Adrian von Bidder: Microsoft declares Bankruptcy

Intellectually, I mean. According to some Media outlets Ballmer (once again) declares war on the Free Software community. The technical “we are better” argument is apparently completely dropped (honesty, at last) and replaced by the pure “we have more money and patents”, this time not even thinly veiled in marketing speak. But I don't think PJ is right when she sais that “FOSS can't survive in an IP regime such as Ballmer has laid before our eyes.”. We've all seen how successful the sue your own customers business model is. At this time, I think it's safe to say that most big Linux users are also big Microsoft customers because Linux in the enterprise still means primarily Linux on the Server, Windows on the Desktop, so going after the big Linux users will be exactly that. Going after the small Linux users is just not interesting. Going after hobbyist FOSS developers will not work because only using patented works commercially actually violates the patent, as far as I understand patent law. Going against the commercial Linux developers will work to some degree, so let's look at this closer: Software patents are strongest in the U.S., so in Europe it will be much harder for MS for one thing. And, Novell and Red Hat excluded (and IBM, who should be big enough to take on MS, especially since their lawyers by now have had extensive training in what Linux is abaut), the commercial Linux space is still dominated by small companies, so MS would have to overload the judges with thousands of small lawsuits all over the world, and since no two companies do exactly the same thing I doubt the lawsuit templating technique adapted by the RIAA for music downloads will work. And, finally, the nice thing about Open Source Software (using this term here on purpose): it's out in the open. So kill one company, its former employees will found three new ones because the former customers (even if they're scared and would like to get rid of their Free Software) will still need support because they can't all switch (back) to MS based environments immediately. And I haven't even talked about public opinion and what antitrust offices might have to say, because those are extremely hard to predict and their effects are uncertain.

8 October 2007

Neil Williams: dpkg-cross 2.0.0 - fragility expected!

OK, changes in dpkg-dev 1.14.7 have forced my hand a little and dpkg-cross 2.0.0 has now been uploaded to Debian unstable (not experimental).
apt-cross 0.2.9 has been uploaded to Emdebian unstable - dependent on
dpkg-cross >= 1.99
apt-cross 0.3.0 has been uploaded to Debian unstable but will spend some time in the NEW queue (hence the 0.2.9 upload to Emdebian). apt-cross 0.3.0 is functionally identical to 0.2.9
emdebian-tools 0.3.9 has been uploaded to Emdebian unstable - dependent on apt-cross >= 0.2.9 and dpkg-cross >= 1.99
All three comprise extensive rewrites. apt-cross, in particular, is barely recognisable. dpkg-cross is a shadow of the former package and v2.0.0 signals the beginning of the long goodbye to both packages. dpkg developers expect to be able to merge all dpkg-cross functionality into dpkg before Lenny. Once that is done, it should be trivial to merge apt-cross behaviour into apt.
Only the new libcache-apt-perl package (containing the modified NorthernCross code from virtuoso) will persist. Once this whole transition is over, I'll upload Cache::Apt::* to CPAN so that it can become its own source package in Debian once apt-cross is removed.

IMPORTANT
dpkg cross-building support is HIGHLY EXPERIMENTAL!
Significant changes are still pending, including remapping the build paths during the
dpkg-buildpackage build and utilising the updated dpkg-shlibdeps code which is a decade ahead of what we had in dpkg-cross 1.39.

  1. dpkg 1.14.7 included dpkg-buildpackage as a perl script instead of shell which breaks dpkg-cross <= 1.39 but the new code to support dpkg-cross >= 1.99 is not implemented yet.

  2. The breakage that Zumbi experienced when using the dpkg version from
    experimental will now occur in unstable.
    http://lists.debian.org/debian-embedded/2007/09/msg00067.html. Note that this breakage would be severity "critical" if filed as a bug because it prevents not only cross-builds but normal Debian builds.

  3. dpkg-cross 2.0.0 fixes this issue. Hence the urgency in uploading dpkg-cross 2.0.0

  4. dpkg-cross 2.0.0 removes the old diversions of dpkg-buildpackage and dpkg-shlibdeps.

  5. dpkg is set to support the removal of the dpkg-cross diversions and code to support this is coming this weekend. This means that there will be a GAP where cross-built packages will fail to find the correct shared libraries. HOWEVER, installing dpkg-cross 2.0.0 restores the ability to build normal Debian packages whilst keeping dpkg-cross installed. This is a far more important gain than a temporary problem in packages that hardly anyone is using yet (our Emdebian crossbuilt packages).

When dpkg-dev 1.14.8 becomes available, apt-cross, dpkg-cross and emdebian-tools will be updated to depend on this version.
Everyone: please note - cross-building support in Debian will be "fragile" for the next week or so. The rewritten packages are being uploaded against experimental dpkg support I expect bugs. :-(

The benefits of these changes are:
  1. The removal of diversions that are a decade out of date.

  2. The implementation of virtuoso's NorthernCross code that transforms apt-cross and fixes lots of awkward bugs.

  3. The incorporation of cross-building support in the heart of dpkg.

  4. The beginning of the long goodbye to dpkg-cross and apt-cross.


RESULTS
With the new packages installed, some packages will produce:
 emdebuild: ERROR /usr/bin/ contains files for the wrong architecture!


i.e. the build appears to succeed but until dpkg-dev 1.14.8 includes support for looking for the correct libraries in /usr/$arch/lib instead of /usr/lib, we get unusable packages. This at least means that packages can be test built and the patches updated.

Russell Coker: I Am #40 in Don Marti s List

Don Marti has written his own equivalent to Technorati based on links from blogs that he reads, and my blog comes in at #40 in the list (last place) [1]. Don does note the fact that such lists mean little and links to a post by Doc Searls [2] which makes the same point more strongly. But it’s still interesting to note. Also if Don chose to release the Perl script in question (or host is as a cgi-bin script) to allow other people to make their own top 40 list then I’m sure that many people would appreciate it (I’ll write such a script myself and release it if he doesn’t). He notes when publishing his list that a blog may be included even if he never reads it. I believe that if a blog is highly ranked according to links from blogs that you read then it’s quite likely that you will be interested in reading it, it may be a blog that is opposed to your beliefs (for example I’m sure I’m not the only person in the Linux community to link to an official Microsoft blog) but if it gets enough links then it would be worth reading at least once (if only to discover why you don’t like it). While I can’t tell Don to read every blog on his top 40 list I’ll certainly read every blog on my top 40 list at least once when I create the list.

Theodore Ts'o: Sous Vide, Revisited

In a previous post, I had recommended the 4130 NIST-Traceable Temperature Controller to control the temperature in a slow cooker. Unfortunately, that particular controller has a range that tops out at 60 degrees C / 140 degrees F, which is enough for cooking beef for long periods of time, but not enough for say, cooking duck confit, which for which a sous vide temperature of 80 degrees C is recommended. In addition, the 4130 is pretty expensive; almost $150. It’s possible to add a resistor to change the range of the 4130, but the temperature displayed by the controller is no longer correct, and you have manually create a conversion table between true temperature and the temperature as seen by the controller. I’ve recently come across a cheaper and better possibility, the Ranco ETC-111000-000 Temperature Controller which is only half the price and comes with a much larger working range (-30 to 220 degrees F). The price with the AC cord already wired in is $75; and the version which just has a 120VAC SPDT relay is only $60. A bit more about food safety. There has always been a lot of concern about bacteria growth and botulism, for good reason — and so therefore the recommendations for cooking temperature have a lot of safety margin in them — to the point now that the USDA recommends that steaks be cooked to at least 145 degrees F, which is well within what had traditionally been called “medium”, and chicken to at least 165 degrees F, which is enough to really destroy taste and texture. Sous vide cooking, especially some of the more low temperature variants, have raised a lot of concerns, to the point where a few years ago New York City (temporarily) banned it, causing a great outcry in the foodie community, since many top restaurants use sous vide techniques. First of all, any recommendation about internal temperatures and food safety that doesn’t also factor in time is massively oversimplifying the problem. Here is a table taken from “Food Safety Hazards and Controls for the Home Food Preparer”, published by the Hospitality Institute of Technology in 1994:
Temperature, F Time, 5D kill Time, 6.5D kill
130 86.42 minutes 112.34 minutes
135 27.33 minutes 35.53 minutes
140 8.64 minutes 11.23 minutes
145 2.73 minutes 3.55 minutes
150 51.85 seconds 1.12 minutes
155 16.40 seconds 21.32 seconds
160 5.19 seconds 6.74 seconds
165 1.64 seconds 2.13 seconds
This table lists the time to reduce bacteria concentrations of Salmonella and E. coli from 100,000 to 1 (5D) or 3,162,277 to 1 (6.5D). The FDA and USDA recommend cooking hamburger to 5D destruction. Since it is extremely unlikely for there to be more than 100 Salmonella organisms per gram of meat, a 5D kill will reduce Salmonella concentrations to no more than 1 organism per kilogram. So whether you cook a piece of meat to an internal temperature of 165 degrees for 1.64 seconds, or hold it at 130 degrees for 90 minutes, the effect on Salmonella and E. coli bacteria will be the same.Of course, one concern is that in many forms of cooking, particularly oven roasting and grilling, the temperature of the food as it heats up may not be even; so how do you guarantee that all parts of the food product has been brought up to the requisite 165 degrees? One way that recipe authors, fearful of liability concerns, have done so is to tell people to cook meat to much higher internal temperatures, to provide that extra safety margin at the expense of dessicated, horribly tasting turkey or chicken. But the advantage of sous vide cooking is by immersing the food in a water bath, the excellent heat conductivity of water helps guarantee that the entire body of meat will get raised to desired temperature relatively quickly. (How quickly depends on the thickness of the meat, obviously). So if you hold a roast beef that has been vacuum packed in a Foodsaver bag for five or six hours, there should be no question that all of the common bacteria has been inactivated, and that amount of time at 130 degrees F should be sufficient to inactivate 99.9 percent of all botulism toxin molecules (not there should be any in a fresh piece of meat, of course!) However, sous vide temperatures are not enough to kill bacterial spores, in particular C. botulinum, which is responsible for botulism. This requires temperatures far in excess of boiling water at sea level. For example, home canning protocols recommend holding the food product at 250 degrees F for at least 15 minutes. This is an issue in the restaurant business because very often food would be cooked sous vide, and then stored in the vacuum sealed bags for potentially weeks (yes, you could be eating an extremely expensive meal at a top-end French restaurant that had been cooked several weeks ago, and reheated just before serving; yummy, no?). If the food packages aren’t cooled quickly enough, and then allowed to warm to the danger zone, it’s possible that in the anerobic environment the C. botulinum spores could germinate and then start producing toxin. But, if you are cooking home sous vide where you are serving the food right after it has been cooked, this shouldn’t be a concern.

Thijs Kinkhorst: Utrechts Kampioenschap Programmeren

Last Saturday we organised the Utrecht Programming Championship, which apart from declaring a local champion also selects for the Benelux championship. Team A-Cognito won by solving 6 out of 8 problems, the best score of all cities organising parallel contests. See the results here. We organise this every year, but this year was special as it was a general rehearsal for the North Western European contest that we are also hosting this autumn. This is part of the ACM ICPC worldwide network of contests which culminate in the ICPC World Finals. Luckily everything went great. Of course there were some problems, but they were resolved in a calm and adequate fashion. It gives good hope that NWERC will be a success, at least from the techinical side...

Russell Coker: Can SE Linux Stop a Linux Storm

Bruce Schneier has just written about the Storm Worm [1] which has apparently been quietly 0wning some Windows machines for most of this year (see the Wikipedia page for more information [2]). I have just been asked whether SE Linux would stop such a worm from the Linux environment. SE Linux does prevent many possible methods of getting local root. If a user who does not have the root password (or is not going to enter it from a user session) has their account taken over by a hostile party then the attacker is not going to get local root (unless there is a kernel vulnerability). Without local root access the activities of the attacker can be seen by a user logged in on another account - processes will be seen by all user sessions if using the SE Linux targeted policy and files and processes can be seen by the sys-admin. If while a user account is 0wned the user runs “su -” (or an equivalent command) then in theory at least the attacker can sniff this and gain local root access (whether enough users do this to make attackers feel that it’s worth their effort to write the code in question is something I couldn’t even guess about). If the user is clueless then the attacker could immediately display a dialog with some message that sound urgent and demand the root password - some users would give it. If the user is even moderately smart the attacker could fake the GUI dialogues for installing updated packages (which have been in Red Hat distributions for ages and have appeared in Debian more recently) and tell the user that they need to enter the root password to install an important security update (oh the irony). In conclusion I think that if a user is ill-educated enough to want to run a program that was sent to them in email by a random person then I expect that the program would have a good chance of coercing them into giving it local root access if the user in question had the capability of doing so. Even if a Linux trojan did not have local root access then it could still do a lot of damage. Any server operations that don’t require ports 1024 (which means most things other than running a web, DNS, or mail server) can still be performed and client access will always work (including sending email). The trojan would have access to all of the user's data (which for a corporate desktop machine usually means a huge network share of secret documents). If a trojan only attempts to perform actions that SE Linux permits (running programs from the user's home directory, accessing servers for DNS, HTTP, IRC, SMTP, and other protocols - a reasonable set of options for a trojan) then the default configuration of SE Linux (targeted policy) won't stop it or even log anything. This is not a problem with SE Linux, just a direct result of the fact that in every situation a trojan can perform all operations that the user can perform - and if the trojan only wants to receive commands via web and IRC servers and send spam via the user's regular mail server then it will be a small sub-set of the permitted operations for the user! If however the trojan tries more aggressive methods then SE Linux will log some AVC messages about access being denied. If the sys-admin has good procedures for analysing log files they will notice such things, understand what they mean, and be able to contain the damage. Also there have been at least two cases where SE Linux prevented local root exploits. Finally, in answer to the original question: SE Linux will stop some of the more aggressive methods that trojans might use. But there are still plenty of things that a trojan could do to cause harm which won't be stopped or audited by SE Linux policy. When Linux gets more market share among users with a small amount of skill and no competent person to do sys-admin work for them we will see some Linux trojans and more Linux worms. It will be interesting to see what methods the trojan authors decide to use.

Ross Burton: Something Ate My Avahi

So the new NAS is happily running Debian now, and I'll write a long blog for anyone who wants to do the same once I have one remaining problem sorted out... Avahi doesn't work. If I start Avahi on the NAS other machines see the announcement and the names it provides will resolve, but after a few minutes they fail to resolve as they timeout. Unless my router is incredibly brain-dead it can't be the cause, because the old NAS is connected in exactly the same way and it works fine. For extra fun, the NAS can't see the rest of the mDNS network. It's almost like the network driver is dropping incoming multicast packets. Before anyone asks, there is no firewall running on the NAS. Does anyone have any ideas? NP: Remembranza, Murcof

Next.

Previous.