Search Results: "ag"

10 October 2007

Russell Coker: Dreamhost and the DMCA

Dreamhost have refused my request (under the DMCA) to be correctly identified as the author of content copied from my blog. I am publishing this so that anyone else who deals with them will know what to expect. Also if someone wishes to sue Dreamhost in regard to content that they host this may help demonstrate a pattern of behaviour. The situation is quite obviously the result of a broken script used by a splogger that doesn’t correctly match author names with articles. The fact that the official Dreamhost policy is to disregard the requirement that the author(s) of copyright material be correctly identified is reprehensible. It also seems likely to open them to the risk of legal action. If you know how to contact a director of Dreamhost then please give them a link to this post and explain the risks to them. For anyone who wants the detail the messages are below.
(more…)

Russell Coker: Who Can Contribute to Free Software

A common misconception is that only programmers can contribute to free software. The first significant reference I recall to this was in a presentation by Pia Waugh [1] where she mentioned that she felt that the way words such as “coder” and “hacker” are used in the community as synonyms for “contributor” are denigrating to people such as herself who aren’t coders! I’m certain that no-one in the Australian Linux community would have any doubt about Pia’s contributions, not even when they mis-use terms such as “coder“. Non-coding ways of contributing include writing documentation, arranging meetings and conferences, and serving on the committee of LUGs and other organisations. I’m sure that there are many other ways of contributing that I can’t think of at the moment. The development of the free software community depends on a wide range of skills, and many of the best coders don’t have great skills in other areas. The meme that you have to be a great coder to contribute causes two problems, one is that there is a lack of contributers to non-coding tasks, and another is that coders end up doing non-coding work that they are often not particularly good at and which takes them away from things that they do well. I recently refused a nomination for the committee of my local LUG because I believe that most members can do the committee work as well as I can and many of them can do it better than me - so it’s best if I spend my time coding and preparing presentations about code that I write instead of joining the LUG committee. I was reminded of Pia’s presentation (from some years ago) by a comment on my blog post about Ideas for a Home University [2] where the commenter seems to believe that they can’t adopt the university degree equivalent via free software contributions that I suggest because of not being a programmer or sys-admin. I really doubt that anyone would care if Pia has a university degree (I don’t know whether she has one), I’m sure that the companies that hire Waugh Partners [3] do so because of the reputation that Pia and Jeff have for getting things done and for their positions in the community and not because of whatever certificates that the partners may have. I suggest to the commenter in question (and anyone else in a similar position) that they become involved in running their local LUG (or starting one if there isn’t one already). You would really be surprised by the number of job opportunities that arise from running such a community organisation.

Steve Kemp: I role and I tumble practically all night long

Today mostly consisted of a new release of the chronicle blog compiler. Interestingly this received several random mails today. I wonder what caused that all of a sudden? The release of the compiler is timely, as it reminds me I've still not managed to find a decent gallery compiler. Although the thought of writing my own, rightly, fills me with depression. I've been interested in reading more about both Git and SELinux upon Planet <strike>Coker</strike> Debian recently. I've switched several small projects over to GIT but I've not yet listed them publically. First of all I would like to see if there is a version of trac that I can install which supports git repositories. I guess that's a job to research tomorrow. I wonder if I would confuse people by hosting GIT projects upon cvsrepository.org? ;)

Martin F. Krafft: Packaging with Git

Introduction I gave a joint presentation with Manoj at Debconf7 about using distributed version control for Debian packaging, and I volunteered to do an on-line workshop about using Git for the task, so it's about time that I should know how to use Git for Debian packaging, but it turns out that I don't. Or well, didn't. After I made a pretty good mess out of the mdadm packaging repository (which is not a big problem as it's just ugly history up to the point when I start to get it right), I decided to get down with the topic and figure it out once and for all. I am writing this post as I put the pieces together. It's been cooking for a week, simply so I could gather enough feedback. I am aware that Git is not exactly a showcase of usability, so I took some extra care to not add to the confusion. It may be the first post in a series, because this time, I am just covering the case of mdadm, for which upstream also uses Git and where I am the only maintainer, and I shall pretend that I am importing mdadm to version control for the first time, so there won't be any history juggling. Future posts could well include tracking Subversion repositories with git-svn, and importing packages previously tracked therewith, but this ain't no promise! (well, that last post is already being drafted, but far from finished; you have been warned!) I realise that git-buildpackage exists, but imposes a rather strict branch layout and tagging scheme, which I don't want to adhere to. And gitpkg (Romain blogged about it recently), deserves another look since, according to its author, it does not impose anything on its user. But in any case, before using such tools (and possibly extending them to allow for other layouts), I'd really rather have done it by hand a couple of times to get the hang of it and find out where the culprits lie. Now, enough of the talking, just one last thing: I expect this blog post to change quite a bit as I get feedback. Changes shall be highlighted in bold typeface.
Setting up the infrastructure First, we prepare a shared repository on git.debian.org for later use (using collab-maint for illustration purposes), download the Debian source package we want to import (version 2.6.3+200709292116+4450e59-3 at time of writing, but I pretend it's -2 because we shall create -3 further down ), set up a local repository, and link it to the remote repository. Note that there are other ways to set up the infrastructure, but this happens to be the one I prefer, even though it's slightly more complicated:
$ ssh alioth
$ cd /git/collab-maint
$ ./setup-repository pkg-mdadm mdadm Debian packaging
$ exit
$ apt-get source --download-only mdadm
$ mkdir mdadm && cd mdadm
$ git init
$ git remote add origin ssh://git.debian.org/git/collab-maint/pkg-mdadm
$ git config branch.master.merge refs/heads/master
Now we can use git-pull and git-push, except the remote repository is empty and we can't pull from there yet. We'll save that for later. Instead, we tell the repository about upstream's Git repository. I am giving you the git.debian.org URL though, simply because I don't want upstream repository (which lives on an ADSL line) hammered in response to this blog post:
$ git remote add upstream-repo git://git.debian.org/git/pkg-mdadm/mdadm
Since we're using the upstream branch of the pkg-mdadm repository as source (and don't want all the other mess I created in that repository), we'll first limit the set of branches to be fetched (I could have used the -t option in the above git-remote command, but I prefer to make it explicit that we're doing things slightly differently to protect upstream's ADSL line).
$ git config remote.upstream-repo.fetch \
    +refs/heads/upstream:refs/remotes/upstream-repo/upstream
And now we can pull down upstream's history and create a local branch off it. The "no common commits" warning can be safely ignored since we don't have any commits at all at that point (so there can't be any in common between the local and remote repository), but we know what we're doing, even to the point that we can forcefully give birth to a branch, which is because we do not have a HEAD commit yet (our repository is still empty):
$ git fetch upstream-repo
warning: no common commits
[ ]
  # in the real world, we'd be branching off upstream-repo/master
$ git checkout -b upstream upstream-repo/upstream
warning: You appear to be on a branch yet to be born.
warning: Forcing checkout of upstream-repo/upstream.
Branch upstream set up to track remote branch
  refs/remotes/upstream-repo/upstream.
$ git branch
* upstream
$ ls   wc -l
77
Importing the Debian package Now it's time to import Debian's diff.gz remember how I pretend to use version control for package maintenance for the first time. Oh, and sorry about the messy file names, but I decided it's best to stick with real data in case you are playing along: Since we're applying the diff against version 2.6.3+200709292116+4450e59, we ought to make sure to have the repository at the same state. Upstream never "released" that version, but I encoded the commit ID of the tip when I snapshotted it: 4450e59, so we branch off there. Since we are actually tracking the git.debian.org pkg-mdadm repository instead of upstream, you can use the tag I made. Otherwise you could consider tagging yourself:
$ #git tag -s mdadm-2.6.3+200709292116+4450e59 4450e59
$ git checkout -b master mdadm-2.6.3+200709292116+4450e59
$ zcat ../mdadm_2.6.3+200709292116+4450e59-2.diff.gz   git apply
The local tree is now "debianised", but Git does not know about the new and changed files, which you can verify with git-status. We will split the changes made by Debian's diff.gz across several branches.
The idea of feature branches We could just create a debian branch, commit all changes made by the diff.gz there, and be done with it. However, we might want to keep certain aspects of Debianisation separate, and the way to do that is with feature branches (also known as "topic" branches). For the sake of this demonstration, let's create the following four branches in addition to the master branch, which holds the standard Debian files, such as debian/changelog, debian/control, and debian/rules:
  • upstream-patches will includes patches against the upstream code, which I submit for upstream inclusion.
  • deb/conffile-location makes /etc/mdadm/mdadm.conf the default over /etc/mdadm.conf and is Debian-specific (thus the deb/ prefix).
  • deb/initramfs includes the initramfs hook and script, which I want to treat separately but not submit upstream.
  • deb/docs similarly includes Debian-only documentation I add to the package as a service to Debian users.
If you're importing a Debian package using dpatch, you might want to convert every dpatch into a single branch, or at least collect logical units into separate branches. Up to you. For now, our simple example suffices. Keep in mind that it's easy to merge two branch and less trivial to split one into two. Why? Well, good question. As you will see further down, the separation between master and deb/initramfs actually makes things more complicated when you are working on an issue spanning across both. However, feature branches also bring a whole lot of flexibility. For instance, with the above separation, I could easily create mdadm packages without initramfs integration (see #434934), a disk-space-conscious distribution like grml might prefer to leave out the extra documentation, and maybe another derivative doesn't like the fact that the configuration file is in a different place from upstream. With feature branches, all these issues could be easily addressed by leaving out unwanted branches from the merge into the integration/build branch (see further down). Whether you use feature branches, and how many, or whether you'd like to only separate upstream and Debian stuff is entirely up to you. For the purpose of demonstration, I'll go the more complicated way.
Setting up feature branches So let's commit the individual files to the branches. The output of the git-checkout command shows modified files that have not been committed yet (which I trim after the first example); Git keeps these across checkouts/branch changes. Note that the ./debian/ directory does not show up as Git does not know about it yet (git-status will tell you that it's untracked, or rather: contains untracked files since Git does not track directories at all):
$ git checkout -b upstream-patches mdadm-2.6.3+200709292116+4450e59
M Makefile
M ReadMe.c
M mdadm.8
M mdadm.conf.5
M mdassemble.8
M super1.c
$ git add super1.c     #444682
$ git commit -s
  # i now branch off master, but that's the same as 4450e59 actually
  # i just do it so i can make this point 
$ git checkout -b deb/conffile-location master
$ git add Makefile ReadMe.c mdadm.8 mdadm.conf.5 mdassemble.8
$ git commit -s
$ git checkout -b deb/initramfs master
$ git add debian/initramfs/*
$ git commit -s
$ git checkout -b deb/docs master
$ git add RAID5_versus_RAID10.txt md.txt rootraiddoc.97.html
$ git commit -s
  # and finally, the ./debian/ directory:
$ git checkout master
$ chmod +x debian/rules
$ git add debian
$ git commit -s
$ git branch
  deb/conffile-location
  deb/docs
* master
  upstream
  upstream-patches
At this time, we push our work so it won't get lost if, at this moment, aliens land on the house, or any other completely plausible event of apocalypse descends upon you. We'll push our work to git.debian.org (the origin, which is the default destination and thus needs not be specified) by using git-push --all, which conveniently pushes all local branches, thus including the upstream code; you may not want to push the upstream code, but I prefer it since it makes it easier to work with the repository, and since most of the objects are needed for the other branches anyway after all, we branched off the upstream branch. Specifying --tags instead of --all pushes tags instead of heads (branches); you couldn't have guessed that! See this thread if you (rightfully) think that one should be able to do this in a single command (which is not git push refs/heads/* refs/tags/*)
$ git push --all
$ git push --tags
Done. Well, almost
Building the package (theory) Let's build the package. There seem to be two (sensible) ways we could do this, considering that we have to integrate (merge) the branches we just created, before we fire off the building scripts:
  1. by using a temporary (or "throw-away") branch off upstream, where we integrate all the branches we have just created, build the package, tag our master branch (it contains debian/changelog), and remove the temporary branch. When a new package needs to be built, we repeat the process.
  2. by using a long-living integration branch off upstream, into which we merge all our branches, tag the branch, and build the package off the tag. When a new package comes around, we re-merge our branches, tag, and build.
Both approaches have a certain appeal to me, but I settled for the second, for two reasons, the first of which leads to the second:
  1. When I upload a package to the Debian archive, I want to create a tag which captures the exact state of the tree from which the package was built, for posterity (I will return to this point later). Since the throw-away branches are not designed to persist and are not uploaded to the archive, tagging the merging commit makes no sense. Thus, the only way to properly identify a source tree across all involved branches would be to run git-tag $branch/$tagname $branch for each branch, which is purely semantic and will get messy sooner or later.
  2. As a result of the above: when Debian makes a new stable release, I would like to create a branch corresponding to the package in the stable archive at the time, for security and other proposed updates. I could rename my throw-away branch, if it still existed, or I could create a new branch and merge all other branches, using the (semantic) tags, but that seems rather unfavourable.
So instead, I use a long-living integration branch, notoriously tag the merge commits which produced the tree from which I built the package I uploaded, and when a certain version ends up in a stable Debian release, I create a maintenance branch off the one, single tag which corresponds to the very version of the package distributed as part of the Debian release. So much for the theory. Let's build, already!
Building the package (practise) So we need a long-living integration branch, and that's easier done than said:
$ git checkout -b build mdadm-2.6.3+200709292116+4450e59
Now we're ready to build, and the following procedure should really be automated. I thus write it like a script, called poor-mans-gitbuild, which takes as optional argument the name of the (upstream) tag to use, defaulting to upstream (the tip):
#!/bin/sh
set -eu
git checkout master
debver=$(dpkg-parsechangelog   sed -ne 's,Version: ,,p')
git checkout build
git merge $ 1:-upstream 
git merge upstream-patches
git merge master
for b in $(git for-each-ref --format='%(refname)' refs/heads/deb/*); do
  git merge -- $b
done
git tag -s debian/$debver
debuild   # will ignore .git automatically
git checkout master
Note how we are merging each branch in turn, instead of using the octopus merge strategy (which would create a commit with more than two parents) for reasons outlined in this post. An octopus-merge would actually work in our situation, but it will not always work, so better safe than sorry (although you could still achieve the same result). If you discover during the build that you forgot something, or the build script failed to run, just remove the tag, undo the merges, checkout the branch to which you need to commit to fix the issue, and then repeat the above build process:
$ git tag -d debian/$debver
$ git checkout build
$ git reset --hard upstream
$ git checkout master
$ editor debian/rules    # or whatever
$ git add debian/rules
$ git commit -s
$ poor-mans-gitbuild
Before you upload, it's a good idea to invoke gitk --all and verify that all goes according to plan:
screenshot of gitk after the above steps
When you're done and the package has been uploaded, push your work to git.debian.org, as before. Instead of using --all and --tags, I now specify exactly which refs to push. This is probably a good habit to get into to prevent publishing unwanted refs:
$ git push origin build tag debian/2.6.3+200709292116+4450e59-3
Now take your dog for a walk, or play outside, or do something else not involving a computer or entertainment device.
Uploading a new Debian version If you are as lucky as I am, the package you uploaded still has a bug in the upstream code and someone else fixes it before upstream releases a new version, then you might be in the position to release a new Debian version. Or maybe you just need to make some Debian-specific changes against the same upstream version. I'll let the commands speak for themselves:
$ git checkout upstream-patches
$ git-apply < patch-from-lunar.diff   #444682 again
$ git commit --author 'J r my Bobbio <lunar@debian.org>' -s
  # this should also be automated, see below
$ git checkout master
$ dch -i
$ dpkg-parsechangelog   sed -ne 's,Version: ,,p'
2.6.3+200709292116+4450e59-3
$ git commit -s debian/changelog
$ poor-mans-gitbuild
$ git push
$ git push origin tag debian/2.6.3+200709292116+4450e59-3
That first git-push may require a short explanation: without any arguments, git-push updates only the intersection of local and remote branches, so it would never push a new local branch (such as build above), but it updates all existing ones; thus, you cannot inadvertedly publish a local branch. Tags still need to be published explicitly.
Hacking on the software Imagine: on a rainy Saturday afternoon you get bored and decide to implement a better way to tell mdadm when to start which array. Since you're a genius, it'll take you only a day, but you do make mistakes here and there, so what could be better than to use version control? However, rather than having a branch that will live forever, you are just creating a local branch, which you will not publish. When you are done, you'll feed your work back into the existing branches. Git makes branching really easy and as you may have spotted, the poor-mans-gitbuild script reserves an entire branch namespace for people like you:
$ git checkout -b tmp/start-arrays-rework master
Unfortunately (or fortunately), fixing this issue will require work on two branches, since the initramfs script and hook are maintained in a separate branch. There are (again) two ways in which we can (sensibly) approach this:
  • create two separate, temporary branches, and switch between them as you work.
  • merge both into the temporary branch and later cherry-pick the commits into the appropriate branches.
I am undecided on this, but maybe the best would be a combination: merge both into a temporary branch and later cherry-pick the commits into two additional, temporary branches until you got it right, and then fast-forward the official branches to their tips:
$ git merge master deb/initramfs
$ editor debian/mdadm-raid                     #  
$ git commit -s debian/mdadm-raid
$ editor debian/initramfs/script.local-top     #  
$ git commit -s debian/initramfs/script.local-top
[many hours of iteration pass ]
[  until you are done]
$ git checkout -b tmp/start-arrays-rework-init master
  # for each commit $c in tmp/start-arrays-rework
  # applicable to the master branch:
$ git cherry-pick $c
$ git checkout -b tmp/start-arrays-rework-initramfs deb/initramfs
  # for each commit $c in tmp/start-arrays-rework
  # applicable to the deb/initramfs branch:
$ git cherry-pick $c
This is assuming that all your commits are logical units. If you find several commits which would better be bundled together into a single commit, this is the time to do it:
$ git cherry-pick --no-commit <commit7>
$ git cherry-pick --no-commit <commit4>
$ git cherry-pick --no-commit <commit5>
$ git commit -s
Before we now merge this into the official branches, let me briefly intervene and introduce the concept of a fast-forward. Git will "fast-forward" a branch to a new tip if it decides that no merge is needed. In the above example, we branched a temporary branch (T) off the tip of an official branch (O) and then worked on the temporary one. If we now merge the temporary one into the official one, Git determines that it can actually squash the ancestry into a single line and push the official branch tip to the same ref as the temporary branch tip. In cheap (poor man's), ASCII notation:
- - - O             >> merge T >>     - - - = - - OT
         - - T      >>  into O >>
This works because no new commits have been made on top of O (if there would be any, we might be able to rebase, but let's not go there quite yet; rebasing is how you shoot yourself in the foot with Git). Thus we can simply do the following:
$ git checkout deb/initramfs
$ git merge tmp/start-arrays-rework-initramfs
$ git checkout master
$ git merge tmp/start-arrays-rework-init
and test/build/push the result. Or well, since you are not an mdadm maintainer (We^W I have open job positions! Applications welcome!), you'll want to submit your work as patches via email:
$ git format-patch -s -M origin/master
This will create a number of files in the current directory, one corresponding for each commit you made since origin/master. Assuming each commit is a logical unit, you can now submit these to an email address. The --compose option lets you write an introductory message, which is optional:
$ git send-email --compose --to your@email.address <file1> <file2> < >
Once you've verified that everything is alright, swap your email address for the bug number (or the pkg-mdadm-devel list address). Thanks (in advance) for your contribution! Of course, you may also be working on a feature that you want to go upstream, in which case you'd probably branch off upstream-patches (if it depends on a patch not yet in upstream's repository), or upstream (if it does not):
$ git checkout -b tmp/cool-feature upstream
[ ]
when a new upstream version comes around After a while, upstream may have integrated your patches, in addition to various other changes, to give birth to mdadm-2.6.4. We thus first fetch all the new refs and merge them into our upstream branch:
$ git fetch upstream-repo
$ git checkout upstream
$ git merge upstream-repo/master
we could just as well have executed git-pull, which with the default configuration would have done the same; however, I prefer to separate the process into fetching and merging. Now comes the point when many Git people think about rebasing. And in fact, rebasing is exactly what you should be doing, iff you're still working on an unpublished branch, such as the previous tmp/cool-feature off upstream. By rebasing your branch onto the updated upstream branch, you are making sure that your patch will apply cleanly when upstream tries it, because potential merge conflicts would be handled by you as part of the rebase, rather than by upstream:
$ git checkout tmp/cool-feature
$ git rebase upstream
What rebasing does is quite simple actually: it takes every commit you made since you branched off the parent branch and records the diff and commit message. Then, for each diff/commit_message pair, it creates a new commit on top of the new parent branch tip, thus rewrites history, and orphans all your original commits. Thus, you should only do this if your branch has never been published or else you would leave people who cloned from your published branch with orphans.
If this still does not make sense, try it out: create a (source) repository, make a commit (with a meaningful commit message), branch B off the tip, make a commit on top of B (with a meaningful message), clone that repository and return to the source repository. There, checkout the master, make a commit (with a ), checkout B, rebase it onto the tip of master, make a commit (with a ), and now git-pull from the clone; use gitk to figure out what's going on.
So you should almost never rebase a published branch, and since all your branches outside of the tmp/* namespace are published on git.debian.org, you should not rebase those. But then again, Pierre actually rebases a published branch in his workflow, and he does so with reason: his patches branch is just a collection of branches to go upstream, from which upstream cherry-picks or which upstream merges, but which no one tracks (or should be tracking). But we can't (or at least will not at this point) do this for our feature branches (though we could treat upstream-patches that way), so we have to merge. At first, it suffices to merge the new upstream into the long-living build branch, and to call poor-mans-gitbuild, but if you run into merge conflicts or find that upstream's changes affect the functionality contained in your feature branches, you need to actually fix those. For instance, let's say that upstream started providing md.txt (which I previously provided in the deb/docs branch), then I need to fix that branch:
$ git checkout deb/docs
$ git rm md.txt
$ git commit -s
That was easy, since I could evade the conflict. But what if upstream made a change to Makefile, which got in the way with my configuration file location change? Then I'd have to merge upstream into deb/conffile-location, resolve the conflicts, and commit the change:
$ git checkout deb/conffile-location
$ git merge upstream
CONFLICT!
$ git-mergetool
$ git commit -s
When all conflicts have been resolved, I can prepare a new release, as before:
$ git checkout master
$ dch -i
$ dpkg-parsechangelog   sed -ne 's,Version: ,,p'
2.6.3+200709292116+4450e59-3
# git commit -s debian/changelog
$ poor-mans-gitbuild
# git push
$ git push origin tag debian/2.6.3+200709292116+4450e59-3
Note that Git often appears smart about commits that percolated upstream: since upstream included the two commits in upstream-patches in his 2.6.4 release, my upstream-patches branch got effectively annihilated, and Git was smart enough to figure that out without a conflict. But before you rejoice, let it be told that this does not always work.
Creating and using a maintenance branch Let's say Debian "lenny" is released with mdadm 2.7.6-1, then:
$ git checkout -b maint/lenny debian/2.7.6-1
You might do this to celebrate the release, or you may wait until the need arises. We've already left the domain of reality ("lenny" is not yet released), so the following is just theory. Now, assume that a security bug is found in mdadm 2.7.6 after "lenny" was released. Upstream is already on mdadm 2.7.8 and commits deadbeef and c0ffee fix the security issue, then you'd cherry-pick them into the maint/lenny branch:
$ git checkout upstream
$ git pull
$ git checkout maint/lenny
$ git cherry-pick deadbeef
$ git cherry-pick c0ffee
If there are no merge conflicts (which you'd resolve with git-mergetool), we can just go ahead to prepare the new package:
$ dch -i
$ dpkg-parsechangelog   sed -ne 's,Version: ,,p'
2.7.6-1lenny1
$ git commit -s debian/changelog
$ poor-mans-gitbuild
$ git push origin maint/lenny
$ git push origin tag debian/2.7.6-1lenny1
Future directions It should be trivial to create the Debian source package directly from the repository, and in fact, in response to a recent blog post of mine on the dispensability of pristine upstream tarballs, two people showed me their scripts to do it. My post also caused Joey Hess to clarify his position on pristine tarballs, before he went out to implement dpkg-source v3. This looks very promising. Yet, as Romain argues, there are benefits with simple patch management systems. Exciting times ahead! In addition to creating source packages from version control, a couple of other ideas have been around for a while:
  • create debian/changelog from commit log summaries when you merge into the build branch.
  • integrate version control with the BTS, bidirectionally:
    • given a bug report, create a temporary branch and apply any patches found in the bug report.
    • upon merging the temporary branch back into the feature branch it modifies, generate a patch, send it to the BTS and tag the bug report + pending patch.
And I am sure there are more. If you have any, I'd be interested to hear about them!
Wrapping up I hope this post was useful. Thank you for reading to the end, this was probably my longest blog post ever. I want to thank Pierre Habouzit, Johannes Schindelin, and all the others on the #git/freenode IRC channel for their tutelage. Thanks also to Manoj Srivastava, whose pioneering work on packaging with GNU arch got me started on most of the concepts I use in the above. And of course, the members of the the vcs-pkg mailing list for the various discussions on this subject, especially those who participated in the thread leading up to this post. Finally, thanks to Linus and Junio for Git and the continuously outstanding high level of support they give. If you are interested in the topic of using version control for distro packaging, I invite you to join the vcs-pkg mailing list and/or the #vcs-pkg/irc.oftc.net IRC channel. NP: Aphex Twin: Selected Ambient Works, Volume 2 (at least when I started writing )

Clint Adams: Good job, rats

[jjjjordi] El a o pasado 1,944 neoyorquinos vieron algo y dijeron algo. Gracias por mantener tus ojos y o dos bien abiertos. Y no guardar tus sospechas para ti mismo. Si ves algo, di algo! Habla con un polic a o con un empleado de la xxx. O llama al 1-888-xxx-xxxx

MJ Ray: Current Issues / Walking and Cycling in WsM

I'm going to the Weston-super-Mare walking and cycling strategy consultation event this evening.

What messages should I take with me?  What topics should I watch out for?  I'll check for replies in the forum and my email and comment form before I leave (5.20pm, I guess).

For cycling, I think the key points will be: no confusing facilities, better route signs and fewer junctions.  I think walking is probably mostly about keeping paths clear, open and safe.  Have I missed something obvious?

Adam Rosi-Kessel: MBTA Responds

To follow up on yesterday’s MBTA philippic, I received a response from the T’s customer service regarding my complaints. Although it doesn’t give me hope that things will improve, it’s nice that they have at least hired someone to write personalized, polished replies. At least they can do that right:
Dear Mr. Kessel, Thank you for your email concerning service on the Needham Line. The past couple of months have been difficult for our passengers, and I want you to know we are very sorry for the inconveniences this has been causing. The causes of these delays have ranged from signal and mechanical issues to long stops boarding passengers at stations, and even track work on other lines. I know that it may seem odd that trains on other lines can impact your trains, but I assure you that is true. The Franklin, Needham, Providence, and Stoughton lines, as well as Amtrak all share the rail between Hyde Park and Boston. All it takes is one delayed train in this area to cause a cascading effect that delays other lines. There’s also the fact that trains may come in from one line to go out as another. For example, a train on the Worcester Line comes to Boston, and either the train set, or the crew (sometimes both) head out as an outbound Needham train. If the inbound is late from Worcester, then the outbound Needham will be late, which also means that its “turn,” the next inbound, will be delayed. In response to your comment about the doors not all being opened, even with a full compliment of conductors, crews are not able to staff every door, and especially at ground level stations we want to ensure that all customers are able to board and detrain safely. This is why conductors do not open all doors to the train. I contacted our Mechanical Department, asking that the PA system on train 625 be checked out to make sure everything is working properly. Please know that delays in service are not something that we take lightly, and all departments are committed to working together to ensure that our customers receive timely, safe, and reliable service. Once again, I am sorry for the inconvenience this has caused you. Thank you for writing. Sincerely, Linda Dillon
MBCR Customer Service Manager

Holger Levsen: meetbot to help IRC meetings

On monday we held our first Debian Edu IRC meeting with the help of MeetBot. MeetBot is just a very simple bot, actually it's a plugin for eggdrop, which can help with structuring a meeting, publishing logs and writting the minutes. It's purpose is to help a human to chair a meeting, not to chair the meeting on it's own. Because except for publishing the logs it cannot do anything alone :-)

Currently the bot idles in #debian-edu, #debconf-team and #debian-boot, drop me a note if you are interested in trying it in another channel. There's a wishlist and a small howto on it's homepage.

Have fun (with shorter post-processing of meetings! :)

Antti-Juhani Kaijanaho: It s time to fix the ABI

SELinux is entirely correct about disallowing dynamic code generation, as it is a major security risk. Disregarding Just-In-Time compilation, the main legitimate need for dynamic code generation is to support (downward) closures that are ABI-compatible with normal C functions. GCC’s local functions extension of C is one example, and many non-C languages need them badly in their foreign-function interfaces (Haskell is one, Ada I’m told is another). A closure is a function pointer that refers to a function that is local to another function. That function has access to the local variables of the parent function, and this access is provided by having the caller give the callee a static link (a pointer to the parent function’s stack frame) as a hidden parameter. If the call is direct (that is, not through a function pointer), the caller knows the appropriate static link and can just pass it. The trouble comes with function pointers, as we need some way of including the static link in the function pointer. The simplest way is to have function pointers be two words long; one of the words contains the classical function pointer (that is, the entry point address), and the other contains the static link. Unfortunately, the prevalent C ABIs, including the SYSV ABI used by GNU/Linux, mandate that function pointers are one word long. The only way I know to work around this is to dynamically generate, when the function pointer is created, a small snippet of code that loads the correct static link to the appropriate place and jumps to the actual code of the function, and use the address of this snippet (usually called a trampoline) as the function pointer. The snippet is generated on the stack or in the heap, and thus requires an executable stack or executable dynamic memory. It’s time to fix the ABI to allow for proper two-word function pointers.

Clint Adams: I need an admin assistant to glue papers to other papers

buxy is on the wrong track. DSA is a fundamentally-flawed organization; all of the self-selecting committees in Debian are: you cannot have a functional and respectable subgroup if it maintains autonomy like that. The solution to the current set of problems that have been plaguing us for years is not to have backroom discussions and negotiations. It is not to add MORE team members, because that will, at best, achieve minor performance improvements. At worst, that will bolster its structural problems. To fix DSA cleanly and healthily, the honorable members need to resign, and Debian needs to fire the rest. Attempt anything else and you will likely become part of the problem.

Andreas Schuldei: video survailance

Concerning Russels post about the pros and cons of video survailance i saw an interesting article about the evaluation of the berlin subway surveilance. According to that the system did not help to prevent crime, instead people seemed to have considered it when they e.g. created new graffity.

Aparently those results of the evaluation were not very welcome and there were effords to suppress them. This gives me the feeling that there is yet an other agenda to the growing and more and more pervasive survailance. I just wonder what it might be.

In unrelated news blogger.com seems to regard this text as written in swedish despite my selection in the preferences.

Michal &#268;iha&#345;: Gammu test version 1.13.93

I just released new version of Gammu. It just adds compatibility with more OBEX phones by providing fallback to AT commands and index based IrMC access. Full list of changes: PS: I also created Jabber conference room for Gammu, see Wammu support page for details.

Adrian von Bidder: I think I git it.

Since it's the topic of the day, apparently not only for me: Thanks to Michel D nzer for filling in the missing link. Apparently this is one of the things that are so obvious when you're inside the git universe that the User's Manual (which is otherwise very good) didn't make it obvious to those who are not familiar with the subject matter yet. Thanks to madduck for elaborating a bit more on the subject, althoug I've figured out most of this stuff after Michel's hint for myself (but I had not discovered the --all switch to gitk yet.) Summary: what I stumbled about was that the documentation is not explicit enough for me on how to track branches from several remote repos and that the error message is horrible if git can't resolve a ref. So the magic sequence of commands is “clone”, “remote add”, “fetch”, “checkout -b”, and I'm told explicit “--track” is not necessary for remote branches. Logical, but very technical. I'm still not thrilled by the UI of this tool.

Dirk Eddelbuettel: Chicago Marathon 2007

Last Sunday, the 30th Chicago Marathon took place. This was meant to be the year to make this marathon even bigger: the field had increased substantially to 45,000 registrations yet it still sold out way earlier than in previous years. As it turns out, the weather did have its own surprises in store. Earlier in the preceding week, the forecast changed from overcast and rainy to sunny. And sunny it was. While we had a bit of cloud cover at the start at 8:00am, temperatures were already in the 70s and keep increasing. This year's marathon is now on the books as the hottest ever: the clouds dissipated and it was a scorcher. Needless to say, it was a rather challenging race. I finished in 3:41:39, which is my slowest time by some margin for the by now seven marathons I've ran. It was not the day for running fast. Of course, I didn't quite grok that at the start and ran ten miles reasonably hard in a quick pace, but then paid for it, and then paid some more. That said, apparently around 10,000 registered entrants didn't even start, and another 10,000 did not make it to the finish. With the heat, several hundred were treated by the medical teams. Worse still, one 35-year old runner from Michigan died (though the autopsy claims he had a heart condition; other reports say that alone cannot have been lethal). The race itself was aborted, and those who had not reached the half-way point by 12:00am were diverted to the finish and urged to walk rather than run. According to the (by now fairly extensive) news coverage, this whole experience left quite a few people mad and bewildered. As of today, a good 48 hours after the race, the City seems to be in some sort of crisis management mode to prevent damage for the oh-so-important bid for the 2016 Olympics. For some flavour of the news coverage, see an early report, some stunning pictures and some suggestions to prevent another one like this, all from the Chicagoist blog.

Russell Coker: Lintian and Executable Stacks

Debian has a program called Lintian that is used to search for common bugs in Debian packages. When it encounters a package with a shared object that requests an executable stack (as described in my previous post about executable stacks and shared objects [1]) it gives a warning such as the following:
W: liblzo1: shlib-with-executable-stack usr/lib/liblzo.so.1.0.0 Lintian is run automatically on Debian servers and has a web site at http://lintian.debian.org/. You can search the site for all packages which have such executable stacks [2]. Of all the packages listed I have only two installed on my system, liblzo1 and libsmpeg0, both of which I had already discovered and built new versions with the correct stack settings (I’ll publish an APT repository shortly). For the rest I am not sure whether they are really bugs. The ones that concern me are xserver-xorg-video-nsc (we don’t want a stack smashing attack on something as important as an X server) and the C libraries libuclibc0 and dietlibc which may cause many programs to run with an executable stack. The above URL shows that libffcall1 [4] has this problem (as Eddy discovered [5]). Eddy filed Debian bug report 445895 [6] about this problem (I have just updated the bug report with a patch to make it work on i386). Linda (an alternative to Lintian) does not currently warn about this. I have filed Debian bug report 445826 about this [3].

Erinn Clark: Whatever

Here are some random links.

Eddy Petri&#537;or: selinux darcs policy; same for oolite

If you enable and enforce the targeted policy in Debian and you use darcs you need to allow it to use execmem:

chcon -t unconfined_execmem_exec_t /usr/bin/darcs


This allows to overcome these denials:

type=AVC msg=audit(1191957463.678:108): avc: denied execmem for pid=14811 comm="darcs" scontext=user_u:system_r:unconfined_t:s0 tcontext=user_u:system_r:unconfined_t:s0 tclass=process

And get this ugly message:

$ darcs w -l
darcs: internal error: getMBlock: mmap: Permission denied
(GHC version 6.6.1 for x86_64_unknown_linux)
Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
Aborted



Since oolite complained of the same issue I also ran this:
chcon -t unconfined_execmem_exec_t /usr/lib/GNUstep/System/Applications/oolite.app/oolite




Update: If you want to know the reasons why darcs (a VCS) to need execmem read the RedHat bugs related to this and the upstream bug report on GHC: https://bugzilla.redhat.com/show_bug.cgi?id=195820
https://bugzilla.redhat.com/show_bug.cgi?id=195821
http://cvs.haskell.org/trac/ghc/ticket/738

Cyril Brulebois: Linking on hppa

Problem present in openscenegraph 1.9.x and 2.x series: bug #425837. Looking at the build logs of unstable or experimental for the hppa architecture, there's a linking problem around. Shorten error message:
/usr/bin/g++ -fPIC -shared -Wl,-soname,libosgUtil.so.6 
    -o ../../lib/libosgUtil.so.6.0.0 "many *.o"
    -L/build/buildd/openscenegraph-2.2/build/osg/lib  -losg
    -lOpenThreads -lGLU -lGL -lSM -lICE -lX11 -lXext -lm -lpthread
    -Wl,-rpath,/build/buildd/openscenegraph-2.2/build/osg/lib 
/usr/bin/ld: CMakeFiles/osgUtil.dir/Optimizer.o(.text._ZNSt8_Rb_tree 
     IjSt4pairIKjSt3setIPN3osg9Texture2DESt4lessIS5_ESaIS5_EEESt10_S 
     elect1stISA_ES6_IjESaISA_EE9_M_insertEPSt18_Rb_tree_node_baseSH 
     _RKSA_("unmangled name"): cannot reach 0000393f__ZNSt10_Select1 
     stISt4pairIKjSt3setIPN3osg9Texture2DESt4lessIS5_ESaIS5_EEEEC1Ev+0,
    recompile with -ffunction-sections
/usr/bin/ld: final link failed: Bad value
First, just in case demangling is needed (that's not the case, but imagine), there's a nice tool to get the signature back (output skipped, it is very huge, that C++!). That's c++filt from the binutils package:
c++filt _ZNSt8_Rb_treeIjSt4pairIKjSt3setIPN3osg9Texture2DESt4lessIS\
5_ESaIS5_EEESt10_Select1stISA_ES6_IjESaISA_EE9_M_insertEPSt18_Rb_tr\
ee_node_baseSH_RKSA_
Then, one tends to trust the compiler. Added the following to the cmake command in debian/rules:
-D CMAKE_CXX_CFLAGS='-ffunction-sections'
But not much better. Oh wait, there's a quite interesting string in the build log now: cannot handle R_PARISC_PCREL17F, which looks like quite arch-dependant and perfect to be searched through the internets. Some old bugreports came out: bug #334959 and bug #416496 and the solution: use -mlong-calls. From http://www.projectblackdog.com/3rdParty/gcc/HPPA-Options.html, the relevant parts: Generate code that uses long call sequences. This ensures that a call is always able to reach linker generated stubs. [ ] It is normally not desirable to use this option as it will degrade performance. However, it may be useful in large applications, particularly when partial linking is used to build the application. It looks like it might do the job. And indeed it does. Conditionally added for the hppa architecture. And -ffunction-sections isn't even needed. Thanks to the http://www.parisc-linux.org/ folks for the access to their cluster, which really helped finding out a fix very quickly! And it looks like the number of blockers for the transition to testing has been reduced. \o/

Axel Beckert: Group packages by origin in aptitude

I always wondered how others recognise non-Debian packages in the aptitude package tree. I also missed the additional priority level in the hierachy well-known from good old dselect. For the last one, I quickly found out that you can set the priority as subsection — it’s straight forward after you’ve read the documentation: Just add ,priority at the end of the default grouping method for package views under “Options → UI Options” in the aptitude menu. Getting the origin as given in the Release file of the repository a package originates from is a little bit more difficult. You need to use the pattern() group function with the appropriate search pattern: pattern(~O) Since already the default default grouping method for package views doesn’t fit into the dialog, I nowadays just edit /etc/apt/apt.conf directly for changes on aptitude’s default grouping method for package views. It now looks like this on several of my machines:
Aptitude::UI  
  Default-Grouping "filter(missing),status,section(subdir,passthrough),pattern(~O),section(topdir),priority";
 ;
In aptitude this looks like this:
[...]
  --- text - Text processing utilities
  --\ utils - Various system utilities
    --- Debian
    --- Mowgli
    --- volatile.debian.org
  --\ web - Web browsers, servers, proxies, and other tools
    --- Debian
    --- Opera Software ASA
  --\ x11 - The X window system and related software
    --\ Debian
      --- contrib - Programs which depend on software not in Debian
      --\ main - The main Debian archive
        --- Priority optional
        --- Priority extra
      --- non-free - Programs which are not free software
    --- Mowgli
[...]
Unfortunately this doesn’t work with all non-Debian repositories since a few repository maintainer, e.g. those from Emdebian, arrogate to just keep “Debian” as their packages’ origin. This could be solved, if there’s a possibility to group by e.g. repository URL (host and/or path). Another problem I haven’t solved yet is that grouping by origin does neither work with locally created nor virtual packages nor tasks — probably since all of them lack an origin. Those branches are just empty or don’t even show up anymore with this configuration. I probably have to dig a little bit more in the aptitude documentation to resolve this.

Now playing: E-Rotic — Max don’t have sex with your ex

9 October 2007

Axel Beckert: Plugins in the Blosxom Project CVS

Since yesterday, my Blosxom plugins are versioned in the Blosxom Project CVS repository together with those of most other Blosxom developers. Cause for this is, that — besides first steps towards Blosxom v4 (we better forget about v3… ;-) and intergrating existing patches (e.g. the Debian config file patch) to Blosxom v2 — the Blosxom developers want to release a Collection of common Blosxom Plugins as a Plugin distribution so that no one needs to gather the often needed plugins from various sites on the net but get them from first hand and also in some kind of a supported way. A first release candidate is on it’s way. And for those who thought good ol’ blosxom is dead: There never was so much traffic on the blosxom developer’s list like in the past two months — over 160 mails each!

Now playing: Eiffel 65 — Blue

Next.

Previous.