Search Results: "tar"

16 October 2007

Biella Coleman: Topics in Digital Media

Next semester I am teaching one (yay one) course and it is a graduate course on digital media. This is the description I have so far and I will post the tentative syllabus in the next few week.
Computers, especially in their networked dimension, have sparked a series of ethical, political, and social debates that often revolve around a series of stark and connected dualities: control and freedom; pleasure and exploitation; creativity and constraint. In this course we will approach topics in digital media via an historical angle that squarely addresses these dualities. To this end, we will often cross-cut readings on similar topics and material whose conclusion about the nature of computing will often vary considerably. The goal, however, is not to determine the correct or right side of these dualities but have students come away with a firm understanding of the following: 1) the history of computing and networking in light of the ways the authors as well as technologists/inventors construct or understand these dualities; 2) the various sources technological, social, and political that may shape or drive any of these elements; 3) unpack the political and social relationships, if any, between them and the stakes involved in how these authors represent the nature of computing and networking. The course primarily concentrates on computers and networks and is roughly chronological, starting with the first digital computers and ending with our digital present. Particular topics we address are: cybernetics and liberalism; networks and the cold war; personal computers and online communities; hackers, the free software movement and intellectual property; labor, development, and computers; peer-to-peer knowledge production; computer gaming; and counter-globalization and computer networking.

Neil Williams: A little humour from dpkg

This arises from upgrading a cross-built package where the prerm maintainer script wanted to use install-info and as Emdebian doesn't use info or man pages, the prerm script failed. I'd already removed the same from postinst but got this little comment from dpkg when I installed an upgraded package on the Balloon3 ARM board:
dpkg: warning - old pre-removal script returned error exit status 127
dpkg - trying script from the new package instead ...
dpkg: ... it looks like that went OK.

Nice trick!

Emdebian postinst patch.

This is the full apt output:

/ # apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages have been kept back:
apt
The following packages will be upgraded:
busybox libreadline5 readline-common
3 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
Need to get 563kB of archives.
After unpacking 12.3kB disk space will be freed.
Do you want to continue [Y/n]?
Get:1 http://buildd.emdebian.org unstable/main readline-common 5.2-3em3 [1882B]
Get:2 http://buildd.emdebian.org unstable/main libreadline5 5.2-3em3 [249kB]
Get:3 http://buildd.emdebian.org unstable/main busybox 1:1.1.3-5em2 [312kB]
Fetched 563kB in 7s (70.5kB/s)
(Reading database ... 489 files and directories currently installed.)
Preparing to replace readline-common 5.2-3em2 (using .../readline-common_5.2-3em3_all.deb) ...
/var/lib/dpkg/info/readline-common.prerm: /var/lib/dpkg/info/readline-common.prerm: 3: install-info: not found
dpkg: warning - old pre-removal script returned error exit status 127
dpkg - trying script from the new package instead ...
dpkg: ... it looks like that went OK.
Unpacking replacement readline-common ...
Preparing to replace libreadline5 5.2-3em2 (using .../libreadline5_5.2-3em3_arm.deb) ...
Unpacking replacement libreadline5 ...
Preparing to replace busybox 1:1.1.3-5em1 (using .../busybox_1%3a1.1.3-5em2_arm.deb) ...
Unpacking replacement busybox ...
Setting up readline-common (5.2-3em3) ...
Setting up libreadline5 (5.2-3em3) ...
Setting up busybox (1:1.1.3-5em2) ...

This is inside a chroot on the Balloon3 board using cross-built packages from the Emdebian target repository. The chroot was created with the 'emsandbox' utility from emdebian-tools 0.4.1 (due for upload to Emdebian tonight) and a little manual help. A few more tweaks to the relevant packages are needed before the chroot (and therefore a root filesystem for Emdebian) can be installed without assistance.

Benjamin Mako Hill: What I'm Up To

So, I finished graduate school at MIT. I presented some of my thesis work at Wikimania and I'll be posting information, code, and the thesis itself, in the next weeks as I find time. I've decided to focus, at least in the immediate future, on several important projects. Here's what I'm up to: This is, of course, in addition to my work with FSF, Debian, and Ubuntu which I'll be continuing. And talks. And writing. (Yikes!) I'll be keeping my office at MIT (yes, like RMS) for the time being and sticking around Cambridge at least until Mika finishes her degree at Harvard School of Public Health. I'll be in wrapping up projects modes for the next few weeks and months and will be posting about them here as I go.

Bdale Garbee: Colorado Rockies in World Series!

I'm not a serious fan of professional sports, but over the years, I've enjoyed watching a wide variety of sporting events at different times and for different reasons. It can be fun when a relatively local team is doing well, and even more fun when they're doing well with style. For the last week or so, I've been paying attention to Major League Baseball for the first time in several years. Last night, from a hotel room on my current business trip, I watched the Colorado Rockies baseball team take the National League championship, thereby winning the right to compete in this year's World Series. It's pretty amazing that they've now won 21 of their last 22 games, including seven straight in the playoffs. But there are two other things that captured my attention last night prompting me to comment here. The first is that this really has been a team effort. With due respect to the various players on the team, none of them seem to be superstars... instead, they come across as a group of very good players who are just doing their jobs really well. And second, they really do seem to be having fun and aren't taking themselves too seriously. I don't know what's going to happen next. They may win, they may lose. But those two observations last night cheered me up after a fairly stressful day, and left me encouraged that doing what you love as well as you can, and finding others to work together with who care as much about the cause at hand as you do can still lead to extraordinary results. That's pretty cool.

Marc 'Zugschlus' Haber: A thousand things I never wanted to know about X

In the last few days, I have replaced the two 20 inch CRT monitors that I have hardly used the last years with two 20 inch TFT displays, and my company (finally) gave me a 19 inch TFT display to accompany my notebook display at work. Maybe I should take that as a hint that they want to see me in the office more frequently rather than in my home office which I generously use these days. At home, I built a “new” computer from mainly used parts to drive the two 20 inchers. I have learned a lot about X in the last days, but spent too much time with it. I have been running my Desktop with older X and two CRTs for quite some time, but I hardly ever used it due to the setup of my now ex-flat. Back then, I learned that the “two monitors, one display, one screen, one desktop” option of X was called Xinerama and statically configured: You write down your display layout in xorg.conf and it uses it. Configuration changes mean restarting the X server. KDE notices Xinerama automatically, designates one monitor the “main monitor” and displays its panel there. You can move windows from one monitor to the other one, but changing virtual desktops always changes what both monitors display. This is not really what I want, since I often have “static” contents (news reader, mail reader, chat windows, help screens etc) on one monitor while I do my normal work on the other monitor, which involves changing virtual desktops. So I was rather happy when I learned upon the arrival of the external TFT display for the notebook that you can work without Xinerama. In that case, the X server comes up with one display and two screens, which KDE uses to display two independent desktops which can be independently configured, but can both be accessed with one mouse and one keyboard. For me, this means that I finally can independently change virtual desktops on both monitors, but pay the price that I cannot display a certain virtual desktop on monitor 1 now, and on monitor 2 in five minutes, and that a window opened on monitor 1 is never going to be shown on monitor 2 without serious interference. This looked ok for me. Additionally, you can use different font sizes on the different monitors as defaults, which comes in handy if both monitors are blatantly different in size (which is the case with the notebook and its external monitor). However, with that setup, KDE seems to be challenged when it comes to saving the configuration. Some things (such as desktop background and panel config) seem to save fine, desktop independent, other things (such as open windows and their position) are lost when the session exitst. My daily visit to Kevin&Kell (which used to be in a browser window that was saved with my session) has badly suffered since then. Then, a few weeks ago, somebody talked me into trying the X server from experimental. Which is when my dual-desktop setup ceased working. The nice people on #debian-x told me that Xinerama is a thing of the past, and is currently in the process of being replaced by Xrandr 1.2. The removal of “my” two monitor, two screens, two desktop setup is collateral damage, and if I still want that behavior, the X.org upstream considers this an issue to be handled by the Window Manager / Desktop Environment now. Gee, thanks. That’s what I call a regression. After spending half a day cursing and ranting, and even considering to go back to lenny’s (or even etch’s) X.org, I finally settled on going with the “new” way as I was going to lose my independent desktops sooner or later anyway. Again, with the friendly help of the people on #debian-x, I found out about a lot of advantages of the latest X servers. In most basic cases, the latest X server does not need a configuration file any more. It autodetects your hardware and chooses whatever modes it finds optimal. It does a pretty decent job in doing so. If one wants to influence how the X system comes up, once can always write an Xorg.conf file - the configfileless server even writes the “virtual config file” it uses to the log as a starting point. Additionally, almost everything regarding screen layout can be reconfigured at run time using the xrandr binary from a command line. Even a hardware change (such a new monitor plugged into the powered on notebook) is correctly detected. Cute. This allows me to reconfigure my notebook depending on where it currently is and which external monitor (or projector) is connected. This can be perfected if I find out whether X offers a hook to plug a reconfiguration script into that is called when the X server starts or when the system is waking up from standby or hibernation with the X server running. A lot to do, and great potential for elegant stuff. However, while I am talking about hibernation, this is a downside of the new driver: When the system wakes up from hibernation, the X server sometimes confuses itself enough to become nearly unuseable: I had it become psychotic when plugging in the external monitor (which can do 1600x1200 points) after waking it up from hibernation at the office. The internal display can only do 1400x1050, and the box thought that the external monitor only had 1050 y axis pixels, displaying bit garbage in the lower part of the big monitor, and showing a useable, but unintelligible panel. When opening any browser or any other application with considerable text amounts in the window, all fonts become unreadable once one scrolls down. Restarting the X server helps and everyting is fine. But I do not use hibernation to force myself to log out of X after waking up. After going back to the “one monitor mode”, the X server now thinks that my virtual desktop is 1600x1200 and only shows 1400x1050, with dead space at the right and lower side of the virtual desktop. xrandr does not allow me to reisize the FB below 1600x1200. No idea what’s going on here. In most of the cases, calling xrandr --auto for each output will fix the issues at run time, but sometimes a session restart is needed. Code needs to stabilize quite a bit before becoming useable in production. There are a few disadvantages that will remain: Both monitors will change their contents when the virtual desktop is changed, and both displays will use the same default font size (which might be inappropriate if both displays are of different size). But I think, when the issues are cleared up, that I can live with these disadvantages. The “new” desktop system that I built has a Matrox G450 dual head graphics card, and unfortunately, the Xrandr 1.2 enabled X server from experiemental has shown to be unuseable: Upon some mode changes, the X server goes into an endless loop and proves itself to be unkillable short of SIGKILL. And if it dies, it sometimes pulls the entire system with it. There is a corresponding bug in the upstream BTS, but it hasn’t received any noticeable attention by the upstream developers. So I might find myself configuring one last Xinerama config before I can finally completly migrate to Xrandr 1.2 on the desktop as well. Then I’ll probably need to bug the KDE upstream people to implement a two-desktops-on-a-single-screen feature in one of their future versions so that I can get my independent-desktop-things back. But that’s a far far away future.

Ross Burton: So Sad

Last week just after we got back from holiday our washing machine broke down. The washing machine repair man had a look, and said it's best just to buy a new one -- to replace the main broken part would be £150 and that is without looking at fixing the leaks and the broken filter... Saturday we went down the road armed with a Which? report on washing machines and a price list from QED. After a little haggling ("When I say price match I mean to high street shops...") I splashed out on a new Bosch Exxcel 2465 . This is very sad, but I was getting excited over a washing machine... 20 minute "freshen" cycles, delayed start, cold "handwash" cycles. It rocks. :)

Ross Burton: Mark Thomas

A couple of nights ago (Tuesday, the 8th) I went to see a live Mark Thomas show. A very enjoyable show, he is wonderful at telling stories, making the entire audience collapse with laughter, and then can suddenly make the entire room silent with a serious point about the terrors the Western world manages to inflict on other nations. Lots of interesting facts, like the poor drunk American who is in prison for three years for mentioning the "day of judgement" and "a burning Bush". The number of people dying daily due to lack of action from us. The propaganda about the current war, forced in our faces every second of every day, until the war is over (as far as the media is concerned), at which point it will be ignored and never mentioned again. I am now a proud white ribbon wearer, supporting How About Not Killing Everyone? A small piece of advice from the wise -- if you see MT live, and he starts to talk about John Major and Edwina Curry... look away. Quickly. You shall be haunted for the rest of your days if you do not.

Raphael Geissert: Print before you think!

What?! I mean... Think before you print!.

It's not very uncommon to see people who always print almost every single email they receive with information (and not just a notification), pdf's, and many other kind of documents.

I've never liked when people do that because they don't just print it, they sometimes even print the whole email conversation (usually gmail users). Some of them even dare to use non-recycled paper for printing some text and trashing it away after reading it!

Today is the Blog Action Day about the environment, so all I want to say is: reconsider, reuse and recycle (yes, in that specific order :D ). I know it's not the classic reduce, reuse, recycle message because I think people should better reconsider and not only reduce.

Here's a nice reminder you can, err, use on your computer's desktop (did you think I was going to say print??, I wasn't):


If you don't like that one, Google will point you to some others.

15 October 2007

Steve Kemp: She couldn't quite explain it

Previously I wrote about two "bugs" with GNU Screen. Now I discover, via a customer, that one of the servers I setup had allowed shell access when it should have been prohibited. A potentially serious security hole in this context. Here is an example, with this small .screenrc file:
#
#  Normally "ctrl-a c" would open a new "window".
#
#  We wish to prohibit that
#
bind c
Start GNU screen. Press "Ctrl-a c" nothing happens. All looks well, you've denied a new window! If you'd setup a shell in /etc/passwd to point to a wrapper which invoked GNU Screen with this configuration file you'd be fine, right? Actually not. Try pressing "Ctrl-a" and whilst those two keys are held down press and release "c". Joy. New window created. Even though it shouldn't be. Update - Turns out I'm stupid "ctrl-a c" is the same as "ctrl-a ctrl-c" by default. I didn't add "bind ^c" appropriately. My bad. For reference if you want to stop shell creation and you invoke screen as a login shell as a wrapper for other things then you must unbind a lot of builtins. eg. "bind :". You should then set:
shell /bin/false
Only then will you be secure. Probably. Thankfully this doesn't affect the Xen shell.

David Moreno Garza: Spectacularly beautiful woman seeking non-douchebag man

So this is New York. Or not.

Martin F. Krafft: CO and the environment

Today is environment blog action day. I don't like memes or mass actions of this sort, but given a recent discussion on the effects of Al Gore's movie An Inconvenient Truth and everyday/everyone environmental protection on a UL-internal discussion list called "opinions", I feel like sharing my views. The original poster suggested everyone to see the movie and was promptly smashed down by two people who responded along the lines of "the movie has no scientific basis," quoting fancy research on how global warming is not as bad as it's hyped to be, and that CO is not as guilty as everyone claims. None of this research is relevant to what I had to say to this, so I won't quote it, but if you're interested, the Intergovernmental Panel on Climate Change (IPCC) is the place to start. One person went further and compared the CO -reduction hype to communism and criticised how we're expecting everyone to take action on the basis of false claims. I was partly disgusted by (all) those replies, not because they weren't accurate, but because they simply miss the point. And because the entire discussion was just another iteration of talk versus action. It may well be true that CO is not that bad, after all. But it's also true that any reasonable attempt to reduce its emission is not going to harm the environment (not to say that it might actually have a net-positive effect). Or as I put it in the thread:
I don't care for politics, I don't even care for research on this matter. I care for my environment, and I don't need to be a scientist to assume that plastic bags and car exhaust and lights left on when out to the pub are not making it any better. And yet, I am not a militant green. I prefer halogen light bulbs to the energy-saving ones and occasionally I'll be at the supermarket without a reusable bag; I don't condemn people using cars instead of a 15 minute walk to the office, and I still haven't managed to talk to a housemate of mine (no, not my Swiss flatmate), who notoriously leaves lights and appliances running when away. But at least I can say that I think of these issues, without scientific basis, but common sense. And I'll say that a thousand people aware of that will make more of a difference than any politician out there.
Not only do I think about these issues, I also act accordingly. But to be able to do that, I had to become aware of them. In as such, I'll stand by the claim in the discussion: awareness is what we ought to strive for, rather than talking about possible ways to start considering how we could potentially one day make leaps towards the solution of the world's problems, of which the environment is one. Awareness and little steps, which everyone can take: turn off lights and appliances, turn down the heater, use reusable shopping bags, prefer local produce to imported products, avoid fancy packaging, and so on In the thread, I asked "What have you done today to help lessen the burden we place on our environment?" I got no replies. I hope that's because people couldn't be bothered to show off their little contributions to the environment. If that were the case, then we are on the right path. NP: Gazpacho: Night

Russell Coker: Blog Copyright Infringement

I have previously written about some of my efforts to counter sploggers [1]. Since then I have had a particularly brazen splogger copy one of my posts entirely and claim to have written it. The only reason I noticed the copyright violation (my blog license is on my About Page [2]) was because the post in question linked to other posts of mine and I saw the links. I was offended by the flagrant violation of all aspects of copyright law (breaking the license and infringing my moral rights by not attributing me as the author) and by the fact that the splog in question was hosted by Dreamhost (who have offended me by refusing a DMCA take-down request). So I decided that merely issuing a DMCA take-down was not enough. I went through the splog and identified content copied from several major journals (including by a journalist I regard as a friend) as well as by one multi-national corporation - and I notified all the relevant people. The splog in question deleted all it’s old content the next day, and immediately started copying new articles from other blogs. I have informed the people who appear to be copyright holders for some of the new articles… I recommend that other people who deal with sploggers also go to the extra effort of notifying other victims. It’s usually quite easy to do you just select a random bit of text from the copied article and paste it into your favourite search engine - usually you get only a single result. Some of the splog posts are edited in small ways so the first search may fail - if so then you merely need to search for a second piece of text. If you only request that your own illegally copied material be taken down then the splogger still has a good business model. They can keep copying content in violation of the license, occasionally take a post down when they get caught, and both the splogger and the ISP continue to make money. If you notify other victims (many of which won’t have the skills to find the content themselves or the background knowledge sufficient to recognise the benefits in having it removed unless you explain it to them) then the splogger loses a lot of content at one go and the ISP will have a more difficult time claiming to be innocent of the process. Also when you notify multi-national corporations you can expect that they have some decent lawyers and a budget assigned to such work. While I would be extremely unlikely to sue an ISP that repeatedly hosts unauthorised copies of my copyright materiel the same can’t be said for a corporation. For more information on splogging see the Wikipedia entry [3].

Benjamin Mako Hill: Comical Gastronomical

When it comes to puns about digestion, Dafydd Harries and I make up for in quantity what we lack in quality. Apologies to Daf if he is embarassed by my public acknowledgement of his contributions in this endeavor.

14 October 2007

Martin F. Krafft: Converting a package to Git

Previously, I demonstrated a Debian packaging workflow using Git and I mentioned the possibility of a follow-up post; well, here it is: you want to use my workflow (or one that's related) for a package that is currently maintained with Subversion on svn.debian.org and you'd like to keep the history during the conversion. Make sure to read the previous post before this one. I am again using the example of mdadm since its Git packaging repository is in a state of shambles and I want to restart to get it right and import the history from the previous Subversion repository. What better way than to write a blog post as I do so? Well, plenty actually. This kind of post isn't really made for a blog, and I have started work on setting up ikiwiki on madduck.net, but it's not yet ready, so I'll stick with the blog for now. I will make sure that links don't break as I move content over, so feel free to bookmark this
Importing the package into Git Thanks to git-svn, the initial step of getting your package imported into Git is a breeze:
$ git-svn clone --stdlayout --no-metadata \
    svn+ssh://svn.debian.org/svn/pkg-mdadm/mdadm mdadm
Sit back and enjoy. If that command exits prematurely with an error such as the following:
Malformed network data: Malformed network data at /usr/local/bin/git-svn line 1029
then you should upgrade to a newer Git version, or have a look here. If your Git does not know --stdlayout then upgrade as well (or use -T trunk -t tags -b branches instead). Sam Vilain notes that it is important to "get the attribution right with the final SVN import - getting the authors map right. I didn't do that. If you look at the repository resulting from the above command, you'll notice strange commit authors, such as madduck@some-unique-uuid-from-svn . git-svn allows you to map these to real names with real email addresses, which ensures that the attributions are good for the whole world to see. When done, switch to the repository and run git-branch -r. As you'll see, git-svn imported all SVN branches and tags as remote branches. You need those if you want to bidirectionally track the Subversion repository, but we are converting, as you may have guessed by the --no-metadata switch above. Therefore, we resort to the Dinosaur method of converting branches to tags, which I'll simplify for mdadm. We also just delete all remote branches after tagging, since mdadm never used branches in the SVN repository. Your mileage may vary.
git branch -r   sed -rne 's, *tags/([^@]+)$,\1,p'   while read tag; do
  echo "git tag debian/$tag tags/$ tag ^; git branch -r -d tags/$tag"
done
git branch -r   while read tag; do
  echo "git branch -r -d $tag"
done
If that seems to work alright, then you can execute the commands. Sam Vilain (again) hints me at git-pack-refs and then to edit .git/packed-refs with an editor. This certainly leaves more room for errors but might be significantly faster.
Cleaning up the SVN references Even though we passed --no-metadata to git-svn, it did leave some traces in .git/, which we can now safely remove:
$ git config --remove-section svn-remote.svn
$ rm -r .git/svn
Setting things straight You can skip this section unless you want to know a bit about how to fix up stuff with Git. There was actually some nasty tagging errors leading up to the 2.5.6-9 release for etch and I could never be bothered to fix those in SVN, but now I can (I love Git!):
$ git tag -d debian/2.5.6-10            # never existed
$ git tag -f debian/2.5.6-8 2.5.6-8~2   # mistagged
$ git checkout -b maint/etch 2.5.6-8    # this is when we diverged
$ git apply < /tmp/mdadm-2.5.6-8..2.5.6-9.diff
$ git add debian/po/gl.po debian/po/pt.po debian/changelog
$ git commit -s
$ git tag debian/2.5.6-9
Now that that's fixed, there is one other thing to worry about, namely the very last commit to SVN, which obsoletes the repository and points to the Git repository. But that's not all of it. I was also silly enough to include a fix in the same commit. Let's see what Git can do. Since the process of obsoletion involves all but adding a file, we can simply --amend the last commit and provide a new log message:
$ git checkout master
$ git rm OBSOLETE debian/OBSOLETE
$ git commit --amend
Now the repository is in an acceptable state.
Making ends meet The pkg-mdadm effort on svn.debian.org only maintained the ./debian/ directory, separate from the upstream code, and boy was that a bad idea. Just to give one example: think about what's involved in preparing a Debian-specific patch against the upstream code this has to end, and we can make it end right here; let's import upstream's code (again not using his ADSL line, but the upstream branch of the pkg-mdadm Git repository; see the previous post for details):
$ git remote add upstream-repo git://git.debian.org/git/pkg-mdadm/mdadm
$ git config remote.upstream-repo.fetch \
    +refs/heads/upstream:refs/remotes/upstream-repo/upstream
$ git fetch upstream-repo
$ git checkout -b upstream upstream-repo/master
Now we have two unconnected ancestries in our repository, and it's time to join them together. The most logical way seems to be to use the last upstream tag for which we have a Debian tag: 2.6.2. For this, we branch off the corresponding Debian tag (2.6.2-1) and merge upstream's 2.6.2 tag into the new branch. This will be a temporary branch Then, we rebase (remember, nothing has been published yet) the master branch on top of this temporary branch, before we end that branch's short life. The Debian tag stays where it is since it describes the state of the repository at time of the release of 2.6.2-1.
$ git checkout -b tmp/join debian/2.6.2-1
$ git merge mdadm-2.6.2
$ git rebase tmp/join master
$ git branch -d tmp/join
It just so happens that the head of the SVN repository, which is identical to the tip of our master branch, corresponds to Debian release 2.6.2-2, so we tag it:
$ git tag debian/2.6.2-2
We are now also "born" in the sense that maintenance in Git has started. Let's mark that point in history. There is no real reason I can foresee for this yet, but nonetheless:
$ git tag -s git-birth
Turning dpatch files into feature branches We want to turn dpatch files into feature branches and we somehow make it "proper". We could branch, apply the patch, delete the patch file, checkout master and delete the patch file there as well, but that appears "improper" to me at least; so instead, we'll cherry-pick:
$ git checkout -b deb/conffile-location
$ debian/patches/01-mdadm.conf-location.dpatch -apply
$ git rm debian/patches/01-mdadm.conf-location.dpatch
$ git commit -s
$ git commit -s $(git ls-files --others --modified)
I should quickly intervene to make sure you are following. I am making use of Git's index here. Applying the patch makes the changes in the working tree, but we did not tell Git that we want those to be part of the commit just yet. Instead, we delete the dpatch with git-rm, which automatically registers the deletion with the index. Thus, the first git-commit creates a commit which deletes the dpatch, while the second git-commit creates a commit with all the changes from the dpatch, using git-ls-files to identify new and modified files. But for now, let's move on. We have two commits in the deb/conffile-location branch, and one of those is relevant to the master branch, we cherry-pick it:
$ git cherry-pick deb/conffile-location^
If you're confused, let me explain: our goal is to have a number of feature branches, of which master is the one in which most of ./debian/ is maintained. All the branches later come together in the long-living build branch, so deb/conffile-location will never be merged back into master. However, once we applied the dpatch to the feature branch, we can delete it from there and the master branch. By cherry-picking, we "import" the deletion to the master branch. I repeat the same procedure for deb/docs, merging all the documentation-related dpatches, but I'll spare you the details.
and then Git let me down In the next step, I found I had misunderstood Git merging: I thought Git was smart, but Linus had his reasons for calling Git the "stupid content tracker" (more on that later). Read on as I am obsoleting dpatch files that upstream had merged: 99-*-FIX.dpatch. For consistency, I wanted to cherry-pick each of the appropriate upstream commits into the master branch along with deleting the corresponding dpatch file. Here is one example: 99-monitor-6+10-FIX.dpatch was obsoleted by upstream's commit 66f8bbb; the -x records the original commit ID in the log:
$ git cherry-pick -x 66f8bbb
$ git rm debian/patches/99-monitor-6+10-FIX.dpatch
$ git commit -s -m"remove dpatch obsoleted by $(git rev-parse --short HEAD)"
I repeated the procedure for the other dpatch files, removed the dpatch infrastructure, and then went on to merge it all into build to build the package. The build branch is a long-living branch off upstream, but which upstream? I'll fast-forward you past a segfault problem with mdadm, which upstream (thought to have) resolved with commit 23dc1ae after 2.6.3, but he had not yet released 2.6.4. Looking at the commits between 23dc1ae and upstream's HEAD at the time, I decided to include them all and snapshot 4450e59:
$ git fetch upstream-repo
$ git checkout upstream
$ git merge upstream-repo
$ git tag mdadm-2.6.3+200709292116+4450e59 4450e59
$ git checkout master
$ git merge --no-commit mdadm-2.6.3+200709292116+4450e59
$ dch -v mdadm-2.6.3+200709292116+4450e59-1
$ git add debian/changelog
$ git commit -s
And then I called poor-mans-gitbuild, which merges master and then deb/* into build. Here is when stuff blew up. I'll make a long story short (read my description of the problem and Linus' answer if you want to know more): I thought Git was smart to identify merges common to both branches and do the right thing, but it turn out that Git does not care at all about commits, it only worries about content and the end result. In our case, unfortunately (or fortunately), the outcome meant a conflict because the upstream branch introduced a simple change (last hunk) in the lines surrounding the patch we cherry-picked, and Git can't handle it. The solution is not to cherry-pick, to cherry-pick all commits touching the context of the dpatch, or to simply merge upstream into all out feature branches. In our case, the first is the easiest solution and since importing dpatch files is a one-time thing (thank $DEITY), I'll leave it at that. Almost. I have spent two days thinking about this more than I should have. And it was this point Linus made which made me appreciate Git even more:
Conflicts aren't bad - they're good. Trying to aggressively resolve them automatically when two branches have done slightly different things in the same area is stupid and just results in more problems. Instead, git tries to do what I don't think anybody else has done: make the conflicts easy to resolve, by allowing you to work with them in your normal working tree, and still giving you a lot of tools to help you see what's going on.
The end This concludes today's report. Importing the changes from the old Git repo, tagging and merging the branches is all covered in my previous post, or at least you'll find enough information there to complete the exercise. I would like to specifically thank Sam Vilain and Linus Torvalds for their help in preparing this post, as well as the #git/freenode inhabitants, as always. If you are interested in the topic of using version control for distro packaging, I invite you to join the vcs-pkg mailing list and/or the #vcs-pkg/irc.oftc.net IRC channel. Also, if you are interested in Git in general, you can find a list of blog posts on the Git wiki. NP: The Police: Zenyatta Mondatta

Russell Coker: AUUG 2007

Today was the final day of the AUUG 2007 conference [1]. Yesterday I gave a talk about SE Linux for about an hour (not sure exactly as I forgot to make an MP3). AUUG is well known for having conferences with very technical delegates and I wasn’t expecting an easy audience. At the start of my talk I asked for a show of hands as to who has used SE Linux before, about 1/3 of the delegates raised their hand. Someone requested that I poll the audience as to who had used SE Linux involuntarily, it wasn’t what I had planned to ask but it’s best to get these things out in the open so I asked the question. More people raised their hand as being unwilling users of SE Linux than those who had firstly admitted to using it! A theme of the AUUG conference was quality, and I had planned to cover some of the ways that SE Linux improves the quality of code by making certain classes of bugs show up (EG file handle leaks) and by allowing the developers and sys-admins to know exactly what programs are doing. But I ended up explaining why you want to use SE Linux, the concepts of policy analysis tools (as compared to the absence of such tools for Unix permissions), the benefits of MAC and why SE Linux is worth using. I believe that the talk did some good and conversations with delegates afterwards revealed that some of them had done some positive things with SE Linux. Today I wore a T-shirt advertising the root password for my new SE Linux Play Machine [2] which will be online shortly (hopefully tomorrow) which got some interest (AFAIK I’m the first person to wear a root password on a T-shirt). When I have my play machine online I plan to wear the shirt whenever I visit an electronics store or any other location where geeks are likely to congregate. ;) After the conference finished about 1/3 of the delegates went to Ginza Teppanyaki [3] for dinner. Some of the guys wanted to photograph me wearing my shirt. Finally, the conference went pretty well, the delegates and speakers all seemed to enjoy themselves and learn some useful things. Congratulations to the AUUG conference organisers!

Mike Hommey: Adding some VCS information in bash prompt

I don’t spend a lot of time customizing my “working” environment nowadays, like enhancing vim configuration, or tweaking the shell. But when I read MadCoder’s zsh git-enabled prompt, I though it was too convenient to not have something like that. Except I don’t work with git only (sadly, but that’s changing), and I don’t like colours in prompt (and a 2 lines prompt is too much). Anyways, since I have a bunch of directories in my $HOME that contain either svk, svn, mercurial, or git working trees, I thought it would be nice to have some information about all this on my prompt. After a few iterations, here are the sample results:
mh@namakemono:~/dd/packages$
mh@namakemono:(svn)~/dd/packages/iceape[trunk:39972]debian/patches$
mh@namakemono:(svk)~/dd/packages/libxml2[trunk:1308]include/libxml$
mh@namakemono:(hg)~/moz/cvs-trunk-mirror[default]uriloader/exthandler$
mh@namakemono:(git)~/git/webkit[debian]JavaScriptCore/wtf$
The script follows, with a bit of explanation intertwined.
_bold=$(tput bold)
_normal=$(tput sgr0)
tput is a tool I only dicovered recently, and avoids the need to know the escape codes. There are also options for cursor placement, colours, etc. It lies in the ncurses-bin package, if you want to play with it.
__vcs_dir()
  local vcs base_dir sub_dir ref
  sub_dir()
    local sub_dir
    sub_dir=$(readlink -f "$ PWD ")
    sub_dir=$ sub_dir#$1
    echo $ sub_dir#/
  
We declare as much as possible as local (even functions), so that we avoid cluttering the whole environment. sub_dir is going to be used in several places below, which is why we declare it as a function. It outputs the current directory, relative to the directory given as argument.
  git_dir()
    base_dir=$(git-rev-parse --show-cdup 2>/dev/null) return 1
    base_dir=$(readlink -f "$base_dir/..")
    sub_dir=$(git-rev-parse --show-prefix)
    sub_dir=$ sub_dir%/
    ref=$(git-symbolic-ref -q HEAD git-name-rev --name-only HEAD 2>/dev/null)
    ref=$ ref#refs/heads/
    vcs="git"
  
This is the first function to detect a working tree, for git this time. Each of these functions set the 4 variables we declared earlier: vcs, base_dir, sub_dir and ref. They are, respectively, the VCS type, the top-level directory of the working tree, the current directory, relative to base_dir, and the branch, revision or a reference in the repository, depending on the VCS in use. These functions return 1 if the current directory is not in a working tree of the currently considered VCS.
The base directory of a git working tree can be deduced from the result of git-rev-parse --show-cdup, which gives the way up to the top-level directory, relative to the current directory. readlink -f then gives the canonical top-level directory. The current directory, relative to the top-level, is simply given by git-rev-parse --show-prefix.
git-name-rev --name-only HEAD gives a nice reference for the current HEAD, especially if you’re on a detached head. But this can turn out to do a lot of work, introducing a slight lag when you cd for the first time in the git working tree, while most of the time, the HEAD is just a symbolic ref. This is why we first try git-symbolic-ref --name-only HEAD.
  svn_dir()
    [ -d ".svn" ] return 1
    base_dir="."
    while [ -d "$base_dir/../.svn" ]; do base_dir="$base_dir/.."; done
    base_dir=$(readlink -f "$base_dir")
    sub_dir=$(sub_dir "$ base_dir ")
    ref=$(svn info "$base_dir" awk '/^URL/ sub(".*/","",$0); r=$0 /^Revision/ sub("[^0-9]*","",$0); print r":"$0 ')
    vcs="svn"
  
Detecting an svn working tree is easier : it contains a .svn directory, be it top-level or sub directory. We look up the top-level directory by checking the last directory containing a .svn sub directory on the way up. This obviously doesn’t work if you checkout under another svn working tree, but I don’t do such things.
For the ref, I wanted something like the name of the directory that has been checked out at the top-level directory (usually “trunk” or a branch name), followed by the revision number.
  svk_dir()
    [ -f ~/.svk/config ] return 1
    base_dir=$(awk '/: *$/ sub(/^ */,"",$0); sub(/: *$/,"",$0); if (match("'$ PWD '", $0"(/ $)")) print $0; d=1; /depotpath/ && d == 1 sub(".*/","",$0); r=$0 /revision/ && d == 1 print r ":" $2; exit 1 ' ~/.svk/config) && return 1
    ref=$ base_dir##*

    base_dir=$ base_dir%%
*
    sub_dir=$(sub_dir "$ base_dir ")
    vcs="svk"
  
svk doesn’t have repository files in the working tree, so we would have to ask svk itself if the current directory is a working tree. Unfortunately, svk is quite slow at that (not that it takes several seconds, but that induces a noticeable delay to display the prompt), so we have to parse its config file by ourselves. We avoid running awk twice by outputing both the informations we are looking for, separated by a carriage return, and then do some tricks with bash variable expansion.
  hg_dir()
    base_dir="."
    while [ ! -d "$base_dir/.hg" ]; do base_dir="$base_dir/.."; [ $(readlink -f "$ base_dir ") = "/" ] && return 1; done
    base_dir=$(readlink -f "$base_dir")
    sub_dir=$(sub_dir "$ base_dir ")
    ref=$(< "$ base_dir /.hg/branch")
    vcs="hg"
  
I don’t use mercurial much, but I happen to have exactly one working tree (a clone of http://hg.mozilla.org/cvs-trunk-mirror/), so I got some basic information. There is no way we can ask mercurial itself for information, it is too slow for that (main culprit being the python interpreter startup), so we take the informations we can (and since I don’t know much about mercurial, that’s really basic). Note that if you’re deep in the VFS tree, but not in a mercurial working tree, the while loop may be slow. I didn’t bother much looking for a better solution.
  git_dir
  svn_dir
  svk_dir
  hg_dir
  base_dir="$PWD"
Here we just run all these functions one by one, stopping at the first that matches. Adding some more for other VCS would be easy.
  echo "$ vcs:+($vcs) $ _bold $ base_dir/$HOME/~ $ _normal $ vcs:+[$ref]$ _bold $ sub_dir $ _normal "

PS1='$ debian_chroot:+($debian_chroot) \u@\h:$(___vcs_dir)\$ '
Finally, we set up the prompt, so that it looks nice with all the gathered information. Update: made the last lines of the script a little better and factorized.

Christian Perrier: RWC: why did France lose...

...at least, in my opinion (which is obviously biased, let's admit that). There are several reasons for this loss: Some *bad* reasons for the English victory:

Joey Hess: lego

The good: I had fun building an enclosure for my Atari <-> PC serial adapter board out of legos, and I love the result. Been too long since I played with legos. I need to dig up the rest of my legos so i can make some enclosures for my ARM boards. The bad: I picked up a 1 gigabyte USB drive in the shape of an off-brand lego block today. The price was right (< $10), but the implementation is depressing -- the fake legos don't stick well and so the cap can't be mounted on the drive body when it's in use. And the size isn't lego standard, so it can't attach to other blocks. The ugly: In a better world, Lego would be an open standard. The patent expired in 1988 (according to Wikipedia. But the company still has lawyers, who seem busy doing what (corporate) lawyers do. Meh. Legos were more fun to play with when playing with them didn't involve worrying about patents and copyrights and the market's propensity to deter standardisation.

13 October 2007

Dirk Eddelbuettel: Makin' Tracks 2007

This morning was the annual Makin' Tracks 5k. As this starts three blocks from my house, supports the track at Concordia, a small college up the street from me, and runs right through my immediate neighbourhood, it's hard not to participate. This year was of course not even a week after the somewhat eventful Chicago Marathon in record-heat so I wasn't sure how well I'd do. Once again, I started way too fast and ran two miles at a 6:00 min/mile pace that I couldn't sustain just a week after a (arguably slow but still full distance) marathon, so I slowed down and finished at 19:41.7 or so for a pace of 6:23, or about eight seconds slower than my best time from last year (when we ran before the marathon). Onto the Frank Lloyd Wright 5k/10k next week, then!

Daniel Burrows: Monad.Reader

The Reader monad is a simple and easy-to-use structure from the Haskell library that solves a common problem.

Very often, it's the case that you have a lot of functions that all need to use the same values. For instance, you might be laying out some text on a page, and you need to keep the details of the page size around in all your layout algorithms. Typically, you handle this situation by adding a new parameter to all your functions which represents the state. So,


layoutPage :: Text -> Page


becomes


layoutPage :: PageLayoutParams -> Text -> Page


This works just fine, but it can get really ugly. Every single function gets a new parameter, which must be threaded through to every related function that it ever calls. This causes visual clutter, particularly for functions that don't actually use the parameter themselves but just pass it on.

A function written in the Reader monad is augmented with a hidden parameter. For the hypothetical example above, we would write:


import Control.Monad.Reader
type LayoutFunc = Reader PageLayoutParams

layoutPage :: Text -> LayoutFunc Page


Note that the Reader monad is partially applied here. The result, LayoutFunc, is a new monad that adds a parameter of type PageLayoutParams to any code that runs inside it.

So, now that we have this monad, how do we use it? The "ask" operation retrieves the parameter that's hidden in the monad:


ask :: Reader a a


or, specialized to LayoutFunc:


ask :: LayoutFunc PageLayoutParams


In order to remain close to the example in question, I will provide specialized type signatures for the rest of the library functions. You can find the generic type signatures in the Haskell library documentation for Control.Monad.Reader.

Suppose that layoutPage started out like this before it was moved into the monad:


layoutPage st txt =
let width = pageWidth st
...


In the Reader monad, this becomes:


layoutPage txt =
do st - ask
let width = pageWidth st
...


As you can see, we've gone from one line of code to two. As this is clearly unacceptable for a clutter-reducing device, the Reader module provides a convenience function encapsulating the above pattern:


asks :: (PageLayoutParams -> a) -> PageLayoutFunc a


This allows us to simplify the monadic implementation to:


layoutPage txt =
do width - asks pageWidth
...


It's quite common, when multiple functions share context information, to invoke a function in an altered context. For instance, we might have a layoutSubPage routine that changes the page width:


layoutSubPage st newWidth txt =
doLayout (st pageWidth = newWidth txt)


The Reader equivalent of this idiom is the function "local":


local :: (PageLayoutParams -> PageLayoutParams) -> LayoutFunc a -> LayoutFunc a


local executes an operation in a context modified by its first parameter. For instance,


layoutSubPage newWidth txt =
local (\st -> st pageWidth = newWidth ) txt


Finally, to actually invoke a computation in the Reader monad, use runReader:


runReader :: LayoutFunc a -> PageLayoutParams -> a

main = showPage (runReader defaultPageParams someText)


For much more information on the Reader monad, see All About Monads, or the Haskell library documentation.

Next.

Previous.