Search Results: "ed"

6 June 2011

Raphaël Hertzog: Official Debian/Ubuntu packages for Dropbox

Dropbox is a popular service to synchronize files between multiple computers. The service is entirely proprietary but the company is Linux friendly and provides Linux binaries ready to use. They even provide Ubuntu packages that wrap the dropbox client and provide integration with Nautilus. A bit of story Unfortunately for Debian users, those packages do not work on Debian due to a dependency that can t be satisfied (because Ubuntu introduced an epoch on the version of their nautilus package that Debian doesn t have). This was even reported in a Squeeze review in Linux Weekly News. At some point, Ivan Borzenkov introduced a dropbox package to Debian but it was not based on the above package, instead it packaged directly the proprietary binaries. This was a bad decision because the binaries bundle a set of LGPL libraries and nobody from Debian wanted to do the required work to provide the corresponding source code. So the package got dropped (see bug #610300). More recently several persons filed ITP (Intent To Package) bugs stating their willingness to re-introduce dropbox in Debian, but after many months lingering in the bug tracking system (see #544499, #613788), they have been turned back to RFP (Request For Package) because they changed their minds. What I did Being a dropbox user myself (despite the recent proof that data stored on dropbox is not 100% private), I offered sponsorship to the volunteers who wanted to package dropbox. But it turns out this was not enough to motivate someone to complete the task. In the mean time I was still using the old dropbox package that was removed (it used to be downloadable from snapshot.debian.org). While this was good enough for me, it s clearly not OK in the long term and way too difficult for the majority of users. So this week-end I spent some hours to create a proper package. It s loosely based on the package provided by Dropbox but I upgraded the packaging and changed the way it works. I patched the dropbox wrapper to provide a dropbox update command that downloads and updates the proprietary binaries. They are now stored in /var/lib/dropbox instead of having a copy in each user s home directory (~/.dropbox-dist/). This update command is run by the postinst so that installing the package immediately downloads the proprietary binaries. Get the Dropbox packages for Debian The package nautilus-dropbox has been uploaded to Debian unstable, it s currently in the NEW queue but will shortly reach the mirrors. Then you will be able to You can install the package with a simple apt-get install nautilus-dropbox (provided that you activated the non-free section since that s where the package is hosted, it can t be part of Debian since it requires the proprietary binaries to be useful). In the mean time you can get the packages from the links below: Get the Dropbox packages for Ubuntu I have setup a PPA for nautilus-dropbox. Feel free to use it in place of the upstream packages.
$ sudo add-apt-repository ppa:hertzog/nautilus-dropbox
$ sudo apt-get update
$ sudo apt-get install nautilus-dropbox
It also includes packages for oneiric, but in theory once the package is accepted into Debian, it should appear in oneiric shortly after. Package maintenance I have done the initial packaging work but I don t really want to maintain it in the long term. I have more than enough to do with dpkg and my other packages. So if you are interested in maintaining this package, please get in touch with me. You should know a bit of python since there are Debian-specific patches of the upstream code. The package is maintained in a git repository. Feedback If you have encountered a problem with one of those packages, feel free to leave a comment. If you re an happy user of the above packages, click here to find out how you can thank me. I hope you enjoy the packages!

13 comments Liked this article? Click here. My blog is Flattr-enabled.

Vincent Bernat: Ready for World IPv6 day?

On June 8, 2011, will be World IPv6 Day. Major web companies like Google, Yahoo! and Facebook will serve their content through IPv6. While some of them already provide IPv6 content, you either need to use a special URL or get your connection from some selected providers, known to provide working IPv6 connection (the selection is done through DNS).

What s World IPv6 Day about? World IPv6 Day is mostly about major content providers and network operators. End users are not expected to take any special actions. The content will be provided for one day on IPv6 and IPv4 without any special trick to circumvent broken IPv6 setup. If you have an IPv4 only connectivity, all will continue to work (this day is not about switching Internet to IPv6 only). If you have a working IPv6 connectivity of any kind, you will use it to access content. However, if you have a broken IPv6 connectivity, you will find yourself isolated during one day until you solve the problem. You can check your connectivity and determine if you will run into trouble on World IPv6 Day. The main point of this day is to check if IPv6 native content can be provided to anybody without leaving out a large part of the users (the ones with a broken connectivity) which is reason why we don t already have IPv6 native content everywhere. Google explains it this way:
We continuously conduct detailed measurements on the quality of IPv6 connectivity, and our latest results show that making Google services generally available over IPv6 at this time would lead to connection problems and increased latency for a small number of users. User experience is very important to us, and we do not want to impact users on networks that do not yet fully support IPv6. We will continue to re-evaluate the situation as the IPv6 Internet evolves.
While content provider are invited to provide IPv6 content, network operator are expected to troubleshoot their customers problem (and they cannot withdraw themselves from this operation since the main change is done by content providers). If they can enable IPv6 connectivity, that would be great too but this is not mandatory. UPDATED: It s a shame but Microsoft s participation to World IPv6 day is a fix to prefer IPv4 over IPv6. Moreover, this fix will be reverted a few days later. Let s hope that network operators don t follow the lead by not setting up DNS hijacking to remove AAAA record.

What can I do? Even if you are neither a network operator nor a major player in web content publishing, it is expected that World IPv6 Day will also be the playground for most IPv6 related things.

As a user Check if your ISP provides native IPv6 connectivity. This can be an opt-in option. For example, in France, Free allows its users to enable native IPv6 connectivity if they choose the appropriate option. That s easy to do and you get IPv6 in a matter of minutes. If your ISP does not provide native IPv6 connectivity, you can try to bug them about this. This is unlikely to work. You can also get IPv6 through a tunnel broker like SixXS (create an account and apt-get install aiccu) or Hurricane Electric. Wikipedia has a list of IPv6 tunnel brokers. Don t rely on any other transition mechanism, they are unreliable.

As a service provider If you host yourself some services, like your web page or your own mail server, you can enable IPv6 connectivity for those as well. Get an IPv6 connectivity for your services, either native or through a tunnel broker. Then, enable IPv6 on each service you provide. Check that they are listening on IPv6 with ss -6l. All the tips below are meant for very simple setup. Ensure that net.ipv6.bindv6only is set to 0. Here is what I have done:
  • Enable SSH access through IPv6. With the default configuration of OpenSSH, this is already done. Check your ListenAddress directive if you have one.
  • Serve Web content through IPv6. I am using Nginx as a web server and you can mix IPv4 and IPv6 in the same virtual host. Setting up IPv6 is easy. Ensure that IPv6 support is compiled in (nginx -V) use listen [::]:80 instead of listen 80. Apache is equally easy to configure and should even work out of the box since the Listen 80 directive will listen to both IPv4 and IPv6 by default. lighttpd also supports IPv6. An easy way to get started is to use server.use-ipv6 = "enable".
  • Enable IPv6 in Postfix, Dovecot, BIND. For all of those, this works out of the box. Nowadays, most daemons will work fine with IPv6 without any special configuration since they will listen to :: by default which means any IPv4 or IPv6 address as long as net.ipv6.bindv6only is set to 0. Unfortunately, there is still some daemons that do not support IPv6.
  • Add appropriate AAAA record to your DNS because providing IPv6 services is of little use if your users (including you) cannot access them without typing the appropriate IP address. You may also want to provide a NS record reachable through IPv6.
  • Update your monitoring to use IPv6 as well. If you advertise IPv6 service and only IPv4 is working, most IPv6 client will be unable to access them.
  • Adapt your firewall. In Linux, Netfilter handles IPv4 and IPv6 separately. You need to ensure you setup a correct firewall for IPv6 as you have done with IPv4.
Here is a minimal firewall that you can adapt:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
ip6tables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
ip6tables -A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
for chain in INPUT OUTPUT; do
  ip6tables -A $chain -m rt --rt-type 0 -j DROP
  ip6tables -A $chain -p icmpv6 --icmpv6-type neighbor-solicitation \
     -m hl --hl-eq 255 -j ACCEPT
  ip6tables -A $chain -p icmpv6 --icmpv6-type neighbor-advertisement \
     -m hl --hl-eq 255 -j ACCEPT
  ip6tables -A $chain -m state --state NEW \
     -p icmpv6 --icmpv6-type echo-request -j ACCEPT
done
ip6tables -A INPUT -i lo -j ACCEPT
ip6tables -A OUTPUT -o lo -j ACCEPT
# Put your rules here!
ip6tables -N REJECT_ALL
ip6tables -A REJECT_ALL -p tcp -j REJECT --reject-with tcp-reset
ip6tables -A REJECT_ALL -p udp -j REJECT --reject-with icmp6-adm-prohibited
ip6tables -A REJECT_ALL -j DROP
for chain in INPUT OUTPUT FORWARD; do
    ip6tables -A $chain -j LOG --log-prefix "$chain-reject: "
    ip6tables -A $chain -j REJECT_ALL
done

Debian status There is still a long way for full IPv6 support of all applications in Debian but it is a release goal since Etch. This is not an easy task since adding IPv6 support on some software can be really difficult. DSA also enabled IPv6 support on many servers used by the Debian project including the main web server, the main mirror, the security mirror and the name servers. You should be able to use Debian with an IPv6 only access.

Enrico Zini: Python list gotcha

Python list gotcha Suppose in python you're building a list of buckets:
>>> a = [[]] * 10
>>> print a
[[], [], [], [], [], [], [], [], [], []]
Looks good. However:
>>> a[5].append(1)
>>> print a
[[1], [1], [1], [1], [1], [1], [1], [1], [1], [1]]
Surprising? What happens here is that multiplying the list replicates the reference to the same empty list. You have the exact same mutable list replicated 10 times: instead of 10 buckets, you have 10 references to 1 bucket: therefore if appending to one it looks like one appends to all. What you need here is a way to invoke the list constructor [] multiple times:
>>> a = [[] for i in range(10)]
>>> print a
[[], [], [], [], [], [], [], [], [], []]
>>> a[5].append(1)
>>> print a
[[], [], [], [], [], [1], [], [], [], []]
a mistake like this can take quite a bit of time to track down.

5 June 2011

Dirk Eddelbuettel: Charles Lloyd and Zakir Hussain

Wonderful concert on Friday evening at Symphony Center: Charles Lloyd (Wikipedia; ts, flute, vocals) was in town with both his quartet and trio. The first part of the set was performed with Zakir Hussain (Wikipedia; tabla, vocals) and up-and-coming drummer Eric Harland (Wikipedia; drums). For the second half, Lloyd and Harland were joined by Jason Moran (Wikipedia; piano) -- who happens to also be a 2010 MacArthur Fellow -- and Larry Grenadier (Wikipedia; bass) before Hussain returned for the final piece and encores. It was evident how much joy Charles Lloyd still gets from performing live at his somewhat advanced age of 73. Surrounded by some extraordinary musicians, and clearly enjoying himself on stage. Great evening with wonderful jazz music from somewhere in between modern post-whatever, free, bop and hard bop, world music and everything else in between. I regret not having seen him earlier in life. Very much recommended.

Christian Perrier: Back into OpenStreetMap

I'm trying to resume some activity in OSM maintenance. As many people, I mapped most parts ofmy neighbourhood, as one can see when looking at Maurepas, France on the map. I stopped about 20 months ago because it was becoming too addictive..:-) I'll start resuming this work by focusing on things I can easily map: my tracks from running sessions. What's good in them is that I know most of these places by heart and can easily remember the terrain features when working on the map. Also, it means only a few dozens of minutes to work on a GPS track, so that's manageable. Another goal I set to myself is improving the map around our country house in Normandy. As you can see there, there is much more to do...but when I'm over there, I have more free time..:-) So, let's see if I can manage my addictability!

Jurij Smakov: Dear Lenovo ...

Let me explain something to you: jacking up prices for your laptops by 10% on June 1st and simultaneously declaring an "incredible, 5-days only, 10% off sale on all Thinkpads" on your website is not how your sell more laptops. That's how you make your potential customers recognize that they are probably better off looking elsewhere for their next laptop. Specifically, I'm talking about an Intel-based Thinkpad Edge 11 with 4GB RAM, which was listed at 663 EUR on lenovo.ie before June 1st, but is now, during the "promotion", is listed at 736 EUR. Notably, even after the discount, it still works out to be more expensive than what Amazon UK has to offer, at 552 GBP for the same laptop (NVY5EUK).

Russell Coker: Rallying for a Carbon Tax 5th June

It s not that common to have a rally in favor of creating a new tax, but today I attended the Melbourne rally in favor of a carbon tax [1], it was the second such event this year. The rally was held in front of the state library, there is a reasonable size park there and it s a great place for a few hundred people, and a few thousand can squeeze in there without any problem. But according to the best estimates 10,000 people attended and the venue was obviously a bad choice as all available space was used and the crowd was so great that it wasn t much fun to be there. This was a great contrast to the previous rally in favor of a carbon tax which had maybe about 8,000 people attend and a much larger area [2]. The state library is a great place to hold a small or medium size rally where people can see it, but if you are going to get 10,000 people you need a bigger venue as the exits were partially blocked by road work I m sure that the local government wouldn t have given a permit if so many people were expected. Usually the city center is very quiet on a Sunday afternoon, but this afternoon everything was packed. An extra 10,000 people really makes a difference although admittedly sunny weather would have helped. Australia wide there were apparently about 100,000 people attending similar rallies, that s 0.5% of the population of the country! The aim of the rally was to advocate a large enough tax on coal, oil, and other energy sources that emit CO2 to provide an economic incentive to minimise use and use alternate energy sources where possible. Currently billions of dollars are being given to polluting industries, instead those industries should be taxed and the money raised given to compensate needy people. looking north Looking North towards where I entered. looking east Two pictures from slightly different angles that are roughly East from where I was standing in the center of the crowd. looking east through the bandlooking south Pictures looking east through the band when people were starting to leave and looking south towards the exit I chose while on the way out.

Craig Small: JFFNMS 0.9.1 Released

jffnms 0.9.1 has the database extracts and updates missing from 0.9.0 This is the most problematic part of the project release; ensuring that the database updates correctly. Version 0.9.1 is functionally the same as 0.9.0, it is just with less bugs!

Joey Hess: announcing olduse.net

As I write this, it's the morning of June 5th, 1981. A few people scattered across the US are waking up, going in to work, sitting down at their terminal with a coffee, and reading Usenet. Usenet is only getting a trickle of posts each day -- it's still in that period where it's easy to read every message posted to it. Many things lie in Usenet's future. It's still running A-News, which doesn't even have a real From header yet. Later this year it will switch over to B-News, and volume will begin to increase. In 1987 there will be The great renaming. And of course in 1994, the first spam will be posted to Usenet. But that's all a long way off, here in 1981. Right now, they're talking about 500 mb disk drives that only cost $38000. And rms is inciting flames about nuclear proliferation. And Postel is publishing an RFC for the new Mail Transfer Protocol. Good morning, Usenet. Who knows what will come next in this fledgeling electronic communications medium! a ten year real-time historical exhibit This morning, I'm announcing a new site: Olduse.net It's Usenet, updated in real time as it was thirty years ago. Planned to be available for the next ten years, unless I run out of inodes (again). If you missed it the first time around, this is your chance to follow Usenet's flowering. made possible by 141 magtape usenet archive
PS: You can post to olduse.net, but it won't show up for at least 30 years. :)

Niels Thykier: Status on Lintian and Vendor Profiles

During the past week I finally got around to give the Vendor Profiles branch some attention. I suspect the most exciting change is that I got support for non-overridable tags. Along with making --ftp-master-rejects an alias for the ftp-master profile. This means that you will get the same behavior from Lintian and from the ftp-master auto-reject mechanism. The only issue is that the overrides are silently ignored. We also made dropped the default profile symlink in favor of using dpkg-vendor to find the best profile. This causes a slight conflict with solving #626476. I have asked the dpkg maintainers if that part of the libdpkg-perl API is stable enough for us to use. Finally I fixed an issue where profiles would ignore all display settings (--display-info etc.) and just show all tags (except experimental tags). Beyond the vendor profile branch, there are also a number of interesting changes applied in the master branch. Together with some people from #debian-python (ScottK and jwilk as I recall), I prepared some python related goodies. Particularly we fixed a false-positive missing-python-build-dependency with python3 and marked dh_pycentral and dh_python as deprecated. On a related note, we have also removed the tag uses-dh-python-with-no-pycompat. Based on feedback from Eric Lavarde, we redefined the classpath-contains-relative-path tag. It now only emits the bad parts of the Classpath and relative entries are allowed in they exists or point to /usr/share/java/$file.jar and there is a strong libX-java dependency. While the change breaks overrides, according to lintian.d.o we do not have any yet. The next release will feature a check for non-empty dependency_libs fields in la files and a fix to a flaw in logic for architecture dependent overrides. We will be reducing our dependency of dpkg-dev to a Suggests (#626476). Currently the xz-utils is in the pseudo-essential set, but it may not stay that way. So in the future you may need to explicitly install xz-utils for Lintian to support .xz source packages. Last, but not least, if you have felt that the lintianrc file has been rather useless to you, then you are not alone. I have made a prototype patch to improve the situation and hopefully soon you get to specify default options in your lintianrc.

Sune Vuorela: Five is a four letter word

and it starts with f While looking at the future of the KDE Platform Frameworks here in Randa, Switzerland, a lot of interesting things has been found in kdelibs and kdepimlibs that no one knew what was there for. On the good side, there is also lots of cool stuff, that of course most people knows about and uses And a lot of great larger solutions for solving different real life problems And lots of other cool stuff And we have currently nice things in the required runtime components for every app built using kdelibs

Jonathan McDowell: Building a FreeDOS bootable USB stick under Linux

This is for my own reference more than anything else; periodically I find myself needing to boot DOS. Usually because I want to upgrade a BIOS and the upgrade tool only works with Windows or DOS. Yes, I'm aware of flashrom, but the scary messages about toasting laptops means I'm much happier going the DOS route there. So I need a writable medium with a DOS boot image. The easy way to do it is to find a DOS floppy image and write that to a USB stick with dd, but it's basically a one off and means you can't fit a whole lot on the image. I wanted to do a native USB boot. The following did the trick for me. Firstly, this is all on a Debian testing box, in particular with dosfstools (3.0.9-1), mbr (1.1.11-4) and syslinux (2:4.04+dfsg-2). I don't think I'm using anything particularly new from these tools, so I suspect Debian stable will work just fine. In the below my USB stick is on /dev/sdb, I've got a copy of FreeDOS in ~/FreeDOS/ (I extracted the files from a balder10.img floppy image that's easy enough to find, but all you need is kernel.sys and command.com), and I wanted things neatly in their own subdirectories so I could try and keep the key for repeated use and know which bits I wanted to keep and which were transient. I also added a copy of the Debian Installer for convenience; it would need a copy of the netinst ISO dropped into the root to actually be useful, as there's not enough to do a full netboot from the wider Internet using just the HD initrd.
    # Create a single bootable LBA VFAT partition spanning entire stick.
    echo 0,,C,*   sfdisk /dev/sdb
    # Format as VFAT and give it a disk label.
    mkfs.vfat -n 'DOS BOOT' /dev/sdb1
    # Install an MBR
    install-mbr /dev/sdb
    # Mount it and create some directories for our files.
    mount /dev/sdb1 /media/DOS\ BOOT/
    cd /media/DOS\ BOOT/
    mkdir boot
    mkdir boot/syslinux/
    mkdir FreeDOS
    # Copy the chain.c32 syslinux tool over.
    cp /usr/lib/syslinux/chain.c32 boot/syslinux/
    # Copy FreeDOS over and move the important bits to /
    cp ~/FreeDOS/* FreeDOS/
    mv FreeDOS/command.com .
    mv FreeDOS/autoexec.bat .
    mv FreeDOS/fdconfig.sys .
    # Add the Debian Installer
    mkdir boot/d-i/
    cd boot/d-i/
    wget http://cdn.debian.net/debian/dists/squeeze/main/installer-amd64/current/images/hd-media/initrd.gz
    wget http://cdn.debian.net/debian/dists/squeeze/main/installer-amd64/current/images/hd-media/vmlinuz
    # Install syslinux
    syslinux -d boot /dev/sdb1
I made a boot/syslinux.help file containing:
    Available boot options are: FreeDOS d-i
and a boot/syslinux.cfg with:
    prompt 1
    display syslinux.help
    default FreeDOS
    label FreeDOS
        COM32 /boot/syslinux/chain.c32
        APPEND freedos=/FreeDOS/kernel.sys
    label d-i
        KERNEL /boot/d-i/vmlinuz
        INITRD /boot/d-i/initrd.gz
Initial testing was done using kvm which was a lot easier than constantly rebooting my laptop. Hopefully this is of help to someone. The only neat bit that I didn't see elsewhere when I was looking was the use of chain.c32 to load kernel.sys rather than having to use a FreeDOS provided boot sector image.

Russ Allbery: git-pbuilder 1.22

I actually released this on May 23rd, but I never posted an announcement here, and I think it's worth at least a brief note (if for no other reason that I can remember what I've been working on). The new version adds support for reading COWBUILDER_BASE from the environment so that you can use a different location for your cowbuilder setups if you'd like. It also removes an extra level of quoting around options in $BASE that caused problems with the shell. The latter is weird, and I still don't completely understand it. I was setting a variable like:
    OPTIONS="$OPTIONS --basepath '$BASE'"
and then using it with:
    pdebuild --pbuilder "$BUILDER" -- $OPTIONS
(with some additional complexity). I expected, since $OPTIONS was not quoted, that this would expand $OPTIONS and then do shell parsing again, which would strip off the other level of quotes. But this doesn't seem to actually work; instead, the quotes are left in, at least for some shells. This is apparently some corner of shell quoting that I don't understand. The chances of the base path containing shell metacharacters is quite low, so I just went with the solution that feels "underquoted." Anyway, you can get the latest version from my scripts page.

4 June 2011

Jan Wagner: Monitoring related package updates

Some more about packaging nagios and icinga related packages can be found at our team site.

Christian Perrier: [life] [running] April/May update

My last update about running activities is dated back to Paris Marathon (April 10th). Time for a new update, then. After a very busy end March-early April, with that crazy succession of 55km Ecotrail and 42km Paris marathon with only two weeks between both, I needed a kind of "break". Moreover, I had another goal set with the 35m trail race named "Trail des Cerfs", on May 15th, in my neighbourhood. So, end of April was a little bit quieter. Still, one week after Paris marathon, I completed a very hard trail run in Seine Valley: only 18km, but about 600-700m positive slope. That was indeed a good moutain-style run (yes, in Seine valley....) and a good way to check that I recovered quite well. Later on, I slowed down a little with even a Sunday (24th) where I *didn't run at all*..:-).. Just a few short uns during the week and a short Saturday run. Well, that was indeed Easter Sunday and Monday is a holiday in France, so I ran an half-marathon on 25th...:-)...on a loop where I already have some references, the fastest being 1h46. This time it was 1h52'...but, well, that was two weeks after a marathon... Still, a small pain at the back of my ankle decided me to stop running long distances until May 15th. So, I only did short trainings up to no more than 12km, even during my stay in Germany for SambaXP. As a consequence, I was very well for the 35km (600m positive slope) Trail des Cerfs, which I completed in 3h31', 168th out of 328 runners. More interestingly, I completed it in negative split with a much faster second half, ending up at about 12km/h. For seomthing that (given the slope and the nature of the paths) is comparable to a marathon, that's very promising. The rest of May was spent quietly with a standard training and progressively resuming longer distances. I originally planned to do my spring solo marathon on May 28th in Versailles Castle park, but I stopped at km28 and completed the distance two days after with a 14km run..:-)... Stopping was wiser as I was feeling that the last loop would have been a great pain. That was indeed the first time I give up on a marathon. I prefer this to happen for a non official race, after all! June will not see any "big goal". I might be running one race somewhere in my neighbourhood but the point will mostly be accumulating kilometers before summer. Summer (which includes DebConf that makes training a little bit more hectic) will not see any race, probably (unless I find one in banja Luka!) but more some specific preparation for Berlin marathon. Here, the "safe" target for me is, first, breaking my record. I'm probably not ready for 3h30 and even 3h35 might be difficult. It seems that I'm now reaching a barrier where progress becomes harder. Before Berlin, I'll run the Bois d'Arcy half-marathon, two weeks before. Last year, I managed to set a very good mark, there, with 1h37. Not sure that i'll try to break this only two weeks before Berlin. After Berlin? We'll see, but my last big goal of the year will be the 67km "Le Puy-Firminy" night race, in November (no web site, this is indeed a "small" local race). Yet another adventure beyond marathon distances, in the unknown...

Mike (stew) O'Connor: My Movein Script

Erich Schubert's blog post reminded me that I've been meaning to writeup a post detailing how I'm keeping parts of my $HOME in git repositories. My goal has been to keep my home directory in a version control system effectively. I have a number of constraints however. I want the system to be modular. I don't always need X related config files in my home directory. Sometimes I want just my zsh related files and my emacs related files. I have multiple machines I check email from, and on those want to keep my notmuch/offlineimap files in sync, but I don't need these on every machine I'm on, expecially since those configurations have more sensitive data. I played around with laysvn for a while, but it never really seemed comfortable. I more recently discovered that madduck had started a "vcs-home" website and mailing list, talking about doing what I'm trying to do. I'm now going with madduck's idea of using git with detached work trees, so that I can have multiple git repositories all using $HOME as their $GIT_WORK_TREE. I have a script inspired by his vcsh script that will create a subshell where the GIT_DIR, GIT_WORK_TREE variables are set for me. I can do my git operations related to just one git repository in that shell, while still operating directly on my config files in $HOME, and avoiding any kind of nasty symlinking or hardlinking. Since I am usually using my script to allow me to quickly "move in" to a new host, I named my script "movein". It can be found here. Here's how I'll typically use it:
    stew@guppy:~$ movein init
    git server hostname? git.vireo.org
    path to remote repositories? [~/git] 
    Local repository directory? [~/.movein] 
    Location of .mrconfig file? [~/.mrconfig] 
    stew@guppy:~$ 
This is just run once. It asks me questions about how to setup the 'movein' environment. Now I should have a .moveinrc storing the answers I gave above, I have a stub of a .mrconfig, and an empty .movein directory. Next thing to do is to add some of my repositories. The one I typically add on all machines is my "shell" repository. It has a .bashrc/.zshrc, an .alias that both source and other zsh goodies I'll generally wish to be around:
    stew@guppy:~$ ls .zshrc
    ls: cannot access .zshrc: No such file or directory
    stew@guppy:~$ movein add shell
    Initialized empty Git repository in /home/stew/.movein/shell.git/
    remote: Counting objects: 42, done.
    remote: Compressing objects: 100% (39/39), done.
    remote: Total 42 (delta 18), reused 0 (delta 0)
    Unpacking objects: 100% (42/42), done.
    From ssh://git.vireo.org//home/stew/git/shell
     * [new branch]      master     -> origin/master
    stew@guppy:~$ ls .zshrc
    .zshrc
So what happened here is that the ssh://git.vireo.org/~/git/shell.git repository was cloned with GIT_WORK_TREE=~ and GIT_DIR=.movein/shell.git. My .zshrc (along with a bunch of other files) has appeared. Next perhaps I'll add my emacs config files:
    stew@guppy:~$ movein add emacs       
    Initialized empty Git repository in /home/stew/.movein/emacs.git/
    remote: Counting objects: 77, done.
    remote: Compressing objects: 100% (63/63), done.
    remote: Total 77 (delta 10), reused 0 (delta 0)
    Unpacking objects: 100% (77/77), done.
    From ssh://git.vireo.org//home/stew/git/emacs
     * [new branch]      emacs21    -> origin/emacs21
     * [new branch]      master     -> origin/master
    stew@guppy:~$ ls .emacs
    .emacs
    stew@guppy:~$ 
My remote repositry has a master branch, but also has an emacs21 branch, which I can use when checking out on older machines which don't yet have newer versions of emacs. Let's say I have made changes to my .zshrc file, and I want to check them in. Since we are working with detached work trees, git can't immediately help us:
    stew@guppy:~$ git status
    fatal: Not a git repository (or any of the parent directories): .git
The movein script allows me to "login" to one of the repositories. It will create a subshell with GIT_WORK_TREE and GIT_DIR set. In that subshell, git operations operate as one might expect:
    stew@guppy:~ $ movein login shell
    stew@guppy:~ (shell:master>*) $ echo >> .zshrc
    stew@guppy:~ (shell:master>*) $ git add .zshrc                                       
    stew@guppy:~ (shell:master>*) $ git commit -m "adding a newline to the end of .zshrc"
    [master 81b7311] adding a newline to the end of .zshrc
     1 files changed, 1 insertions(+), 0 deletions(-)
    stew@guppy:~ (shell:master>*) $ git push
    Counting objects: 8, done.
    Delta compression using up to 2 threads.
    Compressing objects: 100% (6/6), done.
    Writing objects: 100% (6/6), 546 bytes, done.
    Total 6 (delta 4), reused 0 (delta 0)
    To ssh://git.vireo.org//home/stew/git/shell.git
       d24bf2d..81b7311  master -> master
    stew@guppy:~ (shell:master*) $ exit
    stew@guppy:~ $ 
If I want to create a brand new repository from files in my home directory. I can:
    stew@guppy:~ $ touch methere
    stew@guppy:~ $ touch mealsothere
    stew@guppy:~ $ movein new oohlala methere mealsothere
    Initialized empty Git repository in /home/stew/git/oohlala.git/
    Initialized empty Git repository in /home/stew/.movein/oohlala.git/
    [master (root-commit) 7abe5ba] initial checkin
     0 files changed, 0 insertions(+), 0 deletions(-)
     create mode 100644 mealsothere
     create mode 100644 methere
    Counting objects: 3, done.
    Delta compression using up to 2 threads.
    Compressing objects: 100% (2/2), done.
    Writing objects: 100% (3/3), 224 bytes, done.
    Total 3 (delta 0), reused 0 (delta 0)
    To ssh://git.vireo.org//home/stew/git/oohlala.git
     * [new branch]      master -> master
Above, the command movein new oohlala methere mealsothere says "create a new repository containing two files: methere, mealsothere". A bare repository is created on the remote machine, a repository is created in the .movein directory, the files are committed, and the new commit is pushed to the remote repository. New on some other machine, I could run movein add oohlala to get these two new files. The movein script maintains a .mrconfig file, so that joeyh's mr tool can be used to manage the repositories in bulk. Commands like "mr update", "mr commit", "mr push" will act on all the known repositories. Here's an example:
    stew@guppy:~ $ cat .mrconfig
    [DEFAULT]
    include = cat /usr/share/mr/git-fake-bare
    [/home/stew/.movein/emacs.git]
    checkout = git_fake_bare_checkout 'ssh://git.vireo.org//home/stew/git/emacs.git' 'emacs.git' '../../'
    [/home/stew/.movein/shell.git]
    checkout = git_fake_bare_checkout 'ssh://git.vireo.org//home/stew/git/shell.git' 'shell.git' '../../'
    [/home/stew/.movein/oohlala.git]
    checkout = git_fake_bare_checkout 'ssh://git.vireo.org//home/stew/git/oohlala.git' 'oohlala.git' '../../'
    stew@guppy:~ $ mr update
    mr update: /home/stew//home/stew/.movein/emacs.git
    From ssh://git.vireo.org//home/stew/git/emacs
     * branch            master     -> FETCH_HEAD
    Already up-to-date.
    mr update: /home/stew//home/stew/.movein/oohlala.git
    From ssh://git.vireo.org//home/stew/git/oohlala
     * branch            master     -> FETCH_HEAD
    Already up-to-date.
    mr update: /home/stew//home/stew/.movein/shell.git
    From ssh://git.vireo.org//home/stew/git/shell
     * branch            master     -> FETCH_HEAD
    Already up-to-date.
    mr update: finished (3 ok)
    stew@guppy:~ $ mr update        
There are still issues I'd like to address. The big one in my mind is that there is no .gitignore. So when you "movein login somerepository" then run "git status", It tells you about hundreds of untracked files in your home directory. Ideally, I just want to know about the files which are already associated with the repository I'm logged into.

Mike (stew) O'Connor: My Movin Script

Erich Schubert's blog post reminded me that I've been meaning to writeup a post detailing how I'm keeping parts of my $HOME in git repositories. My goal has been to keep my home directory in a version control system effectively. I have a number of constraints however. I want the system to be modular. I don't always need X related config files in my home directory. Sometimes I want just my zsh related files and my emacs related files. I have multiple machines I check email from, and on those want to keep my notmuch/offlineimap files in sync, but I don't need these on every machine I'm on, expecially since those configurations have more sensitive data. I played around with laysvn for a while, but it never really seemed comfortable. I more recently discovered that madduck had started a "vcs-home" website and mailing list, talking about doing what I'm trying to do. I'm now going with madduck's idea of using git with detached work trees, so that I can have multiple git repositories all using $HOME as their $GIT_WORK_TREE. I have a script inspired by his vcsh script that will create a subshell where the GIT_DIR, GIT_WORK_TREE variables are set for me. I can do my git operations related to just one git repository in that shell, while still operating directly on my config files in $HOME, and avoiding any kind of nasty symlinking or hardlinking. Since I am usually using my script to allow me to quickly "move in" to a new host, I named my script "movein". It can be found here. Here's how I'll typically use it:
    stew@guppy:~$ movein init
    git server hostname? git.vireo.org
    path to remote repositories? [~/git] 
    Local repository directory? [~/.movein] 
    Location of .mrconfig file? [~/.mrconfig] 
    stew@guppy:~$ 
This is just run once. It asks me questions about how to setup the 'movein' environment. Now I should have a .moveinrc storing the answers I gave above, I have a stub of a .mrconfig, and an empty .movein directory. Next thing to do is to add some of my repositories. The one I typically add on all machines is my "shell" repository. It has a .bashrc/.zshrc, an .alias that both source and other zsh goodies I'll generally wish to be around:
    stew@guppy:~$ ls .zshrc
    ls: cannot access .zshrc: No such file or directory
    stew@guppy:~$ movein add shell
    Initialized empty Git repository in /home/stew/.movein/shell.git/
    remote: Counting objects: 42, done.
    remote: Compressing objects: 100% (39/39), done.
    remote: Total 42 (delta 18), reused 0 (delta 0)
    Unpacking objects: 100% (42/42), done.
    From ssh://git.vireo.org//home/stew/git/shell
     * [new branch]      master     -> origin/master
    stew@guppy:~$ ls .zshrc
    .zshrc
So what happened here is that the ssh://git.vireo.org/~/git/shell.git repository was cloned with GIT_WORK_TREE=~ and GIT_DIR=.movein/shell.git. My .zshrc (along with a bunch of other files) has appeared. Next perhaps I'll add my emacs config files:
    stew@guppy:~$ movein add emacs       
    Initialized empty Git repository in /home/stew/.movein/emacs.git/
    remote: Counting objects: 77, done.
    remote: Compressing objects: 100% (63/63), done.
    remote: Total 77 (delta 10), reused 0 (delta 0)
    Unpacking objects: 100% (77/77), done.
    From ssh://git.vireo.org//home/stew/git/emacs
     * [new branch]      emacs21    -> origin/emacs21
     * [new branch]      master     -> origin/master
    stew@guppy:~$ ls .emacs
    .emacs
    stew@guppy:~$ 
My remote repositry has a master branch, but also has an emacs21 branch, which I can use when checking out on older machines which don't yet have newer versions of emacs. Let's say I have made changes to my .zshrc file, and I want to check them in. Since we are working with detached work trees, git can't immediately help us:
    stew@guppy:~$ git status
    fatal: Not a git repository (or any of the parent directories): .git
The movein script allows me to "login" to one of the repositories. It will create a subshell with GIT_WORK_TREE and GIT_DIR set. In that subshell, git operations operate as one might expect:
    stew@guppy:~ $ movein login shell
    stew@guppy:~ (shell:master>*) $ echo >> .zshrc
    stew@guppy:~ (shell:master>*) $ git add .zshrc                                       
    stew@guppy:~ (shell:master>*) $ git commit -m "adding a newline to the end of .zshrc"
    [master 81b7311] adding a newline to the end of .zshrc
     1 files changed, 1 insertions(+), 0 deletions(-)
    stew@guppy:~ (shell:master>*) $ git push
    Counting objects: 8, done.
    Delta compression using up to 2 threads.
    Compressing objects: 100% (6/6), done.
    Writing objects: 100% (6/6), 546 bytes, done.
    Total 6 (delta 4), reused 0 (delta 0)
    To ssh://git.vireo.org//home/stew/git/shell.git
       d24bf2d..81b7311  master -> master
    stew@guppy:~ (shell:master*) $ exit
    stew@guppy:~ $ 
If I want to create a brand new repository from files in my home directory. I can:
    stew@guppy:~ $ touch methere
    stew@guppy:~ $ touch mealsothere
    stew@guppy:~ $ movein new oohlala methere mealsothere
    Initialized empty Git repository in /home/stew/git/oohlala.git/
    Initialized empty Git repository in /home/stew/.movein/oohlala.git/
    [master (root-commit) 7abe5ba] initial checkin
     0 files changed, 0 insertions(+), 0 deletions(-)
     create mode 100644 mealsothere
     create mode 100644 methere
    Counting objects: 3, done.
    Delta compression using up to 2 threads.
    Compressing objects: 100% (2/2), done.
    Writing objects: 100% (3/3), 224 bytes, done.
    Total 3 (delta 0), reused 0 (delta 0)
    To ssh://git.vireo.org//home/stew/git/oohlala.git
     * [new branch]      master -> master
Above, the command movein new oohlala methere mealsothere says "create a new repository containing two files: methere, mealsothere". A bare repository is created on the remote machine, a repository is created in the .movein directory, the files are committed, and the new commit is pushed to the remote repository. New on some other machine, I could run movein add oohlala to get these two new files. The movein script maintains a .mrconfig file, so that joeyh's mr tool can be used to manage the repositories in bulk. Commands like "mr update", "mr commit", "mr push" will act on all the known repositories. Here's an example:
    stew@guppy:~ $ cat .mrconfig
    [DEFAULT]
    include = cat /usr/share/mr/git-fake-bare
    [/home/stew/.movein/emacs.git]
    checkout = git_fake_bare_checkout 'ssh://git.vireo.org//home/stew/git/emacs.git' 'emacs.git' '../../'
    [/home/stew/.movein/shell.git]
    checkout = git_fake_bare_checkout 'ssh://git.vireo.org//home/stew/git/shell.git' 'shell.git' '../../'
    [/home/stew/.movein/oohlala.git]
    checkout = git_fake_bare_checkout 'ssh://git.vireo.org//home/stew/git/oohlala.git' 'oohlala.git' '../../'
    stew@guppy:~ $ mr update
    mr update: /home/stew//home/stew/.movein/emacs.git
    From ssh://git.vireo.org//home/stew/git/emacs
     * branch            master     -> FETCH_HEAD
    Already up-to-date.
    mr update: /home/stew//home/stew/.movein/oohlala.git
    From ssh://git.vireo.org//home/stew/git/oohlala
     * branch            master     -> FETCH_HEAD
    Already up-to-date.
    mr update: /home/stew//home/stew/.movein/shell.git
    From ssh://git.vireo.org//home/stew/git/shell
     * branch            master     -> FETCH_HEAD
    Already up-to-date.
    mr update: finished (3 ok)
    stew@guppy:~ $ mr update        
There are still issues I'd like to address. The big one in my mind is that there is no .gitignore. So when you "movein login somerepository" then run "git status", It tells you about hundreds of untracked files in your home directory. Ideally, I just want to know about the files which are already associated with the repository I'm logged into.

Russell Coker: Leaving Three

In February I started the process of moving my phone and my wife s phone to Virgin from Three [1]. The reason is that Three didn t offer any good phones on affordable contracts, the cheapest that was suitable was a HTC Desire HD which would have cost me $55 per month, while I could justify spending that for my own phone (which is used for responding to SMS from Nagios to fix client servers) I didn t have the budget to spend that much on my wife s phone too and I really want us to have the same type of phone for ease of support. So I chose Virgin who offers the Sony Ericsson Xperia X10 for as little as $29 per month I chose a $39 per month deal that included 1500MB of data transfer and also had three months free which makes it effectively $34.12 per month. When using previous phones that weren t particularly smart I had also carried a Netbook and a 3G modem with me most of the time. Now that I have a phone that is a ssh client I don t need that so I tried to cancel the contract today. Three allows you to do almost everything over the Internet except cancel a contract their web site doesn t even give a phone number to call for that purpose. This must keep their support people busy, Vodaphone (which has just merged with Three) has recently had a horrible security breach because their sales booths used public Internet access for all customer data [2]. Also there is currently a law suit against Vodafone for poor network performance and misleading claims about service areas [3]. My experience with Three performance has been reasonably good apart from the fact that they advertised 3G service in Bendigo and provided none. As Three are apparently desperate to retain customers they offered me free service for 6 months if I don t close the account now. So I have a SIM that supports 1G of 3G data transfer per month for no charge until December (worth $90). What can I do with it? I don t own a 3G modem as I gave that to my parents (who are quite happy with pre-paid 3G net access via Three) and the phones that I have which can be used for tethering are a little slow (usable for ssh and basic web access but not for Youtube etc). Is there a way of selling such a SIM? Note that my name is still on the contract and any excess data or roaming fees will be billed to me so I can t just put it on ebay. I guess that one thing I can do is to use the SIM for receiving phone calls. For example if a friend was visiting from another country and wanted to receive calls without paying roaming fees I could lend them a phone. Any ideas?

Rudy Godoy: GSoC: Status report 01

Hello, this is the official project report for the Debian GSoC admins and everyone. It s been few days since the official coding phase started. Most of the time I ve been figuring out the details regarding integration of non-x86 images to the Eucalyptus IaaS and working on the ARM image, as you ll learn later. I m also in the process of starting to design the set of tools we ll be delivering as part of the project. The project s goal is to be a useful resource for Debian developers and porters. I ve joined some porters list in order to learn what are the things they are struggling with and how the project can help to address them. I ll be starting a wiki page with such topics. Bonding period:
Bonding period has been more fruitful than I expected. As mentioned in my previous post, Steffen and I got in touch with the Eucalyptus team, since they show their early interest on the project. We are coordinating cooperation between our teams. Besides my mentor and I defined to start working on the ARM image. Coordination went OK. We ll be having meetings every two weeks. Project Status:
Resuming Dominique s work I began working on the ARM image. Aur lien s work was a great resource. Currently I have a working and updated Debian testing ARM image using the versatile kernel that can be used under qemu-arm. I m currently making tests and figuring out the next phase which is cloudify it. Over this weekend I expect to finish on that side and then began on the design and implementation of the tools that will be the project s result.
Until now I haven t faced much issues, but the Eucalyptus part since the docs are targeting x86 and amd_64 archs. However I think I can run qemu-arm on top. I ll talk to them about the idea.
Future plans: week 3: - Finish tests with the ARM image. - Design the base tool-set, thinking integration with dpkg-buildpackage and qemubuilder. week 4-8: - Write the necessary code. mid-term evaluation. Other bits: Latest days have been quite busy for me, since I m still have to attend class, my semester ends in five weeks. This week was quite overwhelming with media , my University published an article (Spanish) about me and the program. I m the third student of the CS school participating in GSoC, in the past we had Gentoo Project and KDE students. Later, a local Cable TV program interview me on the same matter, it went fine. I managed to talk about Debian and what we do (we get free Advertising yay!) :) also about Google s Summer of Code program and invited students to participate both in FLOSS projects and the program. Still don t know if they have put it on air, I think it will be next week.

3 June 2011

Sam Dunne: Declarative Diversions - Report 1

DECLARATIVE DIVERSIONS ====================== Introduction A diversion is when it is possible to have dpkg not overwrite a file when it reinstalls the package it belongs to, and to have it put the file from the package somewhere else instead. Declarative diversions involves a new control file with a declarative syntax which dpkg will parse and process directly as part of the package unpack and removal phases, eliminating the problems resulting from non-atomic handling of diversions. Topics There are a number of topics involved in implementing this kind of project * What syntax do we use for the new control file? * What dpkg does with the control file * How do we handle diversions to a non-existant directory? * How do we order unpacking a new package which adds a diversion? * How do we order unpacking a new package that removes a diversion? * How do we order removing a package which had a diversion? * How do we handle errors? * What happens to dpkg-divert? - Details - Control File Syntax - It will conform to RFC2822 style with the following format: * Divert-From: * Divert-To: * Blank lines and lines beginning with # will be comments Divert-To will be optional and if it is ommitted then files being diverted will have their filename changed to file.distrib The above style has it s advantages, one of the main ones being that there is no need to escape whitespace within filenames. - Details - Control File Handling - Within control.tar.gz the file should be named diversions This file is then copied to /var/lib/info/$package.diversions This is ensuring we have a copy of the control file just in case it is needed. Any input on this subject would be appreciated. - Details - Handling Diversions to non-existant directories - Diverting files to directories that don t exist can cause a number of problems. If the package does not own the directory it may be left orphaned on removal of the package The package is responsible for ensuring the availability of the target directory in the unpack phase. - Details - Ordering Requirements - =>Unpacking a new package that adds a diversion *1. Unpack folders *2. Add diversions *3. Unpack files ==Notes== Performing diversions this way avoids trouble with creating folders and accidentally overwrtiting the wrong version of files. If we don t use this method the problems that can occur include creating folders before the package itself would create them which may cause permission errors. Another major problem would be extracting a new version of the file and overwriting the old one. This might cause the wrong version of the file to be diverted and the package to break. =>Unpacking a new package that removes a diversion *1. Unpack folders *2. Add diversions *3. Unpack files ==Notes== =>Removing a package which had a diversion *1. Remove files *2. Remove diversions *3. Remove folders ==Notes== This ensures that all files, diversions and folders are removed correctly Details - Error handling Errors in diversions will have to handled with a great deal of care due to the fact that if they are not the package could be broken. This means that a great deal of checks must be done to ensure that all the files can be diverted properly before any actual diverting takes place. If they can t the package installation/update must be stopped and rolled back to avoid the package being installed incorrectly or broken. - Details - dpkg-divert - When we impliment the new diversion method we should keep the current dpkg-divert. This allows maintainers to catch up with the new method without breaking their packages. It also allows maintainers to perform some operations that aren t support by the new method. Example Usage #1 The file to be diverted is /usr/share/foo It needs to be moved to /usr/share/bar The syntax of the control file would be: #Start File Divert-From: /usr/share/foo Divert-To: /usr/share/bar #End File Example Usage #2 In this example the maintainer doesn t want to move the file to any specific folder The syntax of the control file would be: #Start File Divert-From: /usr/share/foo #End File This would divert the file to /usr/share/foo.distrib - Footnotes - RFC2822 Guide: * http://www.faqs.org/rfcs/rfc2822.html First Email Thread on Declarative Diversions (First Message in Thread): * http://lists.debian.org/debian-dpkg/2011/05/msg00102.html Declarative Diversions Wiki: * http://wiki.debian.org/SummerOfCode2011/DeclarativeDiversions My Blog for this Project: * http://blog.sam-dunne.com

Next.

Previous.