• 16935 unique source packages in Debian/sid
• 9977 packages with Vcs-* field in Debian/sid
• 6957 packages without a Vcs-* field in Debian/sid

1. Svn: 4939
2. Git: 4377
3. Darcs: 284
4. Bzr: 247
5. Hg: 61
6. Cvs: 31
7. Arch: 28
8. Mtn: 10

                  e      s     p-u     t      u    t-p-u   l-r    s-u     o    o-p-u   s-r   
---------------------------------------------------------------------------------------------
     all         1096  12034     70  14442  15784     20      1     12   8910    230      0  
    alpha           -      -      -      -      -      -      -      -  13183    297      -  
    amd64        1164  16997    242  18581  19605     39      -     21  13964    342      -  
     arm            -      -      -      -      -      -      -      -  13251    291      -  
    armel         988  16401    239  17879  18680     39      -     21  13411    327      -  
     hppa           -      -      -      -      -      -      -      -  13227    264      -  
  hurd-i386       445      -      -      -  12622      -      -      -      -      -      -  
     i386        1178  17188    242  18727  19751     39      -     21  14377    350      -  
     ia64         951  16036    239  17303  18045     39      -     21  13554    329      -  
kfreebsd-amd64    807  14499    231  15905  16480     39      -     21      -      -      -  
kfreebsd-i386     826  14491    231  15881  16498     39      -     21      -      -      -  
     mips         974  16269    241  17730  18504     39      -     21  13638    323      -  
    mipsel        984  16291    239  17807  18558     39      -     21  13600    319      -  
   powerpc       1078  16677    239  18156  19036     39      -     21  13872    332      -  
     s390        1030  16252    239  17678  18469     39      -     21  13331    327      -  
    source        522  14974     51  16376  17405      1     19      6  12519     75    151  
    sparc         992  16423    239  17895  18673     39      -     21  13542    329      -  

Hole:5 ftp://ftp.rrzn.uni-hannover.de/debian/debian/ squeeze/main extremetuxracer-data all 0.4-4 [28,1 MB]
Hole:6 ftp://ftp.rrzn.uni-hannover.de/debian/debian/ squeeze/main extremetuxracer amd64 0.4-4 [273 kB]

PID & mount namespaces A PID namespace is a view of the processes running on the system. Each PID namespace will have its own numbering of the processes contained in its namespace (including the namespace of the children). Other processes are hidden and therefore, it is not possible to issue any operation requiring a PID on them, like kill(2). Moreover, the process that was created first in a namespace will get PID 1. If it gets killed, the namespace is destroyed and all other processes contained in this namespace are killed as well. systemd, a system and service manager for Linux, targeted as a replacement for the old SysV init, uses could use this feature to launch daemons in their own PID namespace and keep track of them without relying on their cooperation. When it needs to kill a daemon, it just kills the process with PID 1 (which has another PID in the global namespace) and the kernel will do the hard work of killing all processes that were started in this namespace. You can get more details about this namespace in the manual page of clone(2) (search for CLONE_NEWPID). UPDATED: In fact, systemd does not use PID namespaces but cgroups to group processes together. There is no PID numbering here but spawned processes are all enclosed into one cgroup and therefore can be killed easily. For example, here is the result of pstree -p inside a PID namespace:

root@guybrush:/# pstree -p
bash(1) pstree(8)
         sleep(7)

The PID of sleep is 7. In the global namespace, it is 31762 instead. But it is the same processus!

root@guybrush:/# ps -eo pid,command   grep sleep
31762 sleep 1000

Linux have also supported mount namespaces since a long time. Each process can have its own mount points that will be visible only by processes in the same namespace or in children namespaces. When the namespace is destroyed (when no process left), the kernel will do the cleanup itself.

chroot A common way to test experimental software or do packaging work is to build a chroot: debootstrap is used to build a new complete system inside a directory and chroot allows to enter into this new system. Tools like schroot can become handy if you need to do that often. Unfortunately, you only get an isolated filesystem. When you start launching daemons and mounting partitions, you need to keep track of all your actions to undo them. Quitting the chroot will not kill any daemon nor unmount any partition. Extending an existing tool, like schroot to use PID and mount namespaces seems the best way to prevent such problems. I have filed bug #637870 for this purpose. However, this is not as simple as it seems since schroot allows to run a process into an existing session. When the session is a chroot, this is quite easy: just chroot into the same root filesystem and launch the process here. With other kind of namespaces, you need to use a new system call, setns(), to change the current namespace of the running process. However, some bits are still missing. First, the GNU C Library does not implement the appropriate wrapper. Second, Linux (as of 3.0) does not contain the support to change the PID namespace of a running process yet. Other tools exist, like lxc (aka Linux Containers). However, it is more targeted at providing a complete container. I have reported that lxc-execute is not able to spawn a standalone shell but got no luck. UPDATED: systemd ships with systemd-nspawn which was exactly what I needed. However, it requires systemd. Otherwise, it seems less basic than the alternative that I describe below (some sensible mount points are setup, the console is correctly configured, signals are correctly transmitted, ). If you use systemd, look at it!

jchroot Building a chroot clone using namespaces is quite easy. The only difficulty is the use of clone(2). Here is a sketch:

int main()  
  pid_t pid;
  int ret;
  long stack_size = sysconf(_SC_PAGESIZE);
  void *stack = alloca(stack_size) + stack_size;
  /* New process with its own PID/IPC/NS namespace */
  pid = clone(step2,
              stack,
              SIGCHLD   CLONE_NEWPID   CLONE_NEWIPC  
              CLONE_NEWNS   CLONE_FILES,
              NULL);
  /* Wait for the child to terminate */
  while (waitpid(pid, &ret, 0) < 0 && errno == EINTR)
    continue;
  return WIFEXITED(ret)?WEXITSTATUS(ret):EXIT_FAILURE;
 
static int step2(void *a)  
  /* Mount /proc as an example */
  mount("proc", "/proc", "proc", 0, "");
  /* Chroot */
  chroot("/srv/debian");
  chdir("/");
  /* Drop privileges */
  setgid(1000);
  setgroups(0, NULL);
  setuid(1000);
  /* Exec shell */
  execl("/bin/bash", "/bin/bash", NULL);
 

Add command flags, error handling and fstab parsing for the definition of mount points and you get jchroot. Here is an example of use:

$ cat fstab
proc     /proc  proc    defaults                  0  0
sys      /sys   sysfs   defaults                  0  0
/home    /home  none    bind,rw                   0  0
/dev/pts /dev/pts none  bind,rw                   0  0
/var/run /var/run tmpfs rw,nosuid,noexec,mode=755 0  0
/etc/resolv.conf /etc/resolv.conf none bind,ro    0  0
$ sudo jchroot -f fstab -n test /srv/debian/squeeze /bin/bash
# hostname
test
# ps auxww
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  19480  1888 pts/1    S    15:18   0:00 /bin/bash
root         3  0.0  0.0  16536  1184 pts/1    R+   15:18   0:00 ps auxww
# /etc/init.d/postgresql start
Starting PostgreSQL 9.0 database server: main.
# exit
$ ps auxwwc   grep postgres
$

I hope that jchroot won t live for too long and that its features will be integrated into regular chroot tools. I was thinking at proposing a patch for chroot(8). Anyone knows if this would have a chance to be accepted?

Once upon a time, there was Deb and Ian. That was about exactly 18 years ago. We don t talk about the 0.939000 format any more, but they eventually settled on:

	$ ar rc pkg_1.0_all.deb debian-binary control.tar.gz data.tar.gz
	$ hexdump -C pkg_1.0_all.deb   head
	00000000  21 3c 61 72 63 68 3e 0a  64 65 62 69 61 6e 2d 62   !<arch>.debian-b 
	00000010  69 6e 61 72 79 20 20 20  31 33 31 33 36 38 33 35   inary   13136835 
	00000020  32 39 20 20 31 30 30 36  20 20 32 30 30 20 20 20   29  1006  200    
	00000030  31 30 30 36 34 34 20 20  34 20 20 20 20 20 20 20   100644  4        
	00000040  20 20 60 0a 32 2e 30 0a  63 6f 6e 74 72 6f 6c 2e      .2.0.control. 
	00000050  74 61 72 2e 67 7a 20 20  31 33 31 33 36 38 33 35   tar.gz  13136835 
	00000060  32 39 20 20 31 30 30 36  20 20 32 30 30 20 20 20   29  1006  200    
	00000070  31 30 30 36 34 34 20 20  31 33 39 31 20 20 20 20   100644  1391     
	00000080  20 20 60 0a 1f 8b 08 00  00 00 00 00 00 03 ed 59      ............Y 
	00000090  eb 6f db 36 10 f7 d7 f0  af b8 3a 5e 9b 74 b1 f5   .o.6......:^.t.. 
 

By then, systems were a.out(5), and everything was good. (Of course, if you look at the mtimes, you ll notice I faked this. But it s really equivalent to the real thing.

But oh horror! GNU binutils, not always everyone s friend, switched from using BSD style Unix Archiver libraries in ar(1) to SYSV style libraries on elf(5) systems:

	$ ar rc on-elf debian-binary control.tar.gz data.tar.gz
	$ hexdump -C on-elf   head
	00000000  21 3c 61 72 63 68 3e 0a  64 65 62 69 61 6e 2d 62   !<arch>.debian-b 
	00000010  69 6e 61 72 79 2f 20 20  31 33 31 33 36 38 33 35   inary/  13136835 
	00000020  32 39 20 20 31 30 30 36  20 20 32 30 30 20 20 20   29  1006  200    
	00000030  31 30 30 36 34 34 20 20  34 20 20 20 20 20 20 20   100644  4        
	00000040  20 20 60 0a 32 2e 30 0a  63 6f 6e 74 72 6f 6c 2e      .2.0.control. 
	00000050  74 61 72 2e 67 7a 2f 20  31 33 31 33 36 38 33 35   tar.gz/ 13136835 
	00000060  32 39 20 20 31 30 30 36  20 20 32 30 30 20 20 20   29  1006  200    
	00000070  31 30 30 36 34 34 20 20  31 33 39 31 20 20 20 20   100644  1391     
	00000080  20 20 60 0a 1f 8b 08 00  00 00 00 00 00 03 ed 59      ............Y 
	00000090  eb 6f db 36 10 f7 d7 f0  af b8 3a 5e 9b 74 b1 f5   .o.6......:^.t.. 
 

Can you spot the difference?

Of course, ELF is what you want , so there is little choice. Unix Archiver libraries are system dependent, and no format has ever been normed, but DEB files use it as format so what is one to do?

	$ GNUTARGET=a.out-i386-linux ar rc with-aout \
	> debian-binary control.tar.gz data.tar.gz
	$ md5sum pkg_1.0_all.deb with-aout on-elf
	248f78d42f8ca8f2a3560f9800b2bf01  pkg_1.0_all.deb
	248f78d42f8ca8f2a3560f9800b2bf01  with-aout
	09eca70c9b11b6b55bbadcab5c3201fb  on-elf
 

OK, and what do I do on my Debian/m68k system?

ar(1) uses bfd, and GNU binutils can not only forcibly set the target emulation but also show them:

Wow. While binutils share no single supported working target, they can be built multiarch, or (on MirBSD) with --enable-targets=all --enable-64-bit-bfd. Doesn t help if you want to stay portable: GNUTARGET=srec is common on all Debian (sid) binutils versions (single or multiarch), but errors out on older binutils. The a.out-* targets are not common. Sure, you could hack around things, but this is tedious. If you follow things or know me a little, you might already have guessed that I wouldn t let that stand.

pax(1) to the rescue. On MirBSD, we use paxtar, which has cpio(1) and tar(1) front-ends and supports multiple formats (4 cpio and 2 tar variants) and has already been extended a lot and is lovingly called paxmirabilis (mirabilos peace in Latin) it has options to anonymise archives: set uid and gid to zero, set mtime to zero, (for ustar) only write the numeric uid and gid to the archive, (for cpio formats) serialise inodes and device information, write content of hardlinked files only once (breaks partial extraction but saves a lot of space, e.g. 2 MiB off the Grml initrd.gz). And, recently, the ability to append a trailing slash to pathnames of ustar members which are directories (GNU tar does it and I thought some Debian utilities check for it). So why not (the -M dist and fakeroot set the uid/gid to 0)

	$ find debian-binary control.tar.gz data.tar.gz   \
	> mircpio -oHar -Mdist >with-mircpio
	$ mirpax -w -M dist -f with-mirpax -x ar \
	> debian-binary control.tar.gz data.tar.gz
	$ mirtar -M dist -A -cf with-mirtar \
	> debian-binary control.tar.gz data.tar.gz
	$ GNUTARGET=a.out-i386-linux fakeroot ar rc with-aout-ar \
	> debian-binary control.tar.gz data.tar.gz
	$ md5sum with-*
	a466e2fd57cdee141fe585a43245548f  with-aout-ar
	a466e2fd57cdee141fe585a43245548f  with-mircpio
	a466e2fd57cdee141fe585a43245548f  with-mirpax
	a466e2fd57cdee141fe585a43245548f  with-mirtar
 

Voil . I got it, and even appending is possible. It supports the BSD format with special focus on DEB files, and deals with long filenames, but not symbol or filename tables (used by ranlib(1) or strange formats, respectively, but since we don t create *.a files to use with some native linker/binder/loader, we don t need that anyway).

On extraction (oh, and listing!) it deals with SYSV style filenames as well.

	$ mirtar tvf on-elf
	-rw-r--r--  1 tg       tg          4 Aug 18 16:05 debian-binary
	-rw-r--r--  1 tg       tg       1391 Aug 18 16:05 control.tar.gz
	-rw-r--r--  1 tg       tg      18135 Aug 18 16:05 data.tar.gz
	$ mirtar tvf with-aout
	-rw-r--r--  1 tg       tg          4 Aug 18 16:05 debian-binary
	-rw-r--r--  1 tg       tg       1391 Aug 18 16:05 control.tar.gz
	-rw-r--r--  1 tg       tg      18135 Aug 18 16:05 data.tar.gz
 

One of the real benefits is that you can use the front-ends interchangably for example, mirtar tzf foo.cpio.gz would work (which GNU tar can t do), and mircpio s ustar implementation, unlike GNU cpio s, is not horribly broken. Of course, there are some drawbacks: it s not GNU tar or GNU cpio, so there are absolutely zero --long-options. Some of their features are missing (but tar s -O is implemented now), so it s no replacement (but very well usable alongside it). The format called pax, committee-designed to replace ustar, isn t yet supported ironically, but that s on the TODO.

So, what do you think?

	tg@frozenfish:~/Debs/dists/sid/wtf/Pkgs/mircpio $ ll *.deb
	-rw-r--r-- 2 tg freewrt 78140 Aug 17 11:04 mircpio_20110817-0wtf2_amd64.deb
	-rw-r--r-- 3 tg freewrt 72262 Aug 17 11:00 mircpio_20110817-0wtf2_i386.deb
	-rw-r--r-- 1 tg freewrt 67446 Aug 17 18:21 mircpio_20110817-0wtf2_m68k.deb
 

Should I upload this to Debian proper? As for the licence: 3-clause UCB (and 2-clause BSD, which is a subset of it), so no problem. I m asking because the other package which I had been using for a long time and not uploaded, jupp, got uploaded recently (during DebConf) on user input (people wondered why it did not yet exist in Debian proper). I guess the old saying if it s not in Debian, it doesn t exist holds true in many parts of the OSS world. It s up to date wrt. standards btw, and lintian-clean save for two pedantic-class warnings (no upstream changelog file, no homepage link) which aren t fulfillable (could link this wlog entry as homepage). If you know Alioth you re familiar with the software formerly known as SourceForge, formerly known as GForge, currently known as FusionForge. My employer both uses it and contributes to it, we run an adapted (mostly themed, prototyping new functions that often end up in FusionForge itself, and backporting functions from FF to our production codebase ) version.

I ve backported the extratabs plugin to appease project managers and other non-technical people while we move our codebase to FF 5.1, and I did so on an installed version of the plugin rather than the source because the latter was tightly integrated with rather heavy packaging style changes.

	[ ]
	dh_builddeb
	# create fusionforge-plugin-extratabs binary package
	toplev=$$(pwd); cd plugins/fusionforge-plugin-extratabs; \
	p=$$(print -r -- $$(sed -n '/^Package: /s///p' C/control   head -1)); \
	v=$$(print -r -- $$(sed -n '/^Version: /s///p' C/control   head -1)); \
	a=$$(print -r -- $$(sed -n '/^Architecture: /s///p' C/control   head -1)); \
	d=$$ p _$$ v _$$ a .deb; \
	rm -f $$toplev/../$$d control.tar.gz data.tar.gz; \
	(cd control; find .   fgrep -v /.svn   sort   \
	    mircpio -oC512 -Hustar -M0x0B -Mgslash)   gzip -n9 >control.tar.gz; \
	(cd data; find .   fgrep -v /.svn   sort   \
	    mircpio -oC512 -Hustar -M0x0B -Mgslash)   gzip -n9 >data.tar.gz; \
	mirtar -M dist -Acf $$toplev/../$$d debian-binary cont*gz dat*gz; \
	rm -f control.tar.gz data.tar.gz; \
	cd $$toplev; dpkg-distaddfile $$d non-free/devel optional
 

The hardest part of extending debian/rules with that was to get the autobuild and dpkg-distaddfile call right. This works, even though I d call it a temporary kludge. (No need to tell me I should have used && I know. And I only shell out to mksh(1) because the inner part was already there from before, when I still used ar(1). This was slightly edited for the wlog.) In the meanwhile, apt-extracttemplates can deal with SYSV style filenames in DEB files on Debian sid, but not on Kubuntu hardy, which some people are using as Desktop OS still

No comment Liked this article? Click here. My blog is Flattr-enabled.

I m writing to announce three big changes for the project. First, OpenHatch is changing its organizational structure to reflect our not-for-profit goals. Second, we ll emphasize our new work beyond the website, building and promoting outreach events that bring new people into the free software community. Finally, I am taking a year to do that full-time as the project lead of OpenHatch in Somerville, MA.

1. It starts with a boot manager which is similar to the temporary boot device selection menu of most recent BIOSes.
2. When you have selected the device to boot from, it looks into its EFI System partition, identified by its type GUID, as FAT file system, and executes a boot loader from a file at a specific path: /efi/boot/bootx64.efi (on amd64 platforms) by default, specially for removable devices.
3. Once the boot loader is loaded, the boot process continues as usual and is no longer specific to EFI (although the ABI used by the boot loader is specific, of course).

• Multi-arch is finally with us! Great talk from Steve Langasek, and a very useful meeting afterwards to unpick what work is still needed. Quite a bit, it seems... :-)
• Useful meetings with Otavio to discuss future changes to how debian-installer is going to work, and how the debian-cd and debian-installer teams can work together better on that.
• Catching up with Martin to work out what's needed to get pancutanfinally integrated with our regular CD builds.
• Talking with Jimmy about where to go next for regular debian-live builds on our core CD build machine.
• The ARM BoF, where things have moved on from "why should I care about ARM?" to "when will you have armhf ready for going in the archive?". W00t!
• Doing our own day trip in the cars from the UK. We got to meet some very friendly and helpful local people, plus we found some incredible roads to drive on. Even if not all of them were technically roads yet... :-)
• Playing poker in a real casino for the first time, along with a bunch of friends. Nice atmosphere, and very good fun.

Related articles

• Network connections on Ubuntu (flatchat.wordpress.com)

$ lintian --profile emdebian-grip drivel_3.0.2-1em1_amd64.deb 

$ lintian --profile debian drivel_3.0.2-1em1_amd64.deb 

E: drivel: debian-changelog-file-missing

E: drivel: copyright-file-compressed

W: drivel: copyright-without-copyright-notice

E: drivel: description-contains-invalid-control-statement

W: drivel: binary-without-manpage usr/bin/drivel