Search Results: "tar"

24 June 2013

Daniel Pocock: Private WANs may be less secure than VPNs

The latest round of Snowden revelations concern a British GCHQ program dubbed "Mastering the Internet (MTI)". The program involves, among other things, tapping the world's under-sea fibre-optic cables and systematically monitoring all communications. One of the most significant facts that appears is the benefit of using IPsec and VPNs over the public Internet instead of trusting un-encrypted traffic to pass over private WAN connections. The definition of a private WAN connection is quite broad: many ISPs in the UK, for example, use the British Telecom OpenReach platform to tunnel DSL connections back to their data centers and offer virtual WANs to their business customers. Can any wifi be trusted? It should be assumed that all Wifi networks can be compromised at-will by a sophisticated attacker. This is another situation where VPNs may be part of the solution: even WPA2 secured wifi segments should be treated as untrusted public Internet networks and not bridged or routed onto any private subnets. Users with devices on a wifi network should be forced to use a VPN over wifi connection, even when the wifi is within the same building as the private network. Tor may be compromised by an attacker with this much data As I've mentioned in previous blogs with a focus on real-time communications there is no such thing as perfect privacy. Simply encrypting data with IPsec does not prevent the attacker knowing the volume of data transmitted or the time it was transmitted. Using a service like Tor for onion-routing provides some benefit. However, an attacker with sufficient infiltration of many points in the network may be able to deduce the real origins of a data transfer over Tor using some combination of packet sizes and timestamps. "Mastering the Internet (MTI)" may have sufficient coverage to achieve that. Tor could overcome this by putting dummy data into the network and adding fragmentation, but such schemes may add latency, reduce throughput and require significant effort to upgrade existing users. Gaining intelligence from encrypted data VoIP signals are typically compressed using some type of codec. These are dedicated, lossy compression schemes optimised for voice signals. Some codecs use a fixed compression ratio. Other codecs use a variable compression ratio (VBR): a simple example involves sending shorter packets to indicate a period of silence. Encrypting these packets prevents an attacker seeing what is inside them but the attacker can easily detect the lengths of the packets. Researchers have demonstrated that statistical techniques can be applied to VBR dataflows to identify syllables used in speech and then make a hypothesis about what has been said. (This would be an interesting hack to try and reproduce by collecting data with tcpdump and analysing it with R) Identifying phrases in encrypted VoIP using statistical analysis In many cases like those described, it is necessary to go beyond default encryption settings and put extra dummy data or frame padding into the data flow to gain a more fool-proof level of privacy. Simply connecting VoIP phones to an IPsec network doesn't automatically make them secure. The rise of the free-software VPN Even if the default configurations are not always universally resilient to every type of attack, many free software prodcuts provide an excellent building block for VPNs. Having the source code and specifications on hand, free software developers can then innovate to mix-and-match different technologies or add random data flows to mask the real activity on the network. In a recent hosting migration project, I successfully used the rather trivial VPN package tinc to bridge the ethernet segments between virtual machines in the old and new networks. This allowed virtual machines to be moved one-by-one over a period of a week. This is a remarkably simple solution that can protect the integrity of communications on a private subnet. A more comprehensive solution is strongSwan which provides support for industry-standard IPsec in various topologies. They now offer a full IPsec client for Android phones (hopefully it will be made available on f-droid: using a Google account to access Google Play may result in Google accidentally helping you "back up" your settings, including credentials, to their cloud, undermining the privacy of the system) An IPsec VPN can be deployed to all hosts on a network: then they can use opportunistic encryption for any type of communication without needing to worry about the underlying network. Furthermore, it is possible to add smart cards to further boost VPN security while using free software for the whole solution. For those interested in looking beyond Linux solutions, the *BSD range of operating systems, particularly OpenBSD provide a compelling alternative. This paper about OpenBSD's iked, which is also available on other platforms as OpenIKEd is very recent and provides a thorough overview of it's current status. One notable limitation is that their MOBIKE support is not yet complete. Many older VPNs likely to need an ugprade When I first started deploying Linux VPNs in the 90s the market was favourable for any type of Linux VPN. In extreme cases (which were surprisingly common at the time) companies accustomed to paying $20,000 per month for a leased line would happily pay a flat fee of $100,000 to purchase a VPN that was cobbled together using less than $10,000 of commodity hardware and some free software. Many of the first generation of these VPNs were built using RSA key lengths and algorithms no longer considered secure, such as DES. Some of these solutions (or products derived from them) may still remain in use today: managers may be slightly embarrassed to put aside their $100,000 VPN and replace it with a $50 router that is more secure. While that is the more extreme example, there are many intermediate solutions out there in the field. There are clear opportunities here to upgrade users to free software solutions.

23 June 2013

Marko Lalic: Google Summer of Code - Debian Package Tracking System

Introduction
Every May for the last three or four years, I contemplated applying for the Google Summer of Code program. I was never quite sure I would be up for the challenge, though, but this year I finally took the plunge.

Since Python is my favorite language, I was looking to take up a project which could use it. When I saw that one of the possible projects which the Debian Project was offering was a rewrite of the Package Tracking System using Django, my choice became pretty clear. Not only would I use my favorite programming language, I could get a chance to interact with one of the largest and best established open source communities in the world. Moreover, the Package Tracking System is one of the most important parts of Debian's infrastructure and working on making it even better seemed like a perfect way to get into the open source world. The fact that I also had prior experience using Django on a Web project was an additional factor which pushed me to go for it.

Two programming tasks, meant to evaluate the students' knowledge and also give them a chance to get acquainted with the current implementation of the PTS, were given to prospective applicants by the Debian Google Summer of Code team. Neither was particularly challenging in and of itself, but since I had to attend the annual Congress of the Electrical Engineering Students' European Association in Munich during the week the applications were open, I was quite pressed for time. In the end, I managed to complete the application in a timely fashion and I submitted it being pleased that I at least made an attempt to get in the Summer of Code program.

Truthfully, when I got an email from Google on a Monday night congratulating me on being selected for the project, I could not believe my eyes. I needed to take about an hour to read the email again and make sure there wasn't some sort of mistake, but, as it turns out, there really was not. This is when excitement kicked in and I started to draft an email to my mentor for the project, Rapha l Hertzog, to start on the official planning for the summer.

PTS Rewrite in Django
The Package Tracking System is currently made up from a variety of different scripts, ranging from Bash, through Perl, to Python, which are used to pull all information about packages and generate a static HTML page for each of them. This process is repeated as a cron-job a few times each day meaning that the information presented on the pages could easily be outdated. So, a requirement for the new system is that the pages need to be as dynamic as possible.

Consolidating the code into a well thought out Django project hopes to make it easier for other developers to contribute to the project.

Finally, decoupling the application itself from Debian-specific resources, along with making it easy to customize or extend any logic, aims to allow derivatives to deploy their own Package Tracking System by simply making a few modifications to the configuration.

This is quite an ambitious set of goals, but I believe it should be attainable.

Community Bonding Period and the First Week
As I was already quite familiar with the existing codebase and PTS documentation by completing the tasks for the application, I did not need to use the community bonding period to get to know the project itself, but I did use the first few days to read up about using agile methods (particularly Scrum) and Test-Driven Development, since this was what my mentor, Rapha l Hertzog, and co-mentor, Stefano Zacchiroli, saw us using. Naturally, these were not new terms for me, but never having applied either an agile or a test-driven approach on a large scale project, I wanted to be as prepared as I could for the coming weeks.

I was quite eager to start working, though, and as it turns out, so were my mentors. When Rapha l made the suggestion to begin even before the official first week, I accepted and from there, my first "real" work on the project came: trying to define as many of the core stories to describe the system. This was followed by writing exhaustive acceptance tests and giving a rough time estimate on the stories relating to the email interface of the PTS. We are using a Trello board to document and track all this.

As we were planning on applying a Scrum-inspired agile approach, this meant having fixed-duration sprints/iterations which produce a working part of the system. Using a week long iteration seemed like a perfect fit for the situation. The other part, a working deployment, was handled by me setting up an Amazon Web Services EC2 instance where the application is deployed upon completing the week's work.

After all this, I started the coding work, trying to follow a Test-Driven Development approach. Properly applying it is sometimes a challenge, never having used it before, but its usefulness is definitely obvious. Being able to rely on this harness makes refactoring work painless and stress-free. Rapha l's constant feedback on this part is really helping me get into the flow of things and I hope that I will be quite proficient in it in a few weeks' time.

In summation, at the end of the first official week, on Friday, June 21 (during the third week since I started), a large chunk of the PTS's existing email functionality has been reimplemented and deployed. Some kinks and wrinkles are left to iron out, as well as add some long desired improvements to it, but, all in all, I would say that the two iterations produced a nice and usable result.

Anyone interested can already test this out... The email address control@pts.debian.net is used to access the control bot and the different commands. You can send it an email message containing the help command (the word help on a single line) to see the list of the currently implemented commands, along with their description. Currently, though, only a few packages are available to be subscribed to, imaginatively named "dummy-package", "second-package", "third-package" (quotes for clarity). The address for the package-specific messages to be forwarded to subscribers is <package-name>@pts.debian.net.

Conclusion
As the summer and my GSoC work is settling into a stable weekly-iteration rhythm, I have decided to set this blog up where I plan on writing a weekly update about the work being done for the PTS to complement my weekly status reports sent to the Debian GSoC coordination mailing list.

Again, I invite anyone who is interested in contributing to the future of the PTS in some way to test out the weekly deployments, contact me with any questions, feedback or advice, or simply follow the status updates on the blog, the Trello board or the project's Git repository.

Finally, I would like to give a great thank you to the Debian Project community, with the aforementioned mentors, Rapha l Hertzog and Stefano Zacchiroli, at the top, for giving me the vote of confidence to work on this project. The experience and knowledge I will get from it is immeasurable. Special thanks to Rapha l for putting a lot of effort and time into his feedback and advice; it really means a lot to me and I hope I use it in a way which will bring this project to a successful ending.

Gregor Herrmann: RC bugs 2013/24-25

thanks to lucas & his trainees, we're seeing archive rebuilds & tons of new RC bugs again. besides that, there are still some patches in the BTS. here goes my list of recently worked on bugs; most of them in the pkg-perl group

Paul Tagliamonte: Hacking a Computer from Scratch

I ve decided that I d rather like to hack an entire working (ok, not working", but programmable) computer (except for ROM and perhaps I/O) from scratch (just wires, perfboard, transistors (and friends) and solder). It sounds insane, but a simple ALU seems easy enough. Anyone do this before? Anyone have tips? Schematics of similar projects?

I ll be sure to blog my progress if / when I get around to starting it.

Ben Armstrong: Tutorial: wheezy live iso-hybrid with persistence on USB

As of wheezy, Debian no longer offers for download any prebuilt hdd (formerly known as usb-hdd) live images, which, in squeeze, were necessary if you wanted to use persistence. The good news is, since the new wheezy iso-hybrid images use xorisso, you can copy an image to a USB key and repartition it to add a persistence partition. Unfortunately, the filesystem itself is read-only, so you have to manually append the persistence boot parameter each time you boot the image. If that doesn t bother you, you can skip this whole tutorial and just follow the instructions in live-manual for adding a persistence partition and you re all set. This tutorial explains how to extract the contents of a wheezy iso-hybrid live image to a USB key and modify it to boot with this parameter permanently enabled. I am writing it solely as a workaround and do not endorse this as a means to customize Debian Live images in general. I would love to help make a more convenient, supported way to accomplish the same thing for a future release. Meanwhile, the supported and recommended way to make any change to a Debian Live image is to build it from scratch using live-build, as described in live-manual. A simplified Debian Live Images Autobuilder web service is available that works well for many common customizations, if you prefer. Please use one of these methods if you want to do this, or other customizations in the supported way. Otherwise, read on. Prerequisites Identify the USB key device Caution: Always double-check the device you are writing to is the correct one to avoid losing precious data. There are two things to do to protect yourself. The first is, don t write the USB key as root. The second is, first use ls -l /dev/disk/by-id to identify which device is the target USB key.
lrwxrwxrwx 1 root root  9 Jun 22 11:58 
usb-SanDisk_Cruzer_Contour_0000184CA87406BC-0:0 -> ../../sdb
As you can see, my SanDisk Cruzer Countour is /dev/sdb. For the rest of the tutorial, I ll be referring to this device as /dev/sdX to avoid unfortunate cut-and-paste disasters. When you follow along, make sure you substitute the correct device for your USB key on your system. Initialize the USB key Start by plugging in the key and unmounting any automounted partitions to ensure nothing is accessing the device, e.g.
$ umount /dev/sdX1
Now make changes to the partition table. Use parted because it may be run as an ordinary user and has some advanced capabilities beyond more basic tools like fdisk.
$ /sbin/parted
WARNING: You are not superuser.  Watch out for permissions.
GNU Parted 2.3
Using /dev/sdX
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print devices
/dev/sdX (8221MB)
(parted) select                                                           
New device?  [/dev/sdX]?                                                  
Using /dev/sdX
(parted)
Note that when run as an ordinary user, if you only have one removable drive (which may include mp3 players, cameras, e-readers) plugged in, parted immediately selects that device. But just to be sure, use print devices to list the devices and select to if you need to select a different one, as shown above. Repartition the USB key Once you ve verified you have the correct device, start repartitioning. Write a new partition table with mklabel msdos to wipe out all of the old partitions on it. You may skip this step and adapt the remaining instructions if there are partitions you need to keep. This is your last chance to bail if the device is wrong, so make sure it is the correct one before proceeding.
(parted) mklabel msdos
Warning: The existing disk label on /dev/sdX will be destroyed and all data
on this disk will be lost. Do you want to continue?
Yes/No? y
Error: Partition(s) 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 on
/dev/sdX have been written, but we have been unable to inform the kernel of
the change, probably because it/they are in use.  As a result, the old
partition(s) will remain in use.  You should reboot now before making further
changes.
Ignore/Cancel? i
(parted)
The error is normal, and appears when you run parted as an ordinary user, so it is OK each time it appears to press i to ignore it. For the rest of this tutorial I will omit these errors from the output for brevity. Make three partitions, a fat32 partition for the image itself that is 5% larger than the size of the ISO, a 1G ext4 partition for persistence, and an extra fat32 partition for the rest of the space. Put the extra partition first so that other OSes you might use the key with will be able to use it. Use relative positioning from the end of the device to make it easy to put all of the rest of the space in the first partition. Finally, flag the live image partition bootable.
(parted) mkpart primary fat32 1 -1897M
(parted) mkpart primary ext4 -1897M -897M
(parted) mkpart primary fat32 -897M -0
(parted) set 3 boot on
(parted) quit
Information: You may need to update /etc/fstab.
Unplug the USB key and plug it in again so that the kernel will re-read the updated partition table. If there were old filesystems on the key that were automounted, unmount them again. Now we re ready to make the new filesystems.
$ umount /dev/sdX1
$ /sbin/mkdosfs -nEXTRA /dev/sdX1
mkdosfs 3.0.16 (01 Mar 2013)
$ /sbin/mkfs.ext4 -q -Lpersistence /dev/sdX2
$ /sbin/mkdosfs -nLXDE /dev/sdX3
mkdosfs 3.0.16 (01 Mar 2013)
Make the USB key bootable Use install-mbr to install an MBR on the key and syslinux to install the bootloader to boot into your live image partition.
$ /sbin/install-mbr /dev/sdX
$ syslinux -i /dev/sdX3
Mount the partitions Unplug the key and plug it in again so the new partitions will be automounted. Observe where the partitions are mounted with df.
$ df
...
/dev/sdX1        6204568         4   6204564   1% /media/EXTRA
/dev/sdX3         831888         4    831884   1% /media/LXDE
/dev/sdX2         944120      1204    894124   1% /media/persistence
Extract the ISO contents to the key To continue preparing this key as an unprivileged user, use p7zip to extract the ISO. You may alternatively mount the ISO loopback and extract it that way, but you must be root to do that.
$ cd /media/LXDE
$ 7z x ~/debian-live-7.0.0-amd64-lxde-desktop.iso
...
Extracting  live/filesystem.packages-remove
...
When live/filesystem.squashfs is extracted (right after live/filesystem.packages-remove) it will take the longest time of all of the files to extract, as it contains the whole live filesystem. Be patient and eventually it and all other files on the image will finish extracting.
...
Everything is Ok
Folders: 245
Files: 370
Size:       892183367
Compressed: 895483904
Modify the bootloader configuration files The syslinux bootloader configuration directory and files within it are named isolinux when installed on an ISO. You need to rename the directory and two files changing isolinux to syslinux so the bootloader will find them on your fat32 live image partition.
$ mv isolinux syslinux
$ mv syslinux/isolinux.cfg syslinux/syslinux.cfg
$ mv syslinux/isolinux.bin syslinux/syslinux.bin
Enable full persistence Next, append persistence to the live boot parameters and turn on full persistence by putting / union in a persistence.conf file in the persistence partition.
$ sed -i 's/\(append boot=.*\)$/\1 persistence/' syslinux/live.cfg
$ cd /media/persistence
$ echo / union > persistence.conf
Reboot into the live system Now your USB key is ready to boot with full persistence enabled. Have fun!

22 June 2013

Justus Winter: My first week and some promising though hacky results

Hi Planet Debian :) my name is Justus Winter, I'm a computer science student at the University of Hamburg. I'm participating in the GSOC 2013, my task is to make Debian/Hurd systems boot using the same initialization scripts Debian/Linux uses. I spent the community bonding period with some gsoc related administrative tasks, updating my development machine and getting reacquainted with the Hurd system. It was nice to see how far Debian/Hurd has come since my last "serious" encounter with it. I've setup a continuous build solution on one of my Hurd VMs and prepared a package repository. I'll post details once I have some packages to share. I built the hurd, sysvinit and ifupdown packages and dived into the interesting world of literate programming. I'm going to write about that when I start looking into the ifupdown related issues. But most importantly I started writing a tool to automate all the tasks I would do during my work that could possibly be automated. As I'm working on the boot process, it makes sense to use some kind of virtualization software so I can work in the comfort of my usual development environment. My virtualization solution of choice is qemu and I actually wrote a qemu based black-box test solution for a former employer once, so I already was familiar with qemu and wrapping my own code around it. Without further ado, here it is: https://bitbucket.org/teythoon/hurdtest Provided that you have golang installed, you can compile it easily:
teythoon@europa /tmp % export GOPATH= pwd 
teythoon@europa /tmp % go get bitbucket.org/teythoon/hurdtest
teythoon@europa /tmp % bin/hurdtest
Usage: hurdtest [options] <command> [<arguments...>]
Currently it has the following subcommands:
test
Starts a VM, logs in as root, tests the interaction with the VM, verifies that certain VFS locations are writable, tests the network connectivity, the DNS resolution and installs and runs the hello package.
update
Executes aptitude update && aptitude full-upgrade, reboots the system and runs the test subcommand. If successful, it commits the changes to the virtual disk and creates a snapshot.
overlay
Takes as argument the path to a directory that is copied to the VM, reboots the machine and runs the test subcommand.
The update command is nice because it allows me to keep the VM up to date while ensuring that the image remains usable, with respect to what the test command considers a usable state of a Debian system. I'd really like to get some suggestions on how to test the "health" of a Debian system. You can look at how the tests are implemented in test.go, it's quite easy to add more and more interesting tests, and you don't even have to clean up after them or worry about any side-effects as the changes are not committed to the virtual disk. This is hurdtest doing an update:
teythoon@europa ~/repos/hurd/hurd-tests % bin/hurdtest update
2013/06/22 19:59:45 starting kvm [-drives file=image.qcow2,cache=writeback,snapshot=on -serial unix:/tmp/285813369/serial -monitor unix:/tmp/285813369/monitor -nographic -m 512]
2013/06/22 19:59:45 Waiting for grub...
2013/06/22 19:59:46 Waiting for login prompt...
2013/06/22 20:00:05 Updating system
2013/06/22 20:02:52 Get: 1 http://www.local unstable InRelease [8279 B]
Get: 2 http://http.debian.net sid InRelease [205 kB]
Get: 3 http://ftp.debian-ports.org unreleased InRelease [23.9 kB]
[...]
Setting up perl (5.14.2-21) ...
Reading package lists...
Building dependency tree...
Reading state information...
Reading extended state information...
Initializing package states...
Reading task descriptions...
2013/06/22 20:02:52 Update successful. Snapshotting system...
2013/06/22 20:02:53 Rebooting system...
2013/06/22 20:02:54 Waiting for grub...
2013/06/22 20:02:59 Waiting for login prompt...
2013/06/22 20:03:16 Running main.testTrue...
2013/06/22 20:03:16 Running main.testFalse...
2013/06/22 20:03:17 Running main.testClean...
[...]
2013/06/22 20:03:18 Running main.testNetwork...
2013/06/22 20:03:18 Running main.testResolver...
2013/06/22 20:03:20 Running main.testAptitudeInstall...
2013/06/22 20:03:46 Test successful. Reverting system...
2013/06/22 20:03:48 Halting system...
2013/06/22 20:03:53 Committing changes...
2013/06/22 20:03:57 Creating snapshot...
2013/06/22 20:04:04 All done.
bin/hurdtest update  157.31s user 69.71s system 87% cpu 4:18.94 total
The overlay command implements my work flow. I can edit scripts or drop hacked up executables onto my VM and see the results without any manual intervention. There's the -interactive flag that does this in a loop, avoiding the VM startup time using qemu snapshots. With this I can apply an overlay and see the results of the test suite in about a minute on my system. From early experiments I knew that simply running /etc/rcS.d/S* prevented the system from booting correctly, so I figured my first task should be to figure out what went wrong there. Using my tool (and developing it in that process) and many iterations later I came up with a surprisingly small diff of scripts in the initscripts and utilities shipped with the hurd package. For your convenience here is a tarball that you can extract to the root of your Debian/Hurd installation to reproduce my results (of course I invite you to use my tool for that purpose):
% bin/hurdtest overlay test
2013/06/22 21:35:31 starting kvm [-drive file=image.qcow2,cache=writeback,snapshot=on -serial unix:/tmp/345188134/serial -monitor unix:/tmp/345188134/monitor -nographic -m 512]
2013/06/22 21:35:31 Waiting for grub...
2013/06/22 21:35:31 Waiting for login prompt...
2013/06/22 21:35:48 Creating tarball...
./
./etc/
./etc/hurd/
./etc/hurd/rc
./etc/init.d/
./etc/init.d/checkroot.sh
./etc/init.d/mountall.sh
./runme
./lib/
./lib/init/
./lib/init/mount-functions.sh
./sbin/
./sbin/swapon
./bin/
./bin/mount
2013/06/22 21:35:48 Extracting overlay...
2013/06/22 21:35:49 Rebooting system...
2013/06/22 21:35:49 Waiting for grub...
2013/06/22 21:35:55 Writing log to /tmp/bootlog-802006344
2013/06/22 21:35:55 Waiting for login prompt...
2013/06/22 21:36:20 Booting successful.
2013/06/22 21:36:20 Running main.testTrue...
2013/06/22 21:36:21 Running main.testFalse...
2013/06/22 21:36:21 Running main.testClean...
2013/06/22 21:36:21 Running anonymous function...
2013/06/22 21:36:21 Running anonymous function...
2013/06/22 21:36:21 Running anonymous function...
2013/06/22 21:36:21 Running anonymous function...
2013/06/22 21:36:21 Running anonymous function...
2013/06/22 21:36:22 Running anonymous function...
2013/06/22 21:36:22 Running main.testUname...
2013/06/22 21:36:22 Running main.testNetwork...
2013/06/22 21:36:22 Running main.testResolver...
2013/06/22 21:36:23 Running main.testAptitudeInstall...
bin/hurdtest overlay ../hurd-tests-old/test  44.20s user 20.01s system 82% cpu 1:17.94 total
And here is the boot log:
Loading GNU Mach ...
Loading the Hurd ...
GNU Mach 1.3.99-486
AT386 boot: physical memory map from 0x0 to 0x9f400
AT386 boot: physical memory map from 0x100000 to 0x1fffe000
AT386 boot: physical memory from 0x0 to 0x1fffe000
Enabling FXSR
pcibios_init : BIOS32 Service Directory structure at 0xfcff0
pcibios_init : BIOS32 Service Directory entry at 0xfc7ba
pcibios_init : PCI BIOS revision 2.10 entry at 0xfc78c
Probing PCI hardware.
ide: Intel 82371 PIIX3 (dual FIFO) DMA Bus Mastering IDE
    Controller on PCI bus 0 function 9
ide: BM-DMA feature is not enabled (BIOS), enabling
    ide0: BM-DMA at 0xc100-0xc107
    ide1: BM-DMA at 0xc108-0xc10f
hd0: got CHS=762/128/63 CTL=c8 from BIOS
hd0: QEMU HARDDISK, 3001MB w/256kB Cache, CHS=762/128/63, DMA
hd2: QEMU DVD-ROM, ATAPI CDROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
Floppy drive(s): fd0 is 1.44M
FDC 0 is a S82078B
probing scsi 0/22: advansys probing scsi 1/22: BusLogic probing scsi 2/22: UltraStor 14F/34F rev. 4.33.00  probing scsi 3/22: UltraStor 14F/24F/34F probing scsi 4/22: Adaptec 152x SCSI driver; $Revision: 1.1 $ probing scsi 5/22: Adaptec 1542 Invalid address for shpnt with 1542.
Invalid address for shpnt with 1542.
probing scsi 6/22: Adaptec 174x (EISA) probing scsi 7/22:  probing scsi 8/22: Always IN2000 probing scsi 9/22:  probing scsi 10/22:  probing scsi 11/22:  probing scsi 12/22: Trantor T128/T128F/T228 probing scsi 13/22: DTC 3180/3280  probing scsi 14/22: Tekram DC390(T) V1.11 Feb-05-1997 probing scsi 15/22:  probing scsi 16/22: ncr53c8xx - revision 2.5f.1 probing scsi 17/22: EATA (Extended Attachment) PIO driver probing scsi 18/22: Western Digital WD-7000 Failed initialization of WD-7000 SCSI card!
probing scsi 19/22: EATA/DMA 2.0x rev. 4.33.00  probing eata on 1f0EATA0: address 0x1f0 in use, skipping probe.
probing eata on 1c88probing eata on 2c88probing eata on 3c88probing eata on 4c88probing eata on 5c88probing eata on 6c88probing eata on 7c88probing eata on 8c88probing eata on 9c88probing eata on ac88probing eata on bc88probing eata on cc88probing eata on dc88probing eata on ec88probing eata on fc88probing eata on 170EATA0: address 0x170 in use, skipping probe.
probing eata on 230probing eata on 330probing scsi 20/22: AM53C974 probing scsi 21/22: Iomega parport ZIP drive ppa: Version 1.42
ppa: Probing port 03bc
ppa: Probing port 0378
ppa:     SPP port present
ppa:     PS/2 bidirectional port present
ppa: Probing port 0278
done
scsi : 0 hosts.
scsi : detected total.
Partition check (DOS partitions):
 hd0: hd0s1 hd0s2 < hd0s5 >
lpr0: at atbus0, port = 378, spl = 6, pic = 7.
module 0: ext2fs --readonly --multiboot-command-line=$ kernel-command-line  --host-priv-port=$ host-port  --device-master-port=$ device-port  --exec-server-task=$ exec-task  -T typed $ root  $(task-create) $(task-resume) module 1: exec /hurd/exec $(exec-task=task-create)                                                                                                                                                                           2 multiboot modules                                                                                                                                                                                                                    task loaded: ext2fs --readonly --multiboot-command-line=root=device:hd0s1 console=com0 --host-priv-port=1 --device-master-port=2 --exec-server-task=3 -T typed device:hd0s1
task loaded: exec /hurd/exec
start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] exec init proc auth
Using makefile-style concurrent boot in runlevel S.
ERROR: could not open /proc/stat: No such file or directory
Activating swap...done.
Checking root file system...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
ext2fs_check_if_mount: Can't check if filesystem is mounted due to missing mtab file while determining whether /dev/hd0s1 is mounted.
/dev/hd0s1: clean, 44328/181056 files, 288266/723200 blocks
done.
Cleaning up temporary files... /tmp.
mount: /run: Unknown device or filesystem
mount: /run/lock: Unknown device or filesystem
mount: /proc: Unknown device or filesystem
mount: /run/shm: Unknown device or filesystem
/etc/rcS.d/S06mtab.sh: 48: /etc/rcS.d/S06mtab.sh: cannot open /proc/mounts: No such file
Activating lvm and md swap...(default pager): Already paging to partition hd0s5!
done.
Checking file systems...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
done.
Mounting local filesystems...mount: invalid option -- 'O'
Try  mount --help' or  mount --usage' for more information.
failed.
Activating swapfile swap...(default pager): Already paging to partition hd0s5!
done.
df: Warning: cannot read table of mounted file systems: No such file or directory
Cleaning up temporary files....
Configuring network interfaces...inetutils-ifconfig: invalid arguments
ifup: failed to open pid file /run/network/ifup-/dev/eth0.pid: No such file or directory
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
Listening on Socket//dev/eth0
Sending on   Socket//dev/eth0
DHCPDISCOVER on /dev/eth0 to 255.255.255.255 port 67 interval 5
DHCPREQUEST on /dev/eth0 to 255.255.255.255 port 67
DHCPOFFER from 10.0.2.2
DHCPACK from 10.0.2.2
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
bound to 10.0.2.15 -- renewal in 36884 seconds.
done.
Cleaning up temporary files....
Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix.
Using makefile-style concurrent boot in runlevel 2.
Starting enhanced syslogd: rsyslogd.
Starting deferred execution scheduler: atd.
Starting periodic command scheduler: cron.
Starting system message bus: dbusFailed to set socket option"/var/run/dbus/system_bus_socket": Protocol not available.
Starting OpenBSD Secure Shell server: sshd.
GNU 0.3 (debian) (console)
login:
I know it isn't that pretty, but it yields a working Debian/Hurd system that is more like a Debian/Linux system. For example /dev/shm and /run ,/* are connected to a tmpfs translator. I think this is a rather encouraging first result. I added my findings to the list of open issues. Another nice thing about hurdtest is that it is not even Hurd specific (only a tiny bit GNU Mach specific). I'm going to make it work nicely with Debian/Linux and Debian/kFreeBSD, since this is a very handy tool for anyone hacking on the Debian boot process (and with some changes for any OS). If anyone tries it out, I'd love to get feedback and patches. The tool requires the guest to be configured to use the first serial console for grub, the kernel and the userspace. The password for the user root must be empty. The guest has to have a working network connection setup at boot time. I will provide a suitable image as soon as I devise a good way to zero-fill the free space on the virtual disk. Is there any better way than to fill the disk with a huge file containing zeros? Without doing that the image is too large even compressed with xz. So this was my first week working for Debian. Next week I will focus on getting patches for the hurd and initscripts package in shape, I need to add some functionality to the mount utility shipped with the hurd package. It provides a mount-like experience on the Hurd, on which binding a translator (e. g. /hurd/ext2fs) to a filesystem node is an operation that does not involve the kernel (GNU Mach) at all, and is in fact a operation that any unprivileged user may perform. That is one of the nice features the Hurd has, it empowers its users.

21 June 2013

Joachim Breitner: Haskell and Debian talk at HaL8

I just finished my Haskell und Debian talk at the Haskell-Workshop HaL8 in Leipzig. Unfortunately I was thrown a off track by time constraints and since there were much less beginners and interested visitors in the audience than I had anticipated, so I skipped some parts and improvised others somewhat chaotically, so the presentation was not up to the standards that I expect from my talks. If you have attended (or if you have not) I recommend you have a look at the extended abstract (in German), which contains what I skipped and is much clearer than what I said.

Gerfried Fuchs: Happy T-Shirt Day!

Today is T-Shirt Day. There are many sites supporting the day with special offers, and spreadshirt is no exception: they don't charge shipping (use "tday2013" as gift code). I started somewhen last year to put together some shirts that I mainly did for myself, but in case you like my ideas, feel free to grab one or another shirt from my spreadshirt shop. Enjoy!

/misc permanent link Comments: 0 Flattr this

MJ Ray: Cooperatives Fortnight 2013

[Co-operatives Fortnight image] Cooperatives Fortnight 2013 starts tomorrow. Our co-op is taking part in a few activities this year. Come along and meet us, or find other events on the national website. This Sunday, I ll be riding 50 miles in support of Leonard Cheshire. Some of it is familiar roads, some of it new. I ll also be riding 12 miles just to get to the start line. If you re a UK resident and would like to guess my time and make a small donation, please give it a try on guess2give. Next Thursday 27 June, I ll be at Somerset Cooperative Services AGM in Taunton. See their site for further news. There might be something on Monday 1 July. To be announced later maybe. On Thursday 4July, mjkaye will be at Building Our Co-ops in Liverpool. See the national event listing for details. So what will you be doing for co-ops fortnight?

Lars Wirzenius: How long is a string: how many backups are enough?

You've set up a backup repository, and you have been backing up to it every day for a month now: your backup history is getting long enough to be useful. Can you be happy now? Welcome to the world of threat modelling. Backups are about insurance, of mitigating small and large disasters, but disasters can strike backups as well. When are you so safe you no disaster will harm you? There is always a bigger disaster waiting to happen. If you backup to a USB drive on your work desk, and someone breaks in and steals both your computer and the USB drive, the backups did you no good. You fix that by having two USB drives, and you keep one with your computer and the other in a bank vault. That's pretty safe, unless there's an earth quake that destroys both your home and the bank. You fix that by renting online storage space from another country. That's quite good, except there's a bug in the operating system that you use, which happens to be the same operating system the storage provider uses, and hackers happen to break into both your and their systems, wiping all files. You fix that by hiring a 3D printer that prints slabs of concrete on which your data is encoded using QR codes. You're safe until there's a meteorite hits Earth and destroys the entire civilisation. You fix that by sending out satellites with copies of your data, into stable orbits around all nine planets (Pluto is too a planet!) in the solar system. Your data is safe, even though you yourself are dead from the meteorite, until the Sun goes supernova and destroys everything in the system. There is always a bigger disaster. You have to decide which ones are likely enough that you want to consider them, and also decide what the acceptable costs are for protecting against them. A short list of scenarios for thinking about threats: These questions do not cover everything, but they're a good start. For each one, think about: The threat modelling here is about safety against accidents and natural disasters. Threat modelling against attacks and enemies is similar, but also different, and will be the topic of the next episode in the adventures of Bac-Kup.

20 June 2013

Matt Zimmerman: Scaling Human Systems: Alignment

This is part 1 in a series on organizational design and growth. An important lens for thinking about organizations at this stage of growth is alignment. In an organization which is aligned, the efforts of different people and teams all contribute to forward progress in a shared direction. If two teams are pulling in opposite directions they may make little progress despite great effort, and quickly become frustrated. To take an obvious example, if a marketing team is targeting an audience of large enterprises while the product being developed is only suited for small businesses, the end result of both teams doing a good job will be a failure (i.e. unhappy customers). It s important to note that alignment does not imply sameness. Different teams within an organization can function and behave very differently while still being strongly aligned with each other. When an organization is small, alignment comes naturally. Everyone has some visibility on what everyone else is doing, and when something doesn t line up, the people involved can talk it over and resolve the issue relatively easily. But as the organization grows, the propensity for misalignment increases, and these situations become much more difficult and time-consuming to resolve. The metaphor of the right hand, which doesn t know what the left hand is doing, seems like something that would only happen in larger companies, but it begins much earlier, especially when the company goes through a period of rapid growth. Critical infrastructure, such as communication tools and patterns, lags behind the accelerating needs of the people involved, creating a surprising distance between teams. Organizational alignment is a critical part of scaling successfully. With alignment, growth and momentum are assets. Without it, they are liabilities. Further reading: The Advantage: Why Organizational Health Trumps Everything Else in Business

Craig Small: jqplot in Turbogears

I ve been working on the Rosenberg jqPlot GraphsNMS graphs slowly migrating them from using rrdtool graph and using jqplot. While there have been many false-starts and re-works, I now have a working set of graphs, two of which are shown on the page. The graphs look a lot slicker and I have also simplified the admin screens. I found I kept having to type the same thing in over and over for the rrdgraphs and have narrowed down the type of graphs to approximately 5. They re certainly not bulletproof and need more testing but they re a good start. The graphs are based upon the ToscaWidgets2 series of widgets that then provide a nice handle for the jqPlot code. My graphs even have some hand-coded Javascript to give nice units on the Y axis.

Daniel Pocock: Wrong cloud: Victorian schools and Ultranet

wrong cloud
"The blood, sweat and tears that has gone into the Ultranet and the work teachers put in it's soul destroying. I have to face parents who took me on face value when I said: 'This is the best thing since sliced bread every school is going to be using it in the future."
Those are the words of a school principal at Alkira Secondary College in Victoria, Australia. His school is receiving a real-life lesson in the the risks of cloud computing. Ultranet, the state-wide private cloud project for schools is at risk of being cancelled in 10 days due to budget cuts. It is not clear whether the vendor, NEC, is obliged to release data back to the users if the original four year contract is not extended. Commercial, confidential discussions It appears that the possibility of ending the contract was not supposed to be public knowledge. If the finance department had kept their intentions secret then it is possible that users would not even have had the opportunity to try and manually extract some of their data, as they are rushing to do now. Instead of getting on with the business of education itself, teachers and students are having to waste time trying to individually download student projects, photos, feedback and other meta data from the system in an ad-hoc manner. An opportunity for free and open source software This debacle may well be a huge opportunity for the free and open source software community to remind people that the value of retaining control over user data can not be forgotten when evaluating outsourced, private cloud solutions that claim to be "free" or offer low costs to get started. There were apparently 18 schools who initially trialed the program and many more have gradually joined up. This would be the perfect opportunity for advocates of genuinely free software solutions to contact them and let them know they are not alone and that many of us have anticipated these problems for years. While central government administrators tend to be hooked on the idea of administering big contracts with trusted' vendors, these local school administrators and the communities around them are probably more receptive to the free software message than at any other time. List of schools using Ultranet Many of the schools using Ultranet can be found with Google, as it appears to be integrated in their web sites. Click here for a Google search that finds many of the schools and related technical pages, support documents and other material A message to schools everywhere For schools facing this crisis or considering adopting such services in other countries, I would share this very simple advice: if you are offered a solution that doesn't give you complete control to conveniently extract all of your data in bulk whenever you want and use it in your own software on any computer, then it is not a good foundation to build on. The free software community has been actively working to provide truly free solutions for schools and I would strongly encourage schools to look at projects like Skolelinux/Debian-Edu and k12Linux that have been built by educators who also have a deep understanding of software and data freedom issues.

MJ Ray: What s the current state of Windows Anti-Virus?

One of our co-op s clients asked me what I use for anti-virus at the moment and tips for what they should use on their Windows system. Well, flame me now, but I don t actually use any anti-virus at the moment: I rely on system security, firewalling and intrusion detection. The diversity of GNU/Linux software and I use some pretty odd stuff probably helps too. Even if I did want to run antivirus software, most of what s available for GNU is actually aimed at detecting and preventing transmission of Windows viruses. There are few real-world GNU viruses and fewer attack opportunities left open. Also, I prefer firewalling and fairly paranoid security settings because, like an antibiotic, an antivirus is only effective once the virus is already on your system somehow hopefully held in quarantine by the browser or email client and not actively malignant in the processor. There s quite a list at http://en.wikipedia.org/wiki/Comparison_of_antivirus_software#Microsoft_Windows but I expect most of the purchase-free proprietary ones (labelled as Free or Freemium but you usually pay by watching adverts) will try to sell you upgrades, as that s how their production is funded. If you don t mind doing such things, you can disable the ads in at least one of them The only very free ones I found were Immunet (also funded by upgrades not sure if it s actually Free and Open Source Software) and ClamWin (donation-funded) which both use the same scanning engine. If I had to use Microsoft Windows, I think I d probably use and donate to ClamWin, install the (altruism-funded I think) Clam Sentinel alongside it and be rather cautious about what I downloaded or used online. I m a bit worried that it doesn t do great in reviews, though. What do/would you do? I don t really know about paying for security. The only paid product I ve really seen has been Norton and that seemed no better than the ad-funded ones, still getting in the way and always trying to sell upgrades. It also irks me that there s this huge market just to fix fundamental defects in Microsoft s product. There s a Microsoft Security Essentials add-on listed on Wikipedia, but it does fairly badly in this PC Magazine review and do any of them do intrusion detection? And finally, if you do decide to download something new, I strongly suggest getting it from a trusted source and/or triple-checking the link with wikipedia, a magazine review like CNET and a search engine. Don t just trust a search engine, because fake antivirus software is a big way of getting viruses and worse onto computers: there s even one calling itself Microsoft Security Essentials 2011 !

19 June 2013

Matthew Garrett: Mir, the Canonical CLA and skewing the playing field

Mir is Canonical's equivalent to Wayland - a display server, responsible for getting application pixmaps onto a screen. It's intended to scale from mobile devices to the desktop, and as such is expected to turn up in Ubuntu Phone before too long[1]. There's already plenty of discussion about whether the technical differences between Wayland and Mir are sufficient to justify Canonical going their own way, so I'm not planning on talking about that.

Like many Canonical-led projects, Mir is under GPLv3 - a strong copyleft license. There's a couple of aspects of GPLv3 that are intended to protect users from being unable to make use of the rights that the license grants them. The first is that if GPLv3 code is shipped as part of a user product, it must be possible for the user to replace that GPLv3 code. That's a problem if your device is intended to be locked down enough that it can only run vendor code. The second is that it grants an explicit patent license to downstream recipients, permitting them to make use of those patents in derivative works.

One of the consequences of these obligations is that companies whose business models depend on either selling locked-down devices or licensing patents tend to be fairly reluctant to ship GPLv3 software. In effect, this is GPLv3 acting entirely as intended - unless you're willing to guarantee that a user can exercise the freedoms defined by the free software definition, you don't get to ship GPLv3 material. Some companies have decided that shipping GPLv3 code would be more expensive than either improving existing code under a more liberal license or writing new code from scratch. Android's a pretty great example of this - it contains no GPLv3 code, and even GPLv2 code (outside the kernel) is kept to a minimum.

Which, given Canonical's focus on pushing Ubuntu into GPLv3-hostile markets, makes the choice of GPLv3 an odd one. This isn't a problem as long as they're the sole copyright holder, because the copyright holder is obviously free to ship their code under as many licenses as they want. But Canonical still aim to foster community involvement, and ideally that includes accepting external contributions to their code. If Canonical simply accepted those contributions under GPLv3 then they'd no longer have the right to relicense the entire codebase, so any contributions are only accepted if the contributor has signed a Contributor License Agreement.

Canonical's CLA is pretty simple. In essence, it grants Canonical the right to use, modify and distribute your code, and it grants Canonical a patent license under any patents you own that may cover the code in question. But, most importantly, it grants Canonical the right to relicense your contribution under their choice of license. This means that, despite not being the sole copyright holder, Canonical are free to relicense your code under a proprietary license.

Given Canonical's market goals, this makes sense. They can relicense Mir (and any other GPLv3 projects they own) under licenses that keep their hardware partners happy, and they can ship in the phone market. Everyone's a winner.

Except, if Canonical want to ship proprietary versions, why not just license Mir under a license that permits that in the first place? This is where the asymmetry comes in. The Android userland is released under a permissive license that allows anyone to take Google's code, modify it as they wish and ship it on whatever hardware they want. I could legally start a company that provided customised versions of Android to phone vendors without them having any GPLv3 concerns. I won't be able to do that with Ubuntu Phone.

I'm a fan of GPLv3. I think the provisions it contains to support user freedom are important. I hate the growing trend of using free software to build devices that are, effectively, impossible for the end user to modify. If Canonical were releasing software under GPLv3 because of a commitment to free software then that would be an amazing thing. But it's pretty much impossible to square the CLA's requirement that contributors grant Canonical the right to ship under a proprietary license with a commitment to free software. Instead you end up with a situation that looks awfully like Canonical wanting to squash competition by making it impossible for anyone else to sell modified versions of Canonical's software in the same market.

Canonical aren't doing anything illegal or immoral here. They're free to run their projects in any way they choose. But retaining the right to produce proprietary versions of external contributions without granting equivalent reciprocal rights isn't consistent with caring about free software or contributing to the wider Linux community, especially if it means you get to exclude those external contributors from the market you're selling their code into.

(Edit to add: a friend in the contracting industry points out that it also prevents vendors who won't ship GPLv3 from using external contractors to work on Mir - they have to go to Canonical, because only Canonical can relicense contributions under a proprietary license.)

[1] Right now Ubuntu Phone is using Surfaceflinger, the Android display server, but that's apparently just an interim solution.

comment count unavailable comments

DebConf team: DebConf13 DebCamp confirmation and opening of the reconfirmation period (Posted by Didier Raboud)

Going to DebConf13!
DebConf13 DebCamp confirmed! The DebConf Team is very pleased to announce that DebCamp will be open for almost a full week in August. DebCamp will start on Tuesday 6 August 2013 at the main conference venue, Le Camp, and will be immediately followed by DebConf13, starting Sunday 11th August. Opening of the reconfirmation period As the DebConf Registration Team have announced on debconf-announce (to which you should really be subscribed), now is the time to reconfirm your attendance to DebConf13 (and actually register to DebCamp). This is really important for final calculations of food, room and costs, please read the announcement for all the important details. The reconfirmation period is open until the 30 June 2013. We look forward to seeing everyone in August in Vaumarcus. The DebConf team

Daniel Pocock: Australia's war on brains (and immigration)

Some weeks ago, our Prime Minister was slashdotted when she suspended her usual racist attacks on refugees to attack foreign IT workers and the companies that employ them with absurd accusations of "rorts" and "stealing" jobs. Today, she's introduced new laws in the parliament aiming to further bastardization of intelligent, skilled and educated workers and anybody who associates with them, including Australian employers. Unwarranted attention on a minority of IT workers This is not just some random bill before the parliament. There are just two weeks left before the parliament concludes and an election campaign begins. It is clear that what we are seeing now is the Real Julia coming though, choosing to make the small minority of foreign workers in our country at the center of people's thoughts as they go to vote in September. A debt of gratitude to foreign workers These verses from our national anthem, Advance Australia Fair say a lot about how Australia became what it is today:
For those who've come across the seas

We've boundless plains to share;

With courage let us all combine

To Advance Australia Fair.
The last 200 years of Australia's history has been a story of immigration. It is not something to be afraid of: our forefathers celebrated it. Foreign workers: why Australia needs you Australia has had some appalling flops in IT and engineering: There is no doubt in my mind that additional foreign workers would have made a positive contribution to all of these problems or can do so in the future. The sun never sets on IT Every day, I collaborate with dozens of IT specialists all over the world through the virtual workplace that is the Internet, particularly in the free and open source software community. Many of these people, I've never even met and in most cases I don't even know where they are, where they were born or what is the colour of their skin. Those details wouldn't make any difference to the way that we work in IT today. How many IT managers have time to waste dealing with more real world bureaucracy when they've experienced online, global productivity? How many IT workers feel demotivated by having to explain trivial details about their personal life to a Government bureaucrat who doesn't understand their skills and just looks at their colour? If you think about it, any immigration officer who really understands IT wouldn't be an immigration officer. They would be working in IT themselves. Immigration officers, who don't understand IT, are now going to be further empowered to bully companies away from employing some talented workers on the basis of race or nationality. Hiring managers will be intimidated into these prejudiced and biased decisions by delays, processing fees and invasive demands for sensitive documents about business planning and recruitment strategies. Australia's immigration system already has a horrendous reputation. Any visa application seems to take more than a year: no small company can keep a job position vacant that long. Families can't plan their children's schooling. Other life events come and go. There are exhorbitant fees, 1000% higher than in other western countries. Fewer and fewer self-respecting skilled workers are willing to put their spouses and children through the degrading medical examinations. Judging the impact of poor immigration policy While the economic impact of this immigration mess on industry is hard to quantify with an exact figure, we can take some insight from the education system. As the visa system has been hijacked by racists over the last 10 years, there has been a dramatic fall in participation (and revenues) from foreign students. In one year, enrolments (and revenue) fell 30%. This is not just bad for the balance sheets of the universities, it also means that in a future where commerce is global, Australians are more and more isolated and inexperienced culturally. IT workers and their employers have plenty of choices: Australia's close neighbor, Singapore, is one of them. Visas are granted in 2 weeks, no degrading medical exam required, low taxes and tropical sunshine all year round. Many companies that find it impractical to deal with Australia's bureaucracy end up moving their best Australian workers to places like Singapore to be part of a global team. This can't be good for the workforce that is left behind without jobs. The training delusion Government officials continue to rant and rave about companies failing to train Australian workers. The new laws supposedly force companies to "fix" this problem and train Australian workers. This, too, is a delusion: employers are not to blame. Some of the best Australian workers are already long gone to places like Singapore, London and the US. With talented foreign workers denied the opportunity to come and fill the void, there is less opportunity for skills to be acquired by more junior workers in Australian workplaces. It is also extremely difficult for more junior workers to get a foot in the door in the international job market and the primary reason for this is the Australian Government's failure to fund university programs beyond a bachelor's degree. Compare this to Europe and the US where all competent graduates are funded through to a Masters or PhD program. The bottom line is that more junior workers are denied the opportunity to get the best training either at home or abroad and in both cases it's not the foreign workers that can be blamed: it's the Government's own fault. Why do we need skilled foreign workers when Australians can win Nobel Prizes? The Australian press recently went into a frenzy when an Australian won the Nobel Prize for physics. There was a catch though: he's a migrant from the United States (just don't tell the Prime Minister). Dr. Schmidt migrated to Australia 20 years ago when the immigration system was not the same as today. Today, future Nobel Prize winners are being shown a brick wall - maybe we even have one of them rotting away in our death camps or left in the sea for sharks to eat. The BBC recently revealed that Britain's successor to Stephen Hawking may be a young girl who migrated from India - it is chilling to imagine where a child like this may be hidden away under Australia's immigration system. Bureaucracy leads to fraud and exploitation It's been clearly demonstrated that wherever you have elaborate, artificial systems of bureaucracy it leads to inefficiency, it suppresses innovation and in the worst cases it enables fraud and exploitation. The typical examples usually involve police in some third-world African nation setting up road blocks and collecting fees from travellers who want to pass the queues. This type of opportunism has also been found in Australia's immigration system, with one Federal politician already directly implicated and jailed for his role in a visa racket. Gillard's own bullet man A Queensland pensioner made international headlines recently when he was caught sending bullets in the mail to the Prime Minister. His demands were clear: stop immigration. While most world leaders refuse to let nutcases like this dictate their actions, Gillard appears to have been transcribing his racist letters directly into these newest immigration laws. It is a sad reality that Australian politics regularly seeks to appeal to the worst instincts in people like bullet man. The ultimate political failure When politicians stoop to the level of demonizing immigrants it is usually a clue that the politicians themselves are past their use-by dates and out of fresh policy ideas. When former French president Sarkozy tried to play the racist card in his campaign for re-election, it bit him in the bum and he was swept from power by the socialists. As always in politics, there is an element of hypocrisy at work: neither our Head of Government (the Prime Minister) nor our Head of State (the Queen) was born in Australia. Gillard was born in Wales and migrated to Australia as a child. If Australians don't vote for her in September, will she be given 28 days notice to pack her bags and go back home'? From the frying pan and into the fire Tony Abbott, Australia's would-be Prime Minister contemplating his colleague's statements on bestiality The scariest thing is that if Australians see through this racist charade and refuse to vote for it, we could end up with something equally obnoxious: the other major political party is now gaining worldwide attention for their campaign linking gay marriage and homosexuals to bestiality. Which prejudice is the lesser evil: racism or homophobia?

Julien Viard de Galbert: Building a custom debian CD

The French version is available on linuxembedded.fr : Cr e un CD d installation d une debian sp cialis e The goal is to build a debian install CD suitable for the distribution of a complete system including the operating system and applications. Debian already has a tool for that purpose: simple-cdd. Simple-cdd is a set of scripts wrapping debian-cd which is the tool used to build official CDs. In our case, we will include some non-free packages (firmwares for instance) and application specific packages in the system. Using simple-cdd Simple-cdd gets some of its configuration from your host computer, so it s recommended to work on a machine similar to your target (in particular the machine should use the same architecture: i386 or amd64). All the following actions should be done in a working directory that will contain the downloads and configurations for our system. We will start by creting a configuration file my-cdd.conf stating the mirror and components to use:
debian_mirror="http://ftp2.fr.debian.org/debian/"
mirror_components="main contrib non-free"
Then we will launch the build-simple-cdd tool once. This will build a local mirror of the needed packages that we will reused later. So be patient this can be a bit long.
build-simple-cdd --conf ./my-cdd.conf
Once done, you should get a CD iso in the images subfolder. Customizing the image To customize our image we will create a profile my-profile that will define which package to install and which extra package to include on the CD (dev packages for instance). Create a profiles folder:
mkdir profiles
Selecting the packages If you ve got an already installed machine (the one you used to test your application) you can ask dpkg to list the installed packages from that machine and use that as a basis.
dpkg --get-selections >package-list
In this file only the names of packages in the install state are needed:
grep -e '\<install$' package-list   awk ' print $1 ;' >profiles/my-profile.downloads
The file my-profile.downloads is a list of packages to include on the CD-ROM. The list of program to install is my-profile.packages . The tool will resolve dependencies, so we don t need to track down all dependencies add add them to these files. Specific packages We want to add some custom packages that we built ourselves to the CD, those packages not present in the debian archive. Let s create a local_pkg folder and fill it with all of our packages. We now can update the configuration to use all those files and make the profile my-profile the default profile.
debian_mirror="http://ftp2.fr.debian.org/debian/"
mirror_components="main contrib non-free"
simple_cdd_dir=$(pwd)
profiles="my-profile"
auto_profiles="my-profile"
local_packages="$simple_cdd_dir/local_pkg/"
If you now run the tool again, you ll get a full custom install CD!
build-simple-cdd --conf ./my-cdd.conf
However you might have missed some specific packages or their dependencies, in that case they will be missing on the CD. To find this out, check the messages for a line like:
WARNING: missing optional packages from profile my-profile: libevent-2.0-5 [...]
In that case libevent from squeeze-backports was missing. Simple-cdd is not able to download packages from backports, so I just added the packages to local_pkg . Customizing the installer Simple-cdd automatically uses a default profile. This profile might not suite your needs. To override it you just have to place modified file in the profiles folder. To do so, first copy the desired file from /usr/share/simple-cdd/profiles/ and update it. For instance the default.pressed file:
cp /usr/share/simple-cdd/profiles/default.pressed profiles
Then update the wanted options in this file. For example the partitioning options. Also as we enabled contrib and non-free you should uncomment the following lines:
d-i apt-setup/non-free  boolean true
d-i apt-setup/contrib boolean true
Adding extra files to the CD Finally we want to add the documentation, sources and other files to the CD. Simple-cdd can add files to the simple-cdd folder but there is no way to add a complete directory structure. However, as we can override the profiles, we can also override some scripts, especially tools/build/debian-cd that builds the CD.
mkdir -p tools/build
cp /usr/share/simple-cdd/tools/build/debian-cd tools/build
Then add the following lines after the $extras_base_dir cleanup:
if [ -d "$cd_extras" ]; then
  mkdir -p "$extras_base_dir"
  cp -a "$cd_extras/." "$extras_base_dir"
fi
Finally let s add the cd_extras variable to our configuration file :
cd_extras="$simple_cdd_dir/local_extras/"
Be careful to not create files conflicting with the CD content, nothing would warn you about that! Conclusion We now have a way of building a debian install CD including any files we might want to deliver with our system: sources and documentation for example.

18 June 2013

Pietro Abate: Bootstrapping Software Distributions

The paper Bootstrapping Software Distributions , co-authored with Johannes Schauer has been accepted for publication in the proceedings of CBSE 2013, Vancouver, Canada, June 17-21, 2013.

AbstractNew hardware architectures and custom co-processor extensions are introduced to the market on a regular basis. While it is relatively easy to port a proprietary software stack to a new platform, FOSS distributions face major challenges. Bootstrapping distributions proved to be a yearlong manual process in the past due to a large amount of dependency cycles which had to be broken by hand. In this paper we propose an heuristic-based algorithm to remove build dependency cycles and to create a build order for automatically bootstrapping a binary based software distribution on a new platform.

Tollef Fog Heen: An otter, please (or, a better notification system)

Recently, there's been discussions on IRC and the debian-devel mailing list about how to notify users, typically from a cron script or a system daemon needing to tell the user their hard drive is about to expire. The current way is generally "send email to root" and for some bits "pop up a notification bubble, hoping the user will see it". Emailing me means I get far too many notifications. They're often not actionable (apt-get update failed two days ago) and they're not aggregated. I think we need a system that at its core has level and edge triggers and some way of doing flap detection. Level interrupts means "tell me if a disk is full right now". Edge means "tell me if the checksums have changed, even if they now look ok". Flap detection means "tell me if the nightly apt-get update fails more often than once a week". It would be useful if it could extrapolate some notifications too, so it could tell me "your disk is going to be full in $period unless you add more space". The system needs to be able to take in input in a variety of formats: syslog, unstructured output from cron scripts (including their exit codes), snmp, nagios notifications, sockets and fifos and so on. Based on those inputs and any correlations it can pull out of it, it should try to reason about what's happening on the system. If the conclusion there is "something is broken", it should see if it's something that it can reasonably fix by itself. If so, fix it and record it (so it can be used for notification if appropriate: I want to be told if you restart apache every two minutes). If it can't fix it, notify the admin. It should also group similar messages so a single important message doesn't drown in a million unimportant ones. Ideally, this should be cross-host aggregation. The notifications should be possible to escalate if they're not handled within some time period. I'm not aware of such a tool. Maybe one could be rigged together by careful application of logstash, nagios, munin/ganglia/something and sentry. If anybody knows of such a tool, let me know, or if you're working on one, also please let me know.

Next.

Previous.