I ve decided that I d rather like to hack an entire working (ok, not working", but programmable) computer (except for ROM and perhaps I/O) from scratch (just wires, perfboard, transistors (and friends) and solder).
It sounds insane, but a simple ALU seems easy enough. Anyone do this before? Anyone have tips? Schematics of similar projects?
As of wheezy, Debian no longer offers for download any prebuilt hdd (formerly known as usb-hdd) live images, which, in squeeze, were necessary if you wanted to use persistence. The good news is, since the new wheezy iso-hybrid images use xorisso, you can copy an image to a USB key and repartition it to add a persistence partition. Unfortunately, the filesystem itself is read-only, so you have to manually append the persistence boot parameter each time you boot the image. If that doesn t bother you, you can skip this whole tutorial and just follow the instructions in live-manual for adding a persistence partition and you re all set.
This tutorial explains how to extract the contents of a wheezy iso-hybrid live image to a USB key and modify it to boot with this parameter permanently enabled. I am writing it solely as a workaround and do not endorse this as a means to customize Debian Live images in general. I would love to help make a more convenient, supported way to accomplish the same thing for a future release. Meanwhile, the supported and recommended way to make any change to a Debian Live image is to build it from scratch using live-build, as described in live-manual. A simplified Debian Live Images Autobuilder web service is available that works well for many common customizations, if you prefer. Please use one of these methods if you want to do this, or other customizations in the supported way. Otherwise, read on.
Prerequisites
lrwxrwxrwx 1 root root 9 Jun 22 11:58 usb-SanDisk_Cruzer_Contour_0000184CA87406BC-0:0 -> ../../sdbAs you can see, my SanDisk Cruzer Countour is /dev/sdb. For the rest of the tutorial, I ll be referring to this device as /dev/sdX to avoid unfortunate cut-and-paste disasters. When you follow along, make sure you substitute the correct device for your USB key on your system. Initialize the USB key Start by plugging in the key and unmounting any automounted partitions to ensure nothing is accessing the device, e.g.
$ umount /dev/sdX1Now make changes to the partition table. Use parted because it may be run as an ordinary user and has some advanced capabilities beyond more basic tools like fdisk.
$ /sbin/parted WARNING: You are not superuser. Watch out for permissions. GNU Parted 2.3 Using /dev/sdX Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) print devices /dev/sdX (8221MB) (parted) select New device? [/dev/sdX]? Using /dev/sdX (parted)Note that when run as an ordinary user, if you only have one removable drive (which may include mp3 players, cameras, e-readers) plugged in, parted immediately selects that device. But just to be sure, use print devices to list the devices and select to if you need to select a different one, as shown above. Repartition the USB key Once you ve verified you have the correct device, start repartitioning. Write a new partition table with mklabel msdos to wipe out all of the old partitions on it. You may skip this step and adapt the remaining instructions if there are partitions you need to keep. This is your last chance to bail if the device is wrong, so make sure it is the correct one before proceeding.
(parted) mklabel msdos Warning: The existing disk label on /dev/sdX will be destroyed and all data on this disk will be lost. Do you want to continue? Yes/No? y Error: Partition(s) 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16 on /dev/sdX have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use. As a result, the old partition(s) will remain in use. You should reboot now before making further changes. Ignore/Cancel? i (parted)The error is normal, and appears when you run parted as an ordinary user, so it is OK each time it appears to press i to ignore it. For the rest of this tutorial I will omit these errors from the output for brevity. Make three partitions, a fat32 partition for the image itself that is 5% larger than the size of the ISO, a 1G ext4 partition for persistence, and an extra fat32 partition for the rest of the space. Put the extra partition first so that other OSes you might use the key with will be able to use it. Use relative positioning from the end of the device to make it easy to put all of the rest of the space in the first partition. Finally, flag the live image partition bootable.
(parted) mkpart primary fat32 1 -1897M (parted) mkpart primary ext4 -1897M -897M (parted) mkpart primary fat32 -897M -0 (parted) set 3 boot on (parted) quit Information: You may need to update /etc/fstab.Unplug the USB key and plug it in again so that the kernel will re-read the updated partition table. If there were old filesystems on the key that were automounted, unmount them again. Now we re ready to make the new filesystems.
$ umount /dev/sdX1 $ /sbin/mkdosfs -nEXTRA /dev/sdX1 mkdosfs 3.0.16 (01 Mar 2013) $ /sbin/mkfs.ext4 -q -Lpersistence /dev/sdX2 $ /sbin/mkdosfs -nLXDE /dev/sdX3 mkdosfs 3.0.16 (01 Mar 2013)Make the USB key bootable Use install-mbr to install an MBR on the key and syslinux to install the bootloader to boot into your live image partition.
$ /sbin/install-mbr /dev/sdX $ syslinux -i /dev/sdX3Mount the partitions Unplug the key and plug it in again so the new partitions will be automounted. Observe where the partitions are mounted with df.
$ df ... /dev/sdX1 6204568 4 6204564 1% /media/EXTRA /dev/sdX3 831888 4 831884 1% /media/LXDE /dev/sdX2 944120 1204 894124 1% /media/persistenceExtract the ISO contents to the key To continue preparing this key as an unprivileged user, use p7zip to extract the ISO. You may alternatively mount the ISO loopback and extract it that way, but you must be root to do that.
$ cd /media/LXDE $ 7z x ~/debian-live-7.0.0-amd64-lxde-desktop.iso ... Extracting live/filesystem.packages-remove ...When live/filesystem.squashfs is extracted (right after live/filesystem.packages-remove) it will take the longest time of all of the files to extract, as it contains the whole live filesystem. Be patient and eventually it and all other files on the image will finish extracting.
... Everything is Ok Folders: 245 Files: 370 Size: 892183367 Compressed: 895483904Modify the bootloader configuration files The syslinux bootloader configuration directory and files within it are named isolinux when installed on an ISO. You need to rename the directory and two files changing isolinux to syslinux so the bootloader will find them on your fat32 live image partition.
$ mv isolinux syslinux $ mv syslinux/isolinux.cfg syslinux/syslinux.cfg $ mv syslinux/isolinux.bin syslinux/syslinux.binEnable full persistence Next, append persistence to the live boot parameters and turn on full persistence by putting / union in a persistence.conf file in the persistence partition.
$ sed -i 's/\(append boot=.*\)$/\1 persistence/' syslinux/live.cfg $ cd /media/persistence $ echo / union > persistence.confReboot into the live system Now your USB key is ready to boot with full persistence enabled. Have fun!
teythoon@europa /tmp % export GOPATH= pwd teythoon@europa /tmp % go get bitbucket.org/teythoon/hurdtest teythoon@europa /tmp % bin/hurdtest Usage: hurdtest [options] <command> [<arguments...>]Currently it has the following subcommands:
teythoon@europa ~/repos/hurd/hurd-tests % bin/hurdtest update 2013/06/22 19:59:45 starting kvm [-drives file=image.qcow2,cache=writeback,snapshot=on -serial unix:/tmp/285813369/serial -monitor unix:/tmp/285813369/monitor -nographic -m 512] 2013/06/22 19:59:45 Waiting for grub... 2013/06/22 19:59:46 Waiting for login prompt... 2013/06/22 20:00:05 Updating system 2013/06/22 20:02:52 Get: 1 http://www.local unstable InRelease [8279 B] Get: 2 http://http.debian.net sid InRelease [205 kB] Get: 3 http://ftp.debian-ports.org unreleased InRelease [23.9 kB] [...] Setting up perl (5.14.2-21) ... Reading package lists... Building dependency tree... Reading state information... Reading extended state information... Initializing package states... Reading task descriptions... 2013/06/22 20:02:52 Update successful. Snapshotting system... 2013/06/22 20:02:53 Rebooting system... 2013/06/22 20:02:54 Waiting for grub... 2013/06/22 20:02:59 Waiting for login prompt... 2013/06/22 20:03:16 Running main.testTrue... 2013/06/22 20:03:16 Running main.testFalse... 2013/06/22 20:03:17 Running main.testClean... [...] 2013/06/22 20:03:18 Running main.testNetwork... 2013/06/22 20:03:18 Running main.testResolver... 2013/06/22 20:03:20 Running main.testAptitudeInstall... 2013/06/22 20:03:46 Test successful. Reverting system... 2013/06/22 20:03:48 Halting system... 2013/06/22 20:03:53 Committing changes... 2013/06/22 20:03:57 Creating snapshot... 2013/06/22 20:04:04 All done. bin/hurdtest update 157.31s user 69.71s system 87% cpu 4:18.94 totalThe overlay command implements my work flow. I can edit scripts or drop hacked up executables onto my VM and see the results without any manual intervention. There's the -interactive flag that does this in a loop, avoiding the VM startup time using qemu snapshots. With this I can apply an overlay and see the results of the test suite in about a minute on my system. From early experiments I knew that simply running /etc/rcS.d/S* prevented the system from booting correctly, so I figured my first task should be to figure out what went wrong there. Using my tool (and developing it in that process) and many iterations later I came up with a surprisingly small diff of scripts in the initscripts and utilities shipped with the hurd package. For your convenience here is a tarball that you can extract to the root of your Debian/Hurd installation to reproduce my results (of course I invite you to use my tool for that purpose):
% bin/hurdtest overlay test 2013/06/22 21:35:31 starting kvm [-drive file=image.qcow2,cache=writeback,snapshot=on -serial unix:/tmp/345188134/serial -monitor unix:/tmp/345188134/monitor -nographic -m 512] 2013/06/22 21:35:31 Waiting for grub... 2013/06/22 21:35:31 Waiting for login prompt... 2013/06/22 21:35:48 Creating tarball... ./ ./etc/ ./etc/hurd/ ./etc/hurd/rc ./etc/init.d/ ./etc/init.d/checkroot.sh ./etc/init.d/mountall.sh ./runme ./lib/ ./lib/init/ ./lib/init/mount-functions.sh ./sbin/ ./sbin/swapon ./bin/ ./bin/mount 2013/06/22 21:35:48 Extracting overlay... 2013/06/22 21:35:49 Rebooting system... 2013/06/22 21:35:49 Waiting for grub... 2013/06/22 21:35:55 Writing log to /tmp/bootlog-802006344 2013/06/22 21:35:55 Waiting for login prompt... 2013/06/22 21:36:20 Booting successful. 2013/06/22 21:36:20 Running main.testTrue... 2013/06/22 21:36:21 Running main.testFalse... 2013/06/22 21:36:21 Running main.testClean... 2013/06/22 21:36:21 Running anonymous function... 2013/06/22 21:36:21 Running anonymous function... 2013/06/22 21:36:21 Running anonymous function... 2013/06/22 21:36:21 Running anonymous function... 2013/06/22 21:36:21 Running anonymous function... 2013/06/22 21:36:22 Running anonymous function... 2013/06/22 21:36:22 Running main.testUname... 2013/06/22 21:36:22 Running main.testNetwork... 2013/06/22 21:36:22 Running main.testResolver... 2013/06/22 21:36:23 Running main.testAptitudeInstall... bin/hurdtest overlay ../hurd-tests-old/test 44.20s user 20.01s system 82% cpu 1:17.94 totalAnd here is the boot log:
Loading GNU Mach ...
Loading the Hurd ...
GNU Mach 1.3.99-486
AT386 boot: physical memory map from 0x0 to 0x9f400
AT386 boot: physical memory map from 0x100000 to 0x1fffe000
AT386 boot: physical memory from 0x0 to 0x1fffe000
Enabling FXSR
pcibios_init : BIOS32 Service Directory structure at 0xfcff0
pcibios_init : BIOS32 Service Directory entry at 0xfc7ba
pcibios_init : PCI BIOS revision 2.10 entry at 0xfc78c
Probing PCI hardware.
ide: Intel 82371 PIIX3 (dual FIFO) DMA Bus Mastering IDE
Controller on PCI bus 0 function 9
ide: BM-DMA feature is not enabled (BIOS), enabling
ide0: BM-DMA at 0xc100-0xc107
ide1: BM-DMA at 0xc108-0xc10f
hd0: got CHS=762/128/63 CTL=c8 from BIOS
hd0: QEMU HARDDISK, 3001MB w/256kB Cache, CHS=762/128/63, DMA
hd2: QEMU DVD-ROM, ATAPI CDROM drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
Floppy drive(s): fd0 is 1.44M
FDC 0 is a S82078B
probing scsi 0/22: advansys probing scsi 1/22: BusLogic probing scsi 2/22: UltraStor 14F/34F rev. 4.33.00 probing scsi 3/22: UltraStor 14F/24F/34F probing scsi 4/22: Adaptec 152x SCSI driver; $Revision: 1.1 $ probing scsi 5/22: Adaptec 1542 Invalid address for shpnt with 1542.
Invalid address for shpnt with 1542.
probing scsi 6/22: Adaptec 174x (EISA) probing scsi 7/22: probing scsi 8/22: Always IN2000 probing scsi 9/22: probing scsi 10/22: probing scsi 11/22: probing scsi 12/22: Trantor T128/T128F/T228 probing scsi 13/22: DTC 3180/3280 probing scsi 14/22: Tekram DC390(T) V1.11 Feb-05-1997 probing scsi 15/22: probing scsi 16/22: ncr53c8xx - revision 2.5f.1 probing scsi 17/22: EATA (Extended Attachment) PIO driver probing scsi 18/22: Western Digital WD-7000 Failed initialization of WD-7000 SCSI card!
probing scsi 19/22: EATA/DMA 2.0x rev. 4.33.00 probing eata on 1f0EATA0: address 0x1f0 in use, skipping probe.
probing eata on 1c88probing eata on 2c88probing eata on 3c88probing eata on 4c88probing eata on 5c88probing eata on 6c88probing eata on 7c88probing eata on 8c88probing eata on 9c88probing eata on ac88probing eata on bc88probing eata on cc88probing eata on dc88probing eata on ec88probing eata on fc88probing eata on 170EATA0: address 0x170 in use, skipping probe.
probing eata on 230probing eata on 330probing scsi 20/22: AM53C974 probing scsi 21/22: Iomega parport ZIP drive ppa: Version 1.42
ppa: Probing port 03bc
ppa: Probing port 0378
ppa: SPP port present
ppa: PS/2 bidirectional port present
ppa: Probing port 0278
done
scsi : 0 hosts.
scsi : detected total.
Partition check (DOS partitions):
hd0: hd0s1 hd0s2 < hd0s5 >
lpr0: at atbus0, port = 378, spl = 6, pic = 7.
module 0: ext2fs --readonly --multiboot-command-line=$ kernel-command-line --host-priv-port=$ host-port --device-master-port=$ device-port --exec-server-task=$ exec-task -T typed $ root $(task-create) $(task-resume) module 1: exec /hurd/exec $(exec-task=task-create) 2 multiboot modules task loaded: ext2fs --readonly --multiboot-command-line=root=device:hd0s1 console=com0 --host-priv-port=1 --device-master-port=2 --exec-server-task=3 -T typed device:hd0s1
task loaded: exec /hurd/exec
start ext2fs: Hurd server bootstrap: ext2fs[device:hd0s1] exec init proc auth
Using makefile-style concurrent boot in runlevel S.
ERROR: could not open /proc/stat: No such file or directory
Activating swap...done.
Checking root file system...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
ext2fs_check_if_mount: Can't check if filesystem is mounted due to missing mtab file while determining whether /dev/hd0s1 is mounted.
/dev/hd0s1: clean, 44328/181056 files, 288266/723200 blocks
done.
Cleaning up temporary files... /tmp.
mount: /run: Unknown device or filesystem
mount: /run/lock: Unknown device or filesystem
mount: /proc: Unknown device or filesystem
mount: /run/shm: Unknown device or filesystem
/etc/rcS.d/S06mtab.sh: 48: /etc/rcS.d/S06mtab.sh: cannot open /proc/mounts: No such file
Activating lvm and md swap...(default pager): Already paging to partition hd0s5!
done.
Checking file systems...fsck from util-linux 2.20.1
hd2 : tray open or drive not ready
hd2 : tray open or drive not ready
end_request: I/O error, dev 02:00, sector 0
done.
Mounting local filesystems...mount: invalid option -- 'O'
Try mount --help' or mount --usage' for more information.
failed.
Activating swapfile swap...(default pager): Already paging to partition hd0s5!
done.
df: Warning: cannot read table of mounted file systems: No such file or directory
Cleaning up temporary files....
Configuring network interfaces...inetutils-ifconfig: invalid arguments
ifup: failed to open pid file /run/network/ifup-/dev/eth0.pid: No such file or directory
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
Listening on Socket//dev/eth0
Sending on Socket//dev/eth0
DHCPDISCOVER on /dev/eth0 to 255.255.255.255 port 67 interval 5
DHCPREQUEST on /dev/eth0 to 255.255.255.255 port 67
DHCPOFFER from 10.0.2.2
DHCPACK from 10.0.2.2
can't create /var/lib/dhcp/dhclient./dev/eth0.leases: No such file or directory
bound to 10.0.2.15 -- renewal in 36884 seconds.
done.
Cleaning up temporary files....
Setting up X socket directories... /tmp/.X11-unix /tmp/.ICE-unix.
Using makefile-style concurrent boot in runlevel 2.
Starting enhanced syslogd: rsyslogd.
Starting deferred execution scheduler: atd.
Starting periodic command scheduler: cron.
Starting system message bus: dbusFailed to set socket option"/var/run/dbus/system_bus_socket": Protocol not available.
Starting OpenBSD Secure Shell server: sshd.
GNU 0.3 (debian) (console)
login:
I know it isn't that pretty, but it yields a working Debian/Hurd
system that is more like a Debian/Linux system. For example /dev/shm
and /run ,/* are connected to a tmpfs translator. I think this is
a rather encouraging first result. I added my findings to the list of
open issues.
Another nice thing about hurdtest is that it is not even Hurd
specific (only a tiny bit GNU Mach specific). I'm going to make it
work nicely with Debian/Linux and Debian/kFreeBSD, since this is a
very handy tool for anyone hacking on the Debian boot process (and
with some changes for any OS). If anyone tries it out, I'd love to get
feedback and patches.
The tool requires the guest to be configured to use the first serial
console for grub, the kernel and the userspace. The password for the
user root must be empty. The guest has to have a working network
connection setup at boot time. I will provide a suitable image as soon
as I devise a good way to zero-fill the free space on the virtual
disk. Is there any better way than to fill the disk with a huge file
containing zeros? Without doing that the image is too large even
compressed with xz.
So this was my first week working for Debian. Next week I will focus
on getting patches for the hurd and initscripts package in shape,
I need to add some functionality to the mount utility shipped with
the hurd package. It provides a mount-like experience on the Hurd,
on which binding a translator (e. g. /hurd/ext2fs) to a filesystem
node is an operation that does not involve the kernel (GNU Mach) at
all, and is in fact a operation that any unprivileged user may
perform. That is one of the nice features the Hurd has, it empowers
its users.
If you re a Debian user, you are probably already familiar with some of the awesome icons for IceWeasel (rebranded Mozilla Firefox), IceDove (rebranded Mozilla Thunderbird) and IceApe (rebranded Mozilla SeaMonkey).
I was pretty ambivalent about the decision to rebrand Firefox until I saw some of proposed the IceWeasel icons which in my humble opinion were just too cute, and awesome, to pass up.
Until very recently however, IceOwl (rebranded Mozilla Sundbird) had no such awesome icon. Quite a while ago, I filed bug #658664 in Debian complaining that iceowl does not include awesome icy owl icons. I wrote:
I was extremely disappointed when I installed Iceowl and discovered that it does not ship with an awesome logo or icons showing a picture of an IceOwl. Instead, it seems to be represented by picture of a (boring) paper calendar which is very generic and not awesome at all. IceWeasel, IceDove, and IceApe each include extremely awesome logos/icons that have really cool looking white illustrations of icy weasels, doves, and apes. IceOwl needs a similarly awesome logo to use as its icon. This bug seems particularly egregious because owls actually live in icy climates and come in white versions! For example: https://commons.wikimedia.org/wiki/File:Snowy_Owl_-_Schnee-Eule.jpg While illustrators need to imagine what an ice ape or ice weasel might look like, there is no such need for imagination in the case of an ice owl! As far as I m concerned, this bug should be release critical. Hopefully, someone will upload a patch quickly!Finally, after many months of all of us suffering in silence, Nick Morrott came along and fixed the bug with the creation of this new, incredibly awesome, icy owl logo!
![[Co-operatives Fortnight image]](http://www.news.software.coop/wp-content/blogs.dir/6/files/2013/06/wpid-c14-logo.jpg "Co-operatives Fortnight"){kind=logo}
Cooperatives Fortnight 2013 starts tomorrow. Our co-op is taking part in a few activities this year. Come along and meet us, or find other events on the national website.
This Sunday, I ll be riding 50 miles in support of Leonard Cheshire. Some of it is familiar roads, some of it new. I ll also be riding 12 miles just to get to the start line. If you re a UK resident and would like to guess my time and make a small donation, please give it a try on guess2give.
Next Thursday 27 June, I ll be at Somerset Cooperative Services AGM in Taunton. See their site for further news.
There might be something on Monday 1 July. To be announced later maybe.
On Thursday 4July, mjkaye will be at Building Our Co-ops in Liverpool. See the national event listing for details.
So what will you be doing for co-ops fortnight?
You've set up a backup repository, and you have been backing up to
it every day for a month now: your backup history is getting long
enough to be useful. Can you be happy now?
Welcome to the world of threat modelling. Backups are about
insurance, of mitigating small and large disasters, but disasters
can strike backups as well. When are you so safe you no disaster
will harm you?
There is always a bigger disaster waiting to happen. If you backup
to a USB drive on your work desk, and someone breaks in and steals
both your computer and the USB drive, the backups did you no good.
You fix that by having two USB drives, and you keep one with your
computer and the other in a bank vault. That's pretty safe, unless
there's an earth quake that destroys both your home and the bank.
You fix that by renting online storage space from another country.
That's quite good, except there's a bug in the operating system
that you use, which happens to be the same operating system the
storage provider uses, and hackers happen to break into both your
and their systems, wiping all files.
You fix that by hiring a 3D printer that prints slabs of concrete on
which your data is encoded using QR codes. You're safe until there's a
meteorite hits Earth and destroys the entire civilisation.
You fix that by sending out satellites with copies of your data,
into stable orbits around all nine planets (Pluto is too a planet!)
in the solar system. Your data is safe, even though you yourself
are dead from the meteorite, until the Sun goes supernova and
destroys everything in the system.
There is always a bigger disaster. You have to decide which
ones are likely enough that you want to consider them, and also
decide what the acceptable costs are for protecting against them.
A short list of scenarios for thinking about threats:
They say that a certain billionaire is so rich that he would not spend 5 seconds bending over to pick up a $100 banknote on the ground because he makes $500 in 5 seconds anyway. A man from Ethiopia may well run a marathon for the same $100. This little anecdote underlines the point that no two people have exactly the same priorities and it is hard to provide a one-size-fits-all solution. A good technical solution needs to let the user have some control over their own priorities as that will go a lot further in respecting their preferences, their objectives and their environment.
A practical example: some users may not want to be disturbed by a VoIP call while watching a movie with their PC. Other users may be quite happy to receive these interruptions. Some users may want to only allow this interruption from a subset of possible callers.
Building blocks
In the original blog on this subject I suggested that Thunderbird and it's popular plugin, Mozilla Lightning calendar would provide one useful framework for integrating events, alerts and to-do items of various grades of importance. I already use these applications myself, from the re-branded icedove and iceowl-extension packages on Debian. They are also available in Fedora and many other distributions.
Priority with Lightning
An initial look at Lightning reveals a Priority attribute for tasks. It can be set to the values High, Medium or Low. These don't map to the concepts of Urgency and Importance from the Eisenhower method discussed in the original blog.
Potential hacks: there are two ideas that come to mind, with no coding changes:
Finally, after gathering all the events for the user in a mailbox like this, it would be possible for the plugin to be configured to interrupt the user only when some event reaches the user's own urgency/importance threshold. For example:
The Sydney Users of R Forum (SURF)
will be hosting me for a talk on July 10.
The focus will be Rcpp for R and C++ integration,
and the intent is to have this be really applied with lots of motivating examples.
Organizers Louise and Eugene were able to move this to a slightly larger room as the initial capacity of 50 was
filled almost immediately. As of right now, the talk page
shows a few available slots. If you're in the Sydney are in early July, why not register and swing by?
If you're not in the area, the Rcpp Events page lists other upcoming talks too.
This is part 1 in a series on organizational design and growth.
An important lens for thinking about organizations at this stage of growth is alignment. In an organization which is aligned, the efforts of different people and teams all contribute to forward progress in a shared direction. If two teams are pulling in opposite directions they may make little progress despite great effort, and quickly become frustrated. To take an obvious example, if a marketing team is targeting an audience of large enterprises while the product being developed is only suited for small businesses, the end result of both teams doing a good job will be a failure (i.e. unhappy customers).
It s important to note that alignment does not imply sameness. Different teams within an organization can function and behave very differently while still being strongly aligned with each other.
When an organization is small, alignment comes naturally. Everyone has some visibility on what everyone else is doing, and when something doesn t line up, the people involved can talk it over and resolve the issue relatively easily. But as the organization grows, the propensity for misalignment increases, and these situations become much more difficult and time-consuming to resolve. The metaphor of the right hand, which doesn t know what the left hand is doing, seems like something that would only happen in larger companies, but it begins much earlier, especially when the company goes through a period of rapid growth. Critical infrastructure, such as communication tools and patterns, lags behind the accelerating needs of the people involved, creating a surprising distance between teams.
Organizational alignment is a critical part of scaling successfully. With alignment, growth and momentum are assets. Without it, they are liabilities.
Further reading: The Advantage: Why Organizational Health Trumps Everything Else in Business
This is the beginning of a series of articles about the challenges of growing an organization. I m writing them to share some principles that I ve derived from my own experience, as well as many valuable discussions with friends and colleagues, about helping companies grow from being quite small (say, 1-50 employees) to medium-sized (100-500).
There are many different ways to categorize companies by size, and not everyone agrees with me that different organizations tend to face certain similar problems as they grow, based on the number of employees. In any case, hopefully we can all agree that human systems are mind-bogglingly complex entities, and any two organizations will have many important differences such as their culture and market situation which influence their growth and development.
For this reason, I believe there are few if any hard and fast rules, and organizational design patterns can be difficult to translate from one organization to another. One organization s solution can be another s problem. Even when there is a perfect fit, the process of organization change is a feat unto itself, one about which many books have been written.
Even so, I think there is much to be learned by comparing different organizations, and much inspiration to be found in their successes and failures. Two organizations merit specific mention here, as sources of inspiration for me: Canonical, where I worked as Ubuntu CTO from near inception to when it reached nearly 500 people, and Heroku, where I currently serve as VP Engineering as it grows beyond 100 people.
NMS graphs slowly migrating them from using rrdtool graph and using jqplot. While there have been many false-starts and re-works, I now have a working set of graphs, two of which are shown on the page.
The graphs look a lot slicker and I have also simplified the admin screens. I found I kept having to type the same thing in over and over for the rrdgraphs and have narrowed down the type of graphs to approximately 5. They re certainly not bulletproof and need more testing but they re a good start.
The graphs are based upon the ToscaWidgets2 series of widgets that then provide a nice handle for the jqPlot code. My graphs even have some hand-coded Javascript to give nice units on the Y axis.
"The blood, sweat and tears that has gone into the Ultranet and the work teachers put in it's soul destroying. I have to face parents who took me on face value when I said: 'This is the best thing since sliced bread every school is going to be using it in the future."Those are the words of a school principal at Alkira Secondary College in Victoria, Australia. His school is receiving a real-life lesson in the the risks of cloud computing. Ultranet, the state-wide private cloud project for schools is at risk of being cancelled in 10 days due to budget cuts. It is not clear whether the vendor, NEC, is obliged to release data back to the users if the original four year contract is not extended. Commercial, confidential discussions It appears that the possibility of ending the contract was not supposed to be public knowledge. If the finance department had kept their intentions secret then it is possible that users would not even have had the opportunity to try and manually extract some of their data, as they are rushing to do now. Instead of getting on with the business of education itself, teachers and students are having to waste time trying to individually download student projects, photos, feedback and other meta data from the system in an ad-hoc manner. An opportunity for free and open source software This debacle may well be a huge opportunity for the free and open source software community to remind people that the value of retaining control over user data can not be forgotten when evaluating outsourced, private cloud solutions that claim to be "free" or offer low costs to get started. There were apparently 18 schools who initially trialed the program and many more have gradually joined up. This would be the perfect opportunity for advocates of genuinely free software solutions to contact them and let them know they are not alone and that many of us have anticipated these problems for years. While central government administrators tend to be hooked on the idea of administering big contracts with trusted' vendors, these local school administrators and the communities around them are probably more receptive to the free software message than at any other time. List of schools using Ultranet Many of the schools using Ultranet can be found with Google, as it appears to be integrated in their web sites. Click here for a Google search that finds many of the schools and related technical pages, support documents and other material A message to schools everywhere For schools facing this crisis or considering adopting such services in other countries, I would share this very simple advice: if you are offered a solution that doesn't give you complete control to conveniently extract all of your data in bulk whenever you want and use it in your own software on any computer, then it is not a good foundation to build on. The free software community has been actively working to provide truly free solutions for schools and I would strongly encourage schools to look at projects like Skolelinux/Debian-Edu and k12Linux that have been built by educators who also have a deep understanding of software and data freedom issues.
One of our co-op s clients asked me what I use for anti-virus at the moment and tips for what they should use on their Windows system.
Well, flame me now, but I don t actually use any anti-virus at the moment: I rely on system security, firewalling and intrusion detection. The diversity of GNU/Linux software and I use some pretty odd stuff probably helps too. Even if I did want to run antivirus software, most of what s available for GNU is actually aimed at detecting and preventing transmission of Windows viruses. There are few real-world GNU viruses and fewer attack opportunities left open.
Also, I prefer firewalling and fairly paranoid security settings because, like an antibiotic, an antivirus is only effective once the virus is already on your system somehow hopefully held in quarantine by the browser or email client and not actively malignant in the processor.
There s quite a list at http://en.wikipedia.org/wiki/Comparison_of_antivirus_software#Microsoft_Windows but I expect most of the purchase-free proprietary ones (labelled as Free or Freemium but you usually pay by watching adverts) will try to sell you upgrades, as that s how their production is funded. If you don t mind doing such things, you can disable the ads in at least one of them
The only very free ones I found were Immunet (also funded by upgrades not sure if it s actually Free and Open Source Software) and ClamWin (donation-funded) which both use the same scanning engine. If I had to use Microsoft Windows, I think I d probably use and donate to ClamWin, install the (altruism-funded I think) Clam Sentinel alongside it and be rather cautious about what I downloaded or used online. I m a bit worried that it doesn t do great in reviews, though. What do/would you do?
I don t really know about paying for security. The only paid product I ve really seen has been Norton and that seemed no better than the ad-funded ones, still getting in the way and always trying to sell upgrades. It also irks me that there s this huge market just to fix fundamental defects in Microsoft s product. There s a Microsoft Security Essentials add-on listed on Wikipedia, but it does fairly badly in this PC Magazine review and do any of them do intrusion detection?
And finally, if you do decide to download something new, I strongly suggest getting it from a trusted source and/or triple-checking the link with wikipedia, a magazine review like CNET and a search engine. Don t just trust a search engine, because fake antivirus software is a big way of getting viruses and worse onto computers: there s even one calling itself Microsoft Security Essentials 2011 !
Mir is Canonical's equivalent to Wayland - a display server, responsible for getting application pixmaps onto a screen. It's intended to scale from mobile devices to the desktop, and as such is expected to turn up in Ubuntu Phone before too long[1]. There's already plenty of discussion about whether the technical differences between Wayland and Mir are sufficient to justify Canonical going their own way, so I'm not planning on talking about that.
I chose Wickers World for the flight, since they have sites nearby and seemed the most professional. We were lucky enough to fly on the first attempt and had beautiful weather, although there was rain behind us.
Last year, I applied for academic, tenure track, jobs at several communication departments, information schools, and in HCI-focused computer science programs with a tradition of hiring social scientists.
Being on the market as it is called is both scary and time consuming. Like me, many candidates have never been on the market before. Candidates are asked to produce documents in genres e.g., cover letters, research statements, teaching statements, diversity statements that most candidates have never written, read, or even heard of.
Candidates often rely on their supervisors for advice. I did so and my advisors were extremely helpful. The reality, however, is that although candidates advisors may sit on hiring committees, most have not been on candidates side of job market themselves for years or even decades.
The Internet is full of websites, like the academic jobs wiki, Academia StackExchange, and the Chronicle of Higher Education forums for people on the market. Confused and insecure candidates ask questions of the form, Does blank matter? and the answer is usually, Doing/having blank may help/hurt, but it is only one factor of many. The result is that candidates worry about everything. Then they worry about what they should be worrying about, but are not.
The most helpful thing, for me, was to read and synthesize the material submitted by recent successful job market candidates. For example, Michael Bernstein a friend from MIT, now at Stanford published his research and teaching statements on his website and I found both useful as I prepared mine. That said, I was surprised by how little material like this I could find on the web. For example, I could not find any examples of recent job market cover letters from successful candidates in fields close to mine.
So to help fill this gap, I am publishing all of my job market material. I ve posted both the PDFs of the material I submitted as well as the LaTeX templates I used to generate the documents in my packet. My packet included:
In the survival kit of the person suffering from migraine (like yours truly),
a sleep mask is essential.
This is my new one: I made it a couple of days ago, following this
awesome pattern by Bunnytan.
Today, she's introduced new laws in the parliament aiming to further bastardization of intelligent, skilled and educated workers and anybody who associates with them, including Australian employers.
Unwarranted attention on a minority of IT workers
This is not just some random bill before the parliament. There are just two weeks left before the parliament concludes and an election campaign begins. It is clear that what we are seeing now is the Real Julia coming though, choosing to make the small minority of foreign workers in our country at the center of people's thoughts as they go to vote in September.
A debt of gratitude to foreign workers
These verses from our national anthem, Advance Australia Fair say a lot about how Australia became what it is today:
For those who've come across the seasThe last 200 years of Australia's history has been a story of immigration. It is not something to be afraid of: our forefathers celebrated it. Foreign workers: why Australia needs you Australia has had some appalling flops in IT and engineering:
We've boundless plains to share;
With courage let us all combine
To Advance Australia Fair.
A Queensland pensioner made international headlines recently when he was caught sending bullets in the mail to the Prime Minister. His demands were clear: stop immigration.
While most world leaders refuse to let nutcases like this dictate their actions, Gillard appears to have been transcribing his racist letters directly into these newest immigration laws. It is a sad reality that Australian politics regularly seeks to appeal to the worst instincts in people like bullet man.
The ultimate political failure
When politicians stoop to the level of demonizing immigrants it is usually a clue that the politicians themselves are past their use-by dates and out of fresh policy ideas.
When former French president Sarkozy tried to play the racist card in his campaign for re-election, it bit him in the bum and he was swept from power by the socialists.
As always in politics, there is an element of hypocrisy at work: neither our Head of Government (the Prime Minister) nor our Head of State (the Queen) was born in Australia. Gillard was born in Wales and migrated to Australia as a child. If Australians don't vote for her in September, will she be given 28 days notice to pack her bags and go back home'?
From the frying pan and into the fire
The scariest thing is that if Australians see through this racist charade and refuse to vote for it, we could end up with something equally obnoxious: the other major political party is now gaining worldwide attention for their campaign linking gay marriage and homosexuals to bestiality.
Which prejudice is the lesser evil: racism or homophobia?
Next.
