Copyright Digital Slider and the Spinner

A Debian geek s view of NYC

A Debian geek s view of NYC

As an die hard anthropologist, I never thought I would cavort so much with legal types. But given the nature of my project, it is pretty much impossible to avoid. Thankfully, the legal crowd dealing with digital issues, is pretty entertaining, interesting, and fun to listen to. And probably one of my favorite legal thinkers is coming to NYU to give a talk this coming Monday. If you are into this sort of thing and have not heard FVL speak, do make the time to join.

As an die hard anthropologist, I never thought I would cavort so much with legal types. But given the nature of my project, it is pretty much impossible to avoid. Thankfully, the legal crowd dealing with digital issues, is pretty entertaining, interesting, and fun to listen to. And probably one of my favorite legal thinkers is coming to NYU to give a talk this coming Monday. If you are into this sort of thing and have not heard FVL speak, do make the time to join.

Some time ago, I wrote some quick-and-dirty greasemonkey javascripts to parse plain text bodies of mail messages and mark up things such as quotes, so you could collapse/expand them, colour quotes differently depending on depth, etc. As a result of that experience and countless flamewars about top vs. bottom posting, I've often thought that a markup language specifically designed for email was a good idea. top/bottom posting could be a presentation decision of the client; you could GPG-verify quoted mail; conventions like 'Re:' could be internationalized, again on the client side, etc. This morning I discovered Mail Markup Language which is exactly that. Unfortunately, it appears to have some outright bizarre features. The RFC draft describing it is enormous, including a full XSLT description of the markup language (and not in an appendix either); a legal preface claiming that the entire thing was patent pending (and using a subset of MML would be considered patent infringement); it claimes to obsolete 1939, 3501, 5322, which are POP3, IMAP4r1, and Internet Message Format respectively. You shouldn't judge a book by it's cover, but the fact their website displays awfully for me (epiphany 2.22.x) did prepare me somewhat for the zaniness that followed.

Given the fact that patients have been shamelessly and copiously sharing information on the web since, well, it first sprung up, it is nice to see that the medical establishment has gotten on board with the idea of collaboration. Medpedia is firmly an expert project (though it looks like anyone can provide suggestions) and I am not bothered by this limitation. What I do hope to see is a recognition of the controversies that abound in the realm of medicine (Lyme disease, fibromylagia, and a real hot rod issue = autism). There is much about diagnosis and therapy in medicine that exists in the I am not sure (how to diagnose/treat category and I wonder if this will be reflected in this new project. I certainly hope so.

Given the fact that patients have been shamelessly and copiously sharing information on the web since, well, it first sprung up, it is nice to see that the medical establishment has gotten on board with the idea of collaboration. Medpedia is firmly an expert project (though it looks like anyone can provide suggestions) and I am not bothered by this limitation. What I do hope to see is a recognition of the controversies that abound in the realm of medicine (Lyme disease, fibromylagia, and a real hot rod issue = autism). There is much about diagnosis and therapy in medicine that exists in the I am not sure (how to diagnose/treat category and I wonder if this will be reflected in this new project. I certainly hope so.

It's time to drive some buzz this way ;-) Although this post will only be a pointer towards the Spanish post I made on Planeta EDUSOL, for reasons soon to become obvious. In any case, the information I'm posting here is not exactly the same. Can you read Spanish? Please go on to the invitation for the first videoconference for the EDUSOL Seminar. This year, we the organizers of the On-line Encounter of Education and Free Software (EDUSOL) are aiming higher - we are not "just" having a two-week encounter at the end of the year - We are having an all-year-long Seminar, focusing on the collaborative construction of knowledge. People from quite distinct backgrounds will be part of this project, and we aim to drive it towards the publication of a book. We (mainly, Alejandro Miranda and me) have been quite busy bootstrapping this seminar, getting the proposed authors, thinking over the intended communication channels and ways, and setting up the needed infrastructure) and are ready to start the public-facing activities. We will be having monthly videoconferencing sessions, the last Thursday each month, 16:00-18:00 Mexico Central time (currently GMT-6; GMT-5 after the beginning of April). The VC sessions will be also relayed through Ogg streams, and we will have an IRC channel available to offer full interactivity for those who do not have access to a H.323 VC setup. This first session will be moderated by Victor Manuel Mart nez; the speakers will be Alejandro Miranda and myself - The topics we will present are:

• Short project presentation, delineating the list of invited authors and tematic lines we will pursue
• Description of the collaboration scheme we expect to hold, including how everybody (not just invited authors) can participate
• Presentation of one of the topics we will work into in the Seminar: Free Software and the Democratic Construction of the Society

If you have access to videoconferencing facilities, please get in touch with Carlos Cruz, the Videoconferences Coordinator at the Economic Research Institute, as soon as possible for all the needed coordination.

After dropping all (pretense at) regular exercise after leaving high/grammar/secondary school (the german word is Gymnasium) and doing more or less nothing for more than 10 years, I've taken up weight training (after a not very successful attempt at volleyball; I like the sport but the team fell apart) two years ago. Since I actually managed to work out regularly (twice a week) in the first year, and since membership in a gym is not cheap, and since I moved and thus had to find a different solution anyway, I decided to buy a Bowflex Sport home gym. I can't compare it to other home equipment, and it apparently it has been discontinued by Nautilus, but nonetheless, a few thoughts: The conclusion first: Would I buy it again? No. Was it a waste of money? No, it's not that bad. In detail:

• The free-running seat that should allow cardio rowing is not very useful because there's no good foot-rest, and because the range of motion is too limited. This is my main grievance because being able to do cardio was one of the main reasons for chosing this engine over other offers.
• Go for the 310lbs extension at least (I did, luckily.) With leg exercises, 210lbs are just not enough (I'm 70kg/180cm and have been doing weight training two years. Certainly no muscle freak...)
• It's slightly too small. For many exercises, I'd wish for a bigger range of motion. Especially the lat tower could be 10cm higher.
• While the famous power rods haven't shown signs of wearing out after this first year, the resistance is not consistent. The 310lbs is split in rods of 5, 10, 10, 30, 50, 50lbs per side, and I've noticed that 30 + 10 + 10lbs is noticeably lighter than 50lbs. (I haven't measured at which point of the range of motion the nominal resistance applies. Resistance goes from very light at the start of the motion to hard towards the end, so the numbers don't really compare to training with real weights anyway. I haven't had any problem with this, though.)
• The documentation is lacking. Besides quality issues (wrong images, spelling), some of the stuff delivered in the box as well as some of the possible exercises are simply not used/described. (Example: it is possible to do squats, but the manual doesn't describe them.) So don't get this machine if you don't have an idea about what kinds of exercises you want to do.
• Assembling it is quite a project. But the documentation is good enough on the assembly if you have ever managed to assemble a Kinder Surprise toy, and you only do it once anyway, so it's not really an issue.

Closing remarks: I'll probably keep it for quite a while. I've thought about spending much less money and just getting a bench and a few weights, but I really like to have a lat tower. Overall I can do what I want to do with the exception of cardio workouts. The other angle is that there is quite a bit of bad press around about the founder of Nautilus, Arhur Jones. I can't comment on how trustworthy these articles are, and I can't comment on Nautilus' business practices today (Mr. Jones seems to be dead), so take this with a large pinch of salt before making a buying decision based on moral/ethical standards.

I have a number of different email addresses that all wind up in my gmail account. Some of them are personal, some business. Gmail is smart enough to use a particular email address in your reply when someone writes to you with it, but it would be nice if it went a step further and looked at what address you normally use with a given person, and defaulted to that, so that whenever I write person A, it uses the business address, and switches to the personal address when I write new emails to person B. I guess that's one of the problems with relying on software you don't have the source code to.

and the case against bzip2 Compression is with us all the time. I want to talk about general-purpose lossless compression here. There is a lot of agonizing over compression ratios: the size of output for various sizes of input. For some situations, this is of course the single most important factor. For instance, if you re Linus Torvalds putting your code out there for millions of people to download, the benefit of saving even a few percent of file size is well worth the cost of perhaps 50% worse compression performance. He compresses a source tarball once a month maybe, and we are all downloading it thousands of times a day. On the other hand, when you re doing backups, the calculation is different. Your storage media costs money, but so does your CPU. If you have a large photo collection or edit digital video, you may create 50GB of new data in a day. If you use a compression algorithm that s too slow, your backup for one day may not complete before your backup for the next day starts. This is even more significant a problem when you consider enterprises backing up terabytes of data each day. So I want to think of compression both in terms of resulting size and performance. Onward Starting Point I started by looking at the practical compression test, which has some very useful charts. He has charted savings vs. runtime for a number of different compressors, and with the range of different settings for each. If you look at his first chart, you ll notice several interesting things:

• gzip performance flattens at about -5 or -6, right where the manpage tells us it will, and in line with its defaults.
• 7za -2 (the LZMA algorithm used in 7-Zip and p7zip) is both faster and smaller than any possible bzip2 combination. 7za -3 gets much slower.
• bzip2 s performance is more tightly clustered than the others, both in terms of speed and space. bzip2 -3 is about the same speed as -1, but gains some space.

All this was very interesting, but had one limitation: it applied only to the gimp source tree, which is something of a best-case scenario for compression tools. A 6GB Test
I wanted to try something a bit more interesting. I made an uncompressed tar file of /usr on my workstation, which comes to 6GB of data. My /usr contains highly compressible data such as header files and source code, ELF binaries and libraries, already-compressed documentation files, small icons, and the like. It is a large, real-world mix of data. In fact, every compression comparison I saw was using data sets less than 1GB in size hardly representative of backup workloads. Let s start with the numbers:

Tool	MB saved	Space vs. gzip	Time vs. gzip	Cost
gzip	3398	100.00%	100.00%	0.15
bzip2	3590	92.91%	333.05%	0.48
pbzip2	3587	92.99%	183.77%	0.26
lzma -1	3641	91.01%	195.58%	0.28
lzma -2	3783	85.76%	273.83%	0.37

In the MB saved column, higher numbers are better; in all other columns, lower numbers are better. I m using clock seconds here on a dual-core machine. The cost column is clock seconds per MB saved. What does this tell us?

• bzip2 can do roughly 7% better than gzip, at a cost of a compression time more than 3 times as long.
• lzma -1 compresses better than bzip2 -9 in less than twice the time of gzip. That is, it is significantly faster and marginally smaller than bzip2.
• lzma -2 is significantly smaller and still somewhat faster than bzip2.
• pbzip2 achieves better wall clock performance, though not better CPU time performance, than bzip2 though even then, it is only marginally better than lzma -1 on a dual-core machine.

Some Pretty Charts First, let s see how the time vs. size numbers look: Like the other charts, the best area is the lower left, and worst is upper right. It s clear we have two outliers: gzip and bzip2. And a cluster of pretty similar performers. This view somewhat magnifies the differences, though. Let s add cat to the mix: And finally, look at the cost: Conclusions First off, the difference in time is far larger than the difference in space. We re talking a difference of 15% at the most in terms of space, but orders of magnitude for time. I think this pretty definitively is a death knell for bzip2. lzma -1 can achieve better compression in significantly less time, and lzma -2 can achieve significantly better compression in a little less time. pbzip2 can help even that out in terms of clock time on multicore machines, but 7za already has a parallel LZMA implementation, and it seems only a matter of time before /usr/bin/lzma gets it too. Also, if I were to chart CPU time, the numbers would be even less kind to pbzip2 than to bzip2. bzip2 does have some interesting properties, such as resetting everything every 900K, which could provide marginally better safety than any other compressor here though I don t know if lzma provides similar properties, or could. I think a strong argument remains that gzip is most suitable for backups in the general case. lzma -1 makes a good contender when space is at more of a premium. bzip2 doesn t seem to make a good contender at all now that we have lzma. I have also made my spreadsheet (OpenOffice format) containing the raw numbers and charts available for those interested. Update Part 2 of this story is now available, which considers more compression tools, and looks at performance compressing files individually rather than the large tar file.

Thanks to everyone involved. Just finished my pieces of the release process, generating 502 CD/DVD/BD images. And technically it's still February 14th, somewhere... :-) I should go and get some (more!) sleep, but I want to see us completely finished before I do. Update: (15th Feb) People have been asking about the BD images. We decided a while back that we wouldn't ship full-size BD ISO images, simply due to the sizes of those images. For the same reason, we didn't generate them for all architectures, just concentrating on i386, amd64 and source. If you're looking for the BD images, you can download/make them via jigdo. Update #2: (16th Feb) Yes, it's possible that we could make a multi-arch BD image containing all of i386, amd64 and source, similar to the multi-arch DVD. Last time I built that, it would just about fit on a dual-layer disc. That would be a really expensive disc to burn, but let me know by email and I can make the image for you if you really want it.

Congratulations to all developers (DDs or not, we have sponsored uploads, Debian contributors and such, too!) who contributed to the release of Debian GNU/Linux "lenny" 5.0. I must admit that I've been largely inactive recently, I just managed to keep the bugs on my remaining packages low. Funnily, just the day lenny was released I learned about a bug in Enigma on AMD64 that is probably worth fixing through proposed updates ...

Lately, I've been pondering how to solve one annoying aspect of the FreeDesktop, namely how to eject the piece of bloat called Firefox from the picture and get myself a fast web browser that I can rely on and that yet offers a similar UI experience based upon native GTK2 widgets rather than XUL components. Just to investigate possible options, I tested Epiphany, Galeon and Konqueror. My conclusions were than a Gecko-based Epiphany or Gaelon is incredibly fast and that Konqueror offers a similarily efficient experience albeit using QT, rather than GTK2 widgets. Why is Firefox so bloody bloated then, if it uses the same Gecko engine as Epiphany and Galeon? The only possible conclusion I could come up with is that its crappy XUL implementation, rather than leverage existing native GTK2 or QT widgets, tries to reinvent the wheel with its own UI toolkit running as a gigantic RAM blackhole. One option that I wanted to investigate but found too deficient was a Webkit-based port of Epiphany. Unfortunately, at its current stage, the GTK2 port of Webkit simply isn't mature enough to consider, although the fact that Webkit has been succesfully ported to a number of platforms and constitutes the basis for Safari on Mac OS suggests that the potential is significant. That only leaves one single aspect on which Firefox wins hands-down: its UI concept; it just works well, whereas Epiphany simply has a UI concept that is utterly inadequate, because it is too crude, lacking basic necessities such as a session saving feature that also works when purposely unloading the application, rather than only as a crash recovery measure. Simply put, Epiphany's premise that a browser should never be closed and thus ought to always remain open in the background of a desktop session is utterly flawed; it doesn't work like that in real life and Firefox acknowledges this, while Epiphany stubbornly doesn't because someone wreaked Havoc in our desktop paradigm. Anyhow, to me the solution is clear: regardless of which rendering engine it uses, Epiphany's UI concept needs to become more like Firefox, before it can truly gain acceptance as the default browser among Linux distributions.

The title of this book makes me realize how much a good and catchy title matters for a book. This one really caught my attention as I am fond of both irony and technics. Never thought of putting them together but someone surely has and it sounds like he has written a pretty nifty philosophical book on technology.

The title of this book makes me realize how much a good and catchy title matters for a book. This one really caught my attention as I am fond of both irony and technics. Never thought of putting them together but someone surely has and it sounds like he has written a pretty nifty philosophical book on technology.

Alright, Debian GNU/Linux 5.0 AKA as Lenny has been released. Time for a Debian unstable unfreeze party! What does the new stable release bring for system administrators? I ll give an overview what news you might expect when upgrading from Debian GNU/Linux 4.0, codename Etch (released on 8th April 2007) to the current version Debian GNU/Linux 5.0, codename Lenny (released on 14th February 2009). I try to avoid duplicated information so make sure to read the release announcement and the official release notes for Lenny beforehand. Noteworthy Changes

• initrd-tools got replaced by initramfs-tools
• netkit-inetd got replaced by openbsd-inetd
• the default syslog daemon sysklogd got replaced by rsyslog
• new defaults when creating ext2/ext3 file systems: dir_index and resize_inode feature enabled by default and use blocksize = 4096, inode_size = 256 and inode_ratio = 16384 (see /etc/mke2fs.conf)
• improved IPv6 support
• init.d-scripts for dependency-based init systems
• Debian-Volatile (hosting packages providing data that needs to be regularly updated over time, such as timezones definitions, anti-virus signature files, ) is an official service
• EVMS (Enterprise Volume Management System) was removed
• compatibility with the FHS v2.3
• software developed for version 3.2 of the LSB
• official Debian Lenny live systems for the amd64 and i386 architectures
• several new d-i features

Virtualisation

• kvm: works out-of-the-box with Lenny
• OpenVZ: deploying OpenVZ systems works out-of-the-box now thanks to new OpenVZ kernel image flavors (no need for any further, external repositories)
• vserver: the according kernel images are available
• Xen: works (dom0 as well as domU support), see Xen @ Debian-Wiki

Virtualisation related new tools:

• ganeti: Cluster-based virtualization management software
• libvirt-bin: Libvirt is a C toolkit to interact with the virtualization capabilities of recent versions of Linux (and other OSes). The library aims at providing a long term stable C API for different virtualization mechanisms.
• virtinst: Programs to create and clone virtual machines
• virt-manager: desktop application for managing virtual machines
• xen-shell: Console based Xen administration utility
• xenstore-utils: Xenstore utilities for Xen
• xenwatch: Virtualization utilities, mostly for Xen

Desktop oriented packages like virtualbox and qemu are available as well of course. Noteworthy Updates This is a (selective) list of some noteworthy updates:

• apache2: upgrade from 2.2.3-4+etch6 to 2.2.9-10+lenny2
• apt: upgrade from version 0.6.46.4-0.1 to 0.7.20.1, bringing apt-get autoremove
• aptitude: upgrade from version 0.4.4-4 to 0.4.11.11-1~lenny1, bringing aptitude safe-upgrade which replaces aptitude upgrade and aptitude full-upgrade replacing aptitude dist-upgrade
• bash: upgrade from 3.1dfsg-8 to 3.2-4
• bind: upgrade from 1:8.4.7-1 / 1:9.3.4-2etch4 to 1:9.5.0.dfsg.P2-5.1
• cfengine: upgrade from 2.1.20-1 to 2.2.8-1
• courier: upgrade from 0.53.3-5 to 0.60.0-2
• dpkg: upgrade from 1.13.26 to 1.14.24, bringing dpkg triggers
• drbd: upgrade from 0.7.21-4 to 2:8.0.14-2
• dstat: upgrade from 0.6.3-2 to 0.6.7-1, the new version for example provides the topcpu and topmem modules so you can run dstat -c -M topcpu -dng -M topmem to identify cpu and memory intensive jobs
• fai: upgrade from 3.1.8 to version 3.2.16, see NEWS
• gcc/g++: upgrade from 4:4.1.1-15 to 4.3.2-1.1
• git: upgrade from 1:1.4.4.4-4+etch1 to 1:1.5.6.5-2
• initramfs-tools: upgrade from 0.85i to 0.92o, see new Lenny features
• iproute: upgrade from 20061002-3 to 20080725-2
• iptables: upgrade from 1.3.6.0debian1-5 to 1.4.2-5
• gnu libc: upgrade from 2.3.6.ds1-13etch9+b1 to 2.7-18
• lighttpd: upgrade from 1.4.13-4etch11 to 1.4.19-5
• linux-kernel: upgrade from kernel 2.6.18 to 2.6.26, see wiki.debian.org/KernelFAQ#new-features-in-lenny
• lvm2: upgrade from 2.02.06-4etch1 to 2.02.39-6
• mdadm: upgrade from 2.5.6-9 to 2.6.7.1-1, see README.upgrading-2.5.3
• mercurial: upgrade from 0.9.1-1+etch1 to 1.0.1-5.1
• mount: upgrade from 2.12r-19etch1 to 2.13.1.1-1, bringing read-only binds, make-shared, make-slave, make-private, make-unbindable
• mysql: upgrade from 5.0.32-7etch8 to 5.0.51a-23
• nagios: upgrade from 2.6-2+etch1 to 3.0.6-3, see Upgrading From Nagios 2.x and What s New in Nagios 3 for details
• openldap: upgrade from 2.3.30-5+etch2 to 2.4.11-1
• openssh: upgrade from 1:4.3p2-9etch3 to 1:5.1p1-5
• postgresql: upgrade from 7.5.22 / 8.1.15-0etch1 to 8.3.5-1
• perl: upgrade from 5.8.8-7etch6 to 5.10.0-19
• php: upgrade from 5.2.0-8+etch13 to 5.2.6.dfsg.1-1+lenny2
• postfix: upgrade from 2.3.8-2+etch1 to 2.5.5-1.1
• puppet: upgrade from 0.20.1-1 to 0.24.5-3
• python: upgrade from 2.4.4-2 to 2.5.2-3
• ruby: upgrade from 1.8.5-4etch4 to 1.8.7.72-3
• samba: upgrade from 3.0.24-6etch10 to 2:3.2.5-4
• subversion: upgrade from 1.4.2dfsg1-2 to 1.5.1dfsg1-2
• tcl/tk: upgrade from 8.4.12-1etch2 to 8.5.3-4
• utilx-linux: upgrade from 2.12r-19etch1 to 2.13.1.1-1, brings new utilities like:
  • chrt: manipulate real-time attributes of a process
  • ionice: get/set program io scheduling class and priority
  • rename.ul: rename the specified files by replacing the first occurrence of from in their name by to
  • taskset: retrieve or set a process s CPU affinity
• zsh: upgrade from 4.3.2-25 to 4.3.6-6

New packages Lenny ships over 7000 new packages. Lists of new/removed/replaced packages are available online. I ll name 238 sysadmin related packages that might be worth a look. (Note: I don t list addon stuff like optional server-modules, docs-only and kernel-source related packages. I plan to present some of the following packages in more detail in separate blog entries.)

• ack-grep: A grep-like program specifically for large source trees
• acpitail: Show ACPI information in a tail-like style
• adns-tools: Asynchronous-capable DNS client library and utilities
• aggregate: ipv4 cidr prefix aggregator
• aosd-cat: an on screen display tool which uses libaosd
• apt-cacher-ng: Caching proxy for distribution of software packages
• apt-cross: retrieve, build and install libraries for cross-compiling
• aptfs: FUSE filesystem for APT source repositories
• apt-p2p: apt helper for peer-to-peer downloads of Debian packages
• apt-transport-https: APT https transport, use deb https://foo distro main lines in the sources.list
• arp-scan: arp scanning and fingerprinting tool
• array-info: command line tool reporting RAID status for several RAID types
• balance: Load balancing solution and generic tcp proxy
• bash-completion: programmable completion for the bash shell
• blktrace: utilities for block layer IO tracing
• daemonlogger: simple network packet logger and soft tap daemon
• daemontools: a collection of tools for managing UNIX services
• dbndns: Debian fork of djbdns, a collection of Domain Name System tools
• dcfldd: enhanced version of dd for forensics and security
• dctrl2xml: Debian control data to XML converter
• debomatic: automatic build machine for Debian source packages
• desproxy: tunnel TCP traffic through a HTTP proxy
• detox: utility to replace problematic characters in filenames
• di-netboot-assistant: Debian-Installer netboot assistant
• dish: the diligence/distributed shell for parallel sysadmin
• djbdns: a collection of Domain Name System tools
• dns2tcp: TCP over DNS tunnel client and server
• dnscache-run: djbdns dnscache service
• dnshistory: Translating and storing of IP addresses from log files
• dnsproxy: proxy for DNS queries
• dsyslog: advanced modular syslog daemon
• etckeeper: store /etc in git, mercurial, or bzr
• ext3grep: Tool to help recover deleted files on ext3 filesystems
• fair: high availability load balancer for TCP connections
• fatresize: FAT16/FAT32 filesystem resizer
• flog: dump STDIN to file and reopen on SIGHUP
• freeradius-utils: FreeRadius client utilities
• ganeti: Cluster-based virtualization management software
• gfs2-tools: Red Hat cluster suite - global file system 2 tools
• gitosis: git repository hosting application
• gptsync: GPT and MBR partition tables synchronisation tool
• grokevt: scripts for reading Microsoft Windows event log files
• grub2: GRand Unified Bootloader, version 2
• gt5: shell program to display visual disk usage with navigation
• haproxy: fast and reliable load balancing reverse proxy
• havp: HTTP Anti Virus Proxy
• heirloom-mailx: feature-rich BSD mail(1)
• hfsprogs: mkfs and fsck for HFS and HFS+ file systems
• hinfo: Check address ownership and DNSBL listings for spam reporting
• hlbr: IPS that runs over layer 2 (no TCP/IP stack required)
• hobbit: monitoring system for systems, networks and applications - server
• hotwire: Extensible graphical command execution shell
• hunchentoot: the Common Lisp web server formerly known as TBNL
• ifupdown-extra: Network scripts for ifupdown
• ike: Shrew Soft VPN client - Daemon and libraries
• incron: cron-like daemon which handles filesystem events
• inoticoming: trigger actions when files hit an incoming directory
• iodine: tool for tunneling IPv4 data through a DNS server
• iotop: simple top-like I/O monitor
• ipplan: web-based IP address manager and tracker
• ips: Intelligent process status
• iscsitarget: iSCSI Enterprise Target userland tools
• isns: Internet Storage Naming Service
• itop: simple top-like interrupt load monitor
• iwatch: realtime filesystem monitoring program using inotify
• jetring: gpg keyring mantainance using changesets
• john: active password cracking tool
• kanif: cluster management and administration swiss army knife
• keepassx: Cross Platform Password Manager
• keysafe: A safe to put your passwords in
• killer: Background job killer
• kpartx: create device mappings for partitions
• kvm: Full virtualization on x86 hardware
• latencytop: A tool for developers to visualize system latencies
• lbcd: Return system load via UDP for remote load balancers
• ldb-tools: LDAP-like embedded database - tools
• ldnsutils: ldns library for DNS programming
• lfhex: large file hex editor
• live-helper: Debian Live build scripts
• live-magic: GUI frontend to create Debian LiveCDs, netboot images, etc.
• logapp: supervise execution of applications producing heavy output
• lsat: Security auditor tool
• lustre-utils: Userspace utilities for the Lustre filesystem
• lwat: LDAP Web-based Administration Tool
• maatkit: Command-line utilities for MySQL
• mantis: web-based bug tracking system
• memdump: memory dumper
• memlockd: daemon to lock files into RAM
• metainit: Generates init scripts
• mirmon: monitor the state of mirrors
• mkelfimage: utility to create ELF boot images from Linux kernel images
• mongrel: A small fast HTTP library and server for Ruby
• monkey: fast, efficient, small and easy to configure web server
• monkeytail: tail variant designed for web developers monitoring logfiles
• mpy-svn-stats: Simple and easy to use svn statistics generator
• mr: a Multiple Repository management tool
• msr-tools: Utilities for modifying MSRs from userspace
• mtd-utils: Memory Technology Device Utilities
• munge: authentication service to create and validate credentials
• mxallowd: Anti-Spam-Daemon using nolisting/iptables
• mylvmbackup: quickly creating backups of MySQL server s data files
• myrescue: rescue data from damaged harddisks
• mysql-proxy: high availability, load balancing and query modification for mysql
• mysqltuner: high-performance MySQL tuning script
• nagvis: Visualization addon for Nagios
• ncdu: ncurses disk usage viewer
• netrw: netcat like tool with nice features to transport files over network
• netsend: a speedy filetransfer and network diagnostic program
• network-config: Simple network configuration tool
• nfdump: netflow capture daemon
• ngetty: getty replacement - one single daemon for all consoles
• nilfs2-tools: Continuous Snapshotting Log-structured Filesystem
• ninja: Privilege escalation detection system for GNU\Linux
• noip2: client for dynamic DNS service
• nsd3: authoritative domain name server (3.x series)
• ntfs-3g: read-write NTFS driver for FUSE
• nulog: Graphical firewall log analysis interface
• nuttcp: network performance measurement tool
• ocsinventory-server: Hardware and software inventory tool (Communication Server)
• odt2txt: simple converter from OpenDocument Text to plain text
• olsrd: optimized link-state routing daemon (unik-olsrd)
• onesixtyone: fast and simple SNMP scanner
• openais: Standards-based cluster framework (daemon and modules)
• opencryptoki: PKCS#11 implementation for Linux (daemon)
• openvas-client: Remote network security auditor, the client
• ophcrack: Microsoft Windows password cracker using rainbow tables
• op: sudo like controlled privilege escalation
• otpw-bin: OTPW programs for generating OTPW lists
• packeth: Ethernet packet generator
• paperkey: extract just the secret information out ouf OpenPGP secret key
• paris-traceroute: New version of well known tool traceroute
• password-gorilla: a cross-platform password manager
• pathfinderd: Daemon for X.509 Path Discovery and Validation
• pathfinder-utils: Utilities to use with the Pathfinder Daemon
• pcaputils: specialized libpcap utilities
• pcp: System level performance monitoring and performance management
• perlconsole: small program that lets you evaluate Perl code interactively
• pgloader: loads flat data files into PostgreSQL
• pgpool2: connection pool server and replication proxy for PostgreSQL
• pgsnap: PostgreSQL report tool
• pmailq: postfix mail queue manager
• pnputils: Plug and Play BIOS utilities
• policykit: framework for managing administrative policies and privileges
• postfwd: Postfix policyd to combine complex restrictions in a ruleset
• postpone: schedules commands to be executed later
• powertop: Linux tool to find out what is using power on a laptop
• prayer: standalone IMAP-based webmail server
• prelude-correlator: Hybrid Intrusion Detection System [ Correlator ]
• privbind: Allow unprivileged apps to bind to a privileged port
• pssh: Parallel versions of SSH-based tools
• ptop: PostgreSQL performance monitoring tool akin to top
• pyftpd: ftp daemon with advanced features
• rancid-core: rancid Really Awesome New Cisco confIg Differ
• rancid-util: Utilities for rancid
• rdnssd: IPv6 recursive DNS server discovery daemon
• rdup: utility to create a file list suitable for making backups
• reglookup: utility to read and query Windows NT/2000/XP registry
• rgmanager: Red Hat cluster suite - clustered resource group manager
• rinse: RPM installation environment
• rofs: Read-Only Filesystem for FUSE
• rsyslog: enhanced multi-threaded syslogd
• safe-rm: wrapper around the rm command to prevent accidental deletions
• samba-tools: tools provided by the Samba suite
• samdump2: Dump Windows 2k/NT/XP password hashes
• scalpel: A Frugal, High Performance File Carver
• scamper: advanced traceroute and network measurement utility
• scanmem: Locate and modify a variable in a running process
• schedtool: Queries/alters process scheduling policy and CPU affinity
• screenie: a small and lightweight GNU screen(1) wrapper
• scrounge-ntfs: Data recovery program for NTFS filesystems
• ser: Sip Express Router, very fast and configurable SIP proxy
• serverstats: a simple tool for creating graphs using rrdtool
• shutdown-at-night: System to shut down clients at night, and wake them in the morning
• sipcrack: SIP login dumper/cracker
• sks: Synchronizing OpenPGP Key Server
• slack: configuration management program for lazy admin
• sma: Sendmail log analyser
• smbind: PHP-based tool for managing DNS zones for BIND
• smbnetfs: User-space filesystem for SMB/NMB (Windows) network servers and shares
• softflowd: Flow-based network traffic analyser
• speedometer: measure and display the rate of data across a network connection
• spf-milter-python: RFC 4408 compliant Python SPF Milter for Sendmail and Postfix
• spf-tools-perl: SPF tools (spfquery, spfd) based on the Mail::SPF Perl module
• spf-tools-python: sender policy framework (SPF) tools for Python
• sqlgrey: Postfix Greylisting Policy Server
• ssdeep: Recursive piecewise hashing tool
• sshfp: DNS SSHFP records generator
• sshm: A command-line tool to manage your ssh servers
• sshproxy: ssh gateway to apply ACLs on ssh connections
• sslscan: Fast SSL scanner
• strace64: A system call tracer for 64bit binaries
• sucrack: multithreaded su bruteforcer
• supercat: program that colorizes text for terminals and HTML
• superiotool: Super I/O detection tool
• system-config-lvm: A utility for graphically configuring Logical Volumes
• system-config-printer: graphical interface to configure the printing system
• tack: terminfo action checker
• taktuk: efficient, large scale, parallel remote execution of commands
• tcpwatch-httpproxy: TCP monitoring and logging tool with support for HTTP 1.1
• terminator: Multiple GNOME terminals in one window
• timelimit: Simple utility to limit a process s absolute execution time
• tipcutils: TIPC utilities
• tor: anonymizing overlay network for TCP
• tpm-tools: Management tools for the TPM hardware (tools)
• tracker-utils: metadata database, indexer and search tool - commandline tools
• tumgreyspf: external policy checker for the postfix mail server
• ucspi-tcp: command-line tools for building TCP client-server applications
• unbound: validating, recursive, caching DNS resolver
• unhide: Forensic tool to find hidden processes and ports
• uniutils: Tools for finding out what is in a Unicode file
• unsort: reorders lines in a file in semirandom ways
• uphpmvault: upload recovery images to HP MediaVault2 via Ethernet
• usermode: Graphical tools for certain user account management tasks
• utf8-migration-tool: Debian UTF-8 migration wizard
• uuid-runtime: universally unique id library
• vblade-persist: create/manage supervised AoE exports
• vde2: Virtual Distributed Ethernet
• vdmfec: recover lost blocks using Forward Error Correction
• virtinst: Programs to create and clone virtual machines
• virt-manager: desktop application for managing virtual machines
• virtualbox-ose: x86 virtualization solution - binaries
• virt-viewer: Displaying the graphical console of a virtual machine
• watchupstream: Look for newer upstream releases
• whirlpool: Implementation of the whirlpool hash algorithm
• win32-loader: Debian-Installer loader for win32
• xavante: Lua HTTP 1.1 Web server
• xdelta3: A diff utility which works with binary files
• xen-shell: Console based Xen administration utility
• xenstore-utils: Xenstore utilities for Xen
• xenwatch: Virtualization utilities, mostly for Xen
• xfingerd: BSD-like finger daemon with qmail support
• xl2tpd: a layer 2 tunneling protocol implementation
• xrdp: Remote Desktop Protocol (RDP) server
• yersinia: Network vulnerabilities check software
• zerofree: zero free blocks from ext2/3 file-systems
• zipcmp: compare contents of zip archives
• zipmerge: merge zip archives
• ziproxy: compressing HTTP proxy server

Further Ressources

• Debian GNU/Linux 5.0 released - Release Announcement
• Debian GNU/Linux 5.0 - Release Notes
• Etch2Lenny Upgrade @ Debian-Wiki
• New in Lenny @ Debian-Wiki

Ok, so I know many have come across this as it was featured on BB but I would like to archive it here so I can find it later. It was just so well done

Ok, so I know many have come across this as it was featured on BB but I would like to archive it here so I can find it later. It was just so well done