Axel Beckert: Useful but Unknown Unix Tools: How wdiff and colordiff help to choose the right Swiss Army Knife
In light of the fact that it seems possible to fit the plastic caps of
a Debian branded Swiss Army Knife (Last orders
today!) on an existing Swiss Army Knife (German written
howto as PDF), I started to think about which Victorinox Cybertool
would be the best fitting for me.
And because the Victorinox comparison page doesn t really show diffs, just
columns with floating text which are not very helpful for generating
diffs in your head, I used command line tools for that purpose:
wdiff
Because the floating texts are not line- but just whitespace-based,
the tool of choice is not diff but wdiff, a word-based diff. It encloses additions and removals
in + + and [- -] blocks.
(No, those aren t Japanese smileys although they look a lot like some.
^^).
The easiest and clearest way is to copy and paste the texts from Victorinox comparison page into some text files and
compare them with wdiff:
$ wdiff cybertool34.txt cybertool41.txtSo this already extracted the information which are the seven tools which are in the Cybertool 41, but not in the Cybertool 34. Nevertheless the diff is still not easily recognizable on the first glance. There are several ways to help here. First
+Schraubendreher 2.5mm,+ Pinzette, N hahle mit Nadel hr, +Holzs ge,+ Bit-Schl ssel( 5 mm Innensechskant f r die D-SUB Steckverbinder, 4 mm Innensechskant f r Bits, Bit Phillips 0, Bit Phillips 1, Bit-Schlitzschrauben 4 mm, Bit Phillips 2, Bit Hex 4 mm, Bit Torx 8, Bit Torx 10, Bit Torx 15 ), Kombizange( H lsenpresser, Drahtschneider ), Stech-Bohrahle, Kugelschreiber( auch zum DIP-Switch verstellen ), Mehrzweckhaken (Pakettr ger), +Metalls ge( Metallfeile, Nagelfeile, Nagelreiniger ),+ Dosen ffner( kleiner Schraubendreher ), Kleine Klinge, Grosse Klinge, Ring, inox, Mini-Schraubendreher, Kapselheber( Schraubendreher, Drahtabisolierer ), +Holzmeissel / Schaber,+ Bit-Halter, Stecknadel, inox, Schere, Korkenzieher, Zahnstocher
wdiff has an option --no-common (the
according short option is -3) which just shows added and
removed words:
$ wdiff -3 cybertool34.txt cybertool41.txt ====================================================================== +Schraubendreher 2.5mm,+ ====================================================================== +Holzs ge,+ ====================================================================== +Metalls ge( Metallfeile, Nagelfeile, Nagelreiniger ),+ ====================================================================== +Holzmeissel / Schaber,+ ======================================================================This is already way better to quickly recognize the actual differences. But if you still also want to see the common tools of the two knifes you need some visual help: One option is to use
wdiff s --terminal (or
short -t) option. Added words are then displayed inverse
and removed words are shown underlined (background and foreground
colors hardcoded as there is no invert colors style in CSS or HTML):
But some still like to to use color instead of the contrast-rich inverse and the easily to oversee underlining. This is where colordiff comes into play: colordiff colordiff is like syntax highlighting for diffs on the command line. I works with classic and unified diffs as well as with wdiffs and debdiffs (the$ wdiff -t cybertool34.txt cybertool41.txt
Schraubendreher 2.5mm, Pinzette, N hahle mit Nadel hr, Holzs ge, Bit-Schl ssel( 5 mm Innensechskant f r die D-SUB Steckverbinder, 4 mm Innensechskant f r Bits, Bit Phillips 0, Bit Phillips 1, Bit-Schlitzschrauben 4 mm, Bit Phillips 2, Bit Hex 4 mm, Bit Torx 8, Bit Torx 10, Bit Torx 15 ), Kombizange( H lsenpresser, Drahtschneider ), Stech-Bohrahle, Kugelschreiber( auch zum DIP-Switch verstellen ), Mehrzweckhaken (Pakettr ger), Metalls ge( Metallfeile, Nagelfeile, Nagelreiniger ), Dosen ffner( kleiner Schraubendreher ), Kleine Klinge, Druckkugelschreiber, Grosse Klinge, Ring, inox, Mini-Schraubendreher, Kapselheber( Schraubendreher, Drahtabisolierer ), Holzmeissel / Schaber, Bit-Halter, Stecknadel, inox, Schere, Korkenzieher, Zahnstocher
debdiff command is part of the
devscripts package).
$ wdiff cybertool34.txt cybertool41.txt colordiff(Coloured Screenshots done with ANSI HTML Adapter from the package aha.) Some, especially those who are used to git, are probably confused by the default choice of diff colors. This is easily fixable by writing the following into you
+Schraubendreher 2.5mm,+ Pinzette, N hahle mit Nadel hr, +Holzs ge,+ Bit-Schl ssel( 5 mm Innensechskant f r die D-SUB Steckverbinder, 4 mm Innensechskant f r Bits, Bit Phillips 0, Bit Phillips 1, Bit-Schlitzschrauben 4 mm, Bit Phillips 2, Bit Hex 4 mm, Bit Torx 8, Bit Torx 10, Bit Torx 15 ), Kombizange( H lsenpresser, Drahtschneider ), Stech-Bohrahle, Kugelschreiber( auch zum DIP-Switch verstellen ), Mehrzweckhaken (Pakettr ger), +Metalls ge( Metallfeile, Nagelfeile, Nagelreiniger ),+ Dosen ffner( kleiner Schraubendreher ), Kleine Klinge, Grosse Klinge, Ring, inox, Mini-Schraubendreher, Kapselheber( Schraubendreher, Drahtabisolierer ), +Holzmeissel / Schaber,+ Bit-Halter, Stecknadel, inox, Schere, Korkenzieher, Zahnstocher $ wdiff cybertool29.txt cybertool41.txt colordiff
+Schraubendreher 2.5mm,+ Pinzette, N hahle mit Nadel hr, +Holzs ge,+ Bit-Schl ssel( 5 mm Innensechskant f r die D-SUB Steckverbinder, 4 mm Innensechskant f r Bits, Bit Phillips 0, Bit Phillips 1, Bit-Schlitzschrauben 4 mm, Bit Phillips 2, Bit Hex 4 mm, Bit Torx 8, Bit Torx 10, Bit Torx 15 ), +Kombizange( H lsenpresser, Drahtschneider ),+ Stech-Bohrahle, +Kugelschreiber( auch zum DIP-Switch verstellen ), Mehrzweckhaken (Pakettr ger), Metalls ge( Metallfeile, Nagelfeile, Nagelreiniger ),+ Dosen ffner( kleiner Schraubendreher ), Kleine Klinge, [-Druckkugelschreiber,-] Grosse Klinge, Ring, inox, Mini-Schraubendreher, Kapselheber( Schraubendreher, Drahtabisolierer ), +Holzmeissel / Schaber,+ Bit-Halter, Stecknadel, inox, +Schere,+ Korkenzieher, Zahnstocher
~/.colordiffrc:
newtext=green oldtext=red diffstuff=darkblue cvsstuff=darkyellow(See also
/etc/colordiff for the defaults and hints.)
colordiff has by the way two operating modes:
- Without parameter it reads diffs from standard input as seen above.
- With parameters it works as drop-in diff replacement including all diff options as shown below.
$ colordiff cybertool29-lines.txt cybertool34-lines.txt 12a13,14 > Kombizange( H lsenpresser > Drahtschneider ) 13a16,17 > Kugelschreiber( auch zum DIP-Switch verstellen ) > Mehrzweckhaken (Pakettr ger) 16d19 < Druckkugelschreiber 25a29 > SchereOr as unifed diff with some context:
$ colordiff -u cybertool29-lines.txt cybertool34-lines.txt --- cybertool29-lines.txt 2011-08-31 20:55:37.195546238 +0200 +++ cybertool34-lines.txt 2011-08-31 20:55:11.667710504 +0200 @@ -10,10 +10,13 @@ Bit Torx 8 Bit Torx 10 Bit Torx 15 ) +Kombizange( H lsenpresser +Drahtschneider ) Stech-Bohrahle +Kugelschreiber( auch zum DIP-Switch verstellen ) +Mehrzweckhaken (Pakettr ger) Dosen ffner( kleiner Schraubendreher ) Kleine Klinge -Druckkugelschreiber Grosse Klinge Ring inox @@ -23,5 +26,6 @@ Bit-Halter Stecknadel inox +Schere Korkenzieher ZahnstocherSo if you want nicely colored diffs with Subversion like you re used to with git, you can use
svn diff colordiff.
TeleMini Dual-deploy altimeter with telemetry now available
TeleMini is a miniature dual-deploy flight computer with data logging
and radio telemetry. Small enough to fit comfortably in an 18mm tube,
this powerful package does everything you need on a single board:
User BobH asked
This is my fifth and final report about dextools for the 2011 Google Summer of Code program. During the summer, I worked with Matt Zimmerman and Stefano Zacchiroli to create a replacement web dashboard for the Debian Derivatives Exchange (DEX). The dashboard displays a list of projects and their respective tasks and then allows users to easily update the status of these tasks. The dashboard also contains two graphs for tracking the progress of projects and providing instant recognition of all contributions to a project.
At the start of the summer, I knew that I wanted to to work on improving the DEX infrastructure. I had worked with the team on the ancient-merges project, and while a lot of good work was accomplished, the process for tracking our progress was a bit clunky and difficult to interact with. Anyone who read my initial proposal for the Summer of Code would probably agree that it was quite vague. I really did not have a solid plan for what I wanted to accomplish.
This began to change after my initial phone call with my mentor, Matt Zimmerman. We decided that the first thing I would work on would be a basic dashboard. Our plan was to have all of the data stored on the Debian BTS. This would allow Debian Maintainers to benefit from DEX without needing to learn about DEX s tools and workflows. The dashboard would support multiple derivatives and multiple projects for each derivative. Each project would be made up of a list of tasks, where each task is linked to a BTS bug.
I prepared a mockup and some initial code based on that plan. The dashboard was able to successfully display a list of BTS bugs and their information. It generated the lists by assuming that bugs would be usertagged with a user of debian-derivatives@lists.debian.org and a tag in the format of dex 
Here is the short version: to get better performance on your 
The most important plot is the top one. The blue line is the attempted
number of transactions per second (
It is able to get the same performance as stud (763 
As you can see in the plot above, only stud is able to scale
properly. stunnel is not able to take advantage of the cores and its
performances are worse than with a single core. I think this may be
due to its threaded model and the fact that the userland for this
32bit system is a bit old. nginx is able to achieve the same 
This plot also explains why stud performances fall after the
maximum: because of the failed transactions, the session cache is not
as efficient. This phenomenon does not exist when session reuse is not enabled.
(This was going to be a comment on 
So, on July 31st 2011, it was exactly 10 years since I am a Debian
developer. What happened in the meantime? What lead me to this? What
turned this unskilled dude into a sometimes quite visible contributor
of one of the major free software projects?
If you're interested in that, please read on. Otherwise, well, this is
just yet another "bubulle talks about self" post and you can skip it.
Well, first of all, how did I end up being a DD?
And first of firsts, how did I end up not being a random user of
Windows, playing games on his desktop computer at work?
I am not a computer scientist, an "informaticien" as we sometimes say
in French (most often, what people who are not in computing stuff
say...thinking that all folks working more or less closely to
computers know everything about them).
I graduated with a PhD in
Materials science, in 1989 after conducting a research on "Influence
of Yttrium Oxide dispersion on the strength of titanium alloys", at
Onera, a French public research institution for aerospace and defence.
I was then hired, still at Onera (where I'm still working, 25 years
after starting my PhD work) to lead the Mechanical Testing Laboratory in the
Materials Science department.
So, my lab had many big machines
designed to conduct creep and fatigue tests often at interestingly
high temperatures such as 2000 C, on samples of various
materials that are used in aircrafts structures, engines, etc. or in
various space thingies (remember Herm s, the european shuttle?) or in
various "things that fly but just one way and you shouldn't be there
when they land".
So, we had computers handling data acquisition for these tests. So I
became involved with designing data acquisition setups, or even
programming data acquisition programs (one of mine, written in Forth,
ran all Onera creep tests until 2004 and successfully passed Y2K
because I knew that 2000 was a leap year. It could even pass 2100 as I
knew this is not a leap year..:-)
One day, I had to buy a modem in order to communicate with out italian
counterpart (CIRA) and exchange tests results. Then I had to learn how
modems work in MS-Dos(yeah...). Then, I discovered "online"
resources...indeed more that strange world that was then called "BBS"
(Bulletin Board Systems), those mysterious things ran by some happy
few who were using modems at their home place to communicate in
"forums" and everything related to technology.
PC-Board, Remote
Access, Fidonet, etc. became familiar to me these days, back in
1988-1990. So familiar that I ended up running my own BBS at home,
killing our phone bills and using very sophisticated techniques such as
US Robotics "High Speed Transfer" modems that could be used at
19200bps asymetrical for very high speed transfers of kilobytes and
kilobytes of useless MS-Dos "freeware" and "shareware".
Indeed, my
very first BBS didn't even have one of these: it was using a cheaper
V.22bis modem operating at 2400bps. I was waiting for my order of a
sophisticated HST modem to arrive from USA through obscure import
channels meant to circumvent the French telephone company regulations
that were forbidding the use of "unapproved" systems, in order to
protect the famous French Telephone System from interferences brought
by Bad American Material.
Bubulle System was born.
During those years, I discovered very interesting things: computers can
run together once you draw a wire between them. That's called Local
Area Network and you can even transfer data at 1Mbps between two
computers, assuming you don't forget to put terminating resistors at
the end of the line on these funny 10-baseT connectors at the back of
your home-assembled PC that was using 2MB of RAM (bought for very
cheap through the help of an American friend who was in touch with
some Chinese folks who were selling 256kb RAM sticks for half the
retail price....assuming you want to drive in a mysterious storing
place close to Charles de Gaulle airport or Eurodisney).
Hello Gordon. Yes, I know, we're still friends on Facebook and you're
probably still using that weird programming language which you were,
IIRC, the only person in the world to use.
2MB, that should be enough for barely anything, including running
*multitasking* software where, miracle, I could run two tasks at the
same time on my one and only PC at home. Miracle, I don't have to
shutdown my BBS if I want to read forums on my friends BBS. Yay for
Desqview/386! Multitasking for dummies
Still, I have to build one of these "networks" at home. Elizabeth
won't like that as it means one more box (home-made of course) in our
living room and some more wires. And why the hell is it running 24/7?
So that friends can visit the BBS, darling... And I can even
communicate with them: I write a message, I get an answer the day
after and so on. In one full week, we can have a great conversation
that would have taken *minutes* to have in the real life. Isn't this
the miracle of technlogy?
And, yes, this is a good reason for having
phone bills raising up to 500 Francs/month: people can "exchange"
programs through my BBS, that horrible white box running in the living
room. Often, these programs are written for free and some of them are
even given with "source code", which allows people to *modify* them.
And that even makes friends, you know? Imagine that some day we have
to move from one house to another: then I can just call out "who
wants to help bubulle moving?", and probably a dozen of (sometimes
scary but always nice and polite...and sometimes even showered) geeks
will pop up and happily carry boxes full of my vinyl LP collection all
day long. An entire world of friends.
"bubulle", you say, dear? What's that? That's my nickname. It was
invented by one of these friends, a really strange guy named Ren
Cougnenc who wrote this "free software" program anmed BBTH, which
allows you to use modems to connect to BBS, and even to those many
"Minitel" BBS we have in France, thanks to our wonderful world-leading
technology using V.23 communication.
Many people know me as "bubulle" because, you know, Perrier water has
bubbles and Ren likes Gaston Lagaffe fish companion who he named "bubulle".
Ren , I love you. You chose to leave this world back in 1998. We'll no
longer have our "p'tits midis" in Antony where you were showing me the
marvels of what's coming in next episode.
All this was around 1988 and about 1992, doing all these mysterious
things at home (between Jean-Baptiste and Sophie's diapers) while
still working with data acquisition MS-Dos machines at work.
How did this end up being a Debian Developer? You'll know in the next
episode..:-)
