On my everyday netbook (a very reliable first generation ASUS EeePC 701 4G) the disk (4 GB as the product name suggests :-) is nearly always close to full. TL;DWTR? Jump directly to the HowTo. :-) So I came up with a few techniques to save some more disk space. Installing localepurge was one of the earliest. Another one was to implement aptitude filters to do interactively what deborphan does non-interactively. Yet another one is to use du and friends a lot ncdu is definitely my favourite du-like tool in the meanwhile. Using du and friends I often noticed how much disk space /usr/share/doc takes up. But since I value the contents of /usr/share/doc a lot, I condemn how Nokia solved that on the N900: They let APT delete all files and directories under /usr/share/doc (including the copyright files!) via some package named docpurge. I also dislike Ubuntu s solution to truncate the shipped changelog files (you can still get the remainder of the files on the web somewhere) as they re an important source of information for me. So when aptitude showed me that some package suddenly wanted to use up quite some more disk space, I noticed that the new package version included the upstream changelog twice. So I started searching for duplicate files under /usr/share/doc. There are quite some tools to find duplicate files in Debian. hardlink seemed most appropriate for this case. First I just looked for duplicate files per package, which even on that less than four gigabytes installation on my EeePC found nine packages which shipped at least one file twice. As recommended I rather opted for an according Lintian check (see bugs. Niels Thykier kindly implemented such a check in Lintian and its findings are as reported as tags duplicate-changelog-files (Severity: normal, from Lintian 2.5.2 on) and duplicate-files (Severity: minor, experimental, from Lintian 2.5.0 on). Nevertheless, some source packages generate several binary packages and all of them (of course) ship the same, in some cases quite large (Debian) changelog file. So I found myself running hardlink /usr/share/doc now and then to gain some more free disk space. But as I run Sid and package upgrades happen more than daily, I came to the conclusion that I should run this command more or less after each aptitude run, i.e. automatically. Having taken localepurge s APT hook as example, I added the following content as /etc/apt/apt.conf.d/98-hardlink-doc to my system:

// Hardlink identical docs, changelogs, copyrights, examples, etc
DPkg
 
Post-Invoke  "if [ -x /usr/bin/hardlink ]; then /usr/bin/hardlink -t /usr/share/doc; else exit 0; fi"; ;
 ;

So now installing a package which contains duplicate files looks like this:

~ # aptitude install perl-tk
The following NEW packages will be installed:
  perl-tk 
0 packages upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 2,522 kB of archives. After unpacking 6,783 kB will be used.
Get: 1 http://ftp.ch.debian.org/debian/ sid/main perl-tk i386 1:804.029-1.2 [2,522 kB]
Fetched 2,522 kB in 1s (1,287 kB/s)  
Selecting previously unselected package perl-tk.
(Reading database ... 121849 files and directories currently installed.)
Unpacking perl-tk (from .../perl-tk_1%3a804.029-1.2_i386.deb) ...
Processing triggers for man-db ...
Setting up perl-tk (1:804.029-1.2) ...
Mode:     real
Files:    15423
Linked:   3 files
Compared: 14724 files
Saved:    7.29 KiB
Duration: 4.03 seconds
localepurge: Disk space freed in /usr/share/locale: 0 KiB
localepurge: Disk space freed in /usr/share/man: 0 KiB
localepurge: Disk space freed in /usr/share/gnome/help: 0 KiB
localepurge: Disk space freed in /usr/share/omf: 0 KiB
Total disk space freed by localepurge: 0 KiB

Sure, that wasn t the most space saving example, but on some installations I saved around 100 MB of disk space that way and I still haven t found a case where this caused unwanted damage. (Use of this advice on your own risk, though. Pointers to potential problems welcome. :-)

In software development, you often need to know: when was this feature released? in which releases was this changeset included?

Typical example is: you're debugging a problem, you find out what's the changeset introducing it, now you want to know when that changeset was firstly released (i.e. included in a tag) - how to do that?

hg log -r "cset:: and tag()"

where cset is the changeset you're looking for.

So know I know that Sphinx new way to load searchindex.js was introduced in the this changeset, and released for the first time in 1.0b2:

$ hg log -r "423faa03c908:: and tag()"
changeset: 2261:b494009dccf1
tag: 1.0b2
user: Georg Brandl <georg@python.org>
date: Sun May 30 19:52:08 2010 +0200
summary: Release preparation for 1.0b2.
...

awesome!

PS: why did I need to know that? because Matplotlib is still using the old way, but Debian needs the new one.

RDieHarder is an R package providing access to the DieHarder battery of tests for random number generators developed by Robert G. Brown and others. DieHarder had been updated to version 3.1.1 a while back, and I had been a little behind with updating RDieHarder. Version 0.1.2 rectifies this. The package still comes with a vignette describing both DieHarder and the RDieHarder package. And because pictures speak louder than a thousand (blogged) words, here is the first chart from the vignette: On the left, we have a poor random-number generator (RNG), the older ran0 function. The histogram illustrating the distribution of test scores is somewhat uneven. An ideal (and asymptotic) outcode is a uniform distribution of p-values from the test. The empirical cumulative distribution function (ECDF) below indicates a somewhat pronounced departure from the diagonal. Informally speaking, this is what the (Kuiper-)Kolmogorov-Smirnov test quantifies, and we see (in the text in the chart) that the null of can be rejected an conventional levels. Based on this example (which had a short run-time with few samples) we would indeed mistrust this (known bad) RNG. On the right, we have a more recent and trusted RNG, the well-known Mersenne Twister. The ten histogram buckets are all closer to the expected value of one-tenth, the estimated density is closer to flat, the ECDF is closer to the diagonal and the tests don't reject---so no reason to mistrust this RNG based on this test alone. RDieHarder lets you run a battery of such tests against a boatload of known RNGs. Here is a second example, comparing the six RNGs built into R itself: And these six look fine, as you'd expect. (And yes, the ECDF charts should each be on a square plot. Another time...) Courtesy of CRANberries, there is also a diffstat report for 0.1.2 relative to the older 0.1.1 release. More detailed information is on the RDieHarder page.

Many websites have addresses that use a www.- prefix. Some people are encouraging this practice, other are advising against it. I, for one, think this prefix makes sense and is really useful, so here is an explanation of my opinion. In a nutshell: the Wold Wide Web is one Internet service among many other such as domain name, mail, mailboxes or instant messenging; using a dedicated prefix allows to distribute all these services to specific servers in a convenient way.On Internet, you can expose several services, the most common being domain names (DNS), websites (HTTP), incoming mail (SMTP), outgoing mail submission (submission), mailbox fetching (POP) or consultating (IMAP) and instant messenging (XMPP). Constraints Modern Internet services can have constraints such as load balancing, failover and most importantly the possibility to use specific servers for each service instead of having all of them on the same server. Dedicated records Some service were designed or revised to take these contraints into account; this include the domain name, incoming mail and instant messenging, by using dedicated DNS records:

example.com.                    NS              roquefort.example.com.
example.com.                    MX  10          morbier.example.com.
_xmpp-server._tcp.example.com.  SRV 10 100 5269 morbier.example.com.

In that example, the name service of example.com. is provided by the server Roquefort while the incoming mail and instant messenging service are provided by Morbier Dedicated prefixes On the contrary, some services were not designed to take these constraint into account, and may be too fossilized to be revised for that; this includes the Web and mailbox service. A convenient way to work around the antiquated design of these services is to use a well known prefix:

www.example.com.                CNAME           ossau-iraty.example.com.
smtp.example.com.               CNAME           morbier.example.com.
pop.example.com.                CNAME           morbier.example.com.
imap.example.com.               CNAME           morbier.example.com.

In that example, the Web service of example.com. is provided by the server Ossay-Iraty while the email submission and mailbox services are provided by Morbier. Readability and flexibility Using dedicated records and prefixes increases the consistency and readability of a DNS zone, since it allows to refer to the servers by their canonical host name, and to define their IP addresses once and for all:

; Services
example.com.                    NS              roquefort.example.com.
example.com.                    MX  10          morbier.example.com.
pop.example.com.                CNAME           morbier.example.com.
imap.example.com.               CNAME           morbier.example.com.
www.example.com.                CNAME           ossau-iraty.example.com.
; Hosts
roquefort.example.com.          A               192.2.0.12
                                AAAA            2001:db8::12
morbier.example.com             A               192.2.0.42
                                AAAA            2001:db8::42
ossau-iraty.example.com.        A               192.2.0.51
                                AAAA            2001:db8::51

You can think of that practice as connecting dots between services and hosts:

name service    Roquefort
incoming mail   Morbier
mailboxes         Ossau-Iraty
website        

With such a DNS zone, it is very easy to identify which server is hosting each service. Using host names rather than IP addresses multiple times reduces the information duplication, which in turn facilitates future chances and reduces the risk of error (it is easier to swith the website from Ossay-Iraty to Morbier than from 192.2.0.51/2001:db8::51 to 192.2.0.42/2001:db8::42).

The Debian Project is pleased to announce that registration is now open for DebConf12, which will take place in Managua, Nicaragua, from Sunday 8 July to Saturday 14 July 2012. The conference will be preceded as usual by one week of DebCamp, from Sunday 1 July to Saturday 7 July. As previous years, three different registration options will be available: Sponsored, Professional and Corporate. Through the Sponsored category, the Debian Project intend to help some people attend who would not be able to pay for their own travel to DebConf. The Corporate category, with a USD 1300 (EUR 1000) registration fee is intended for those attending DebConf as representatives of companies. The Professional category, with a USD 650 (EUR 500) registration fee, is intended for any individual or company who wants to help the conference by paying the direct costs of their attendence. It will only be possible to apply for food and accommodation grants and/or travel grants until 15 May. For more detailed information about how to register, please see the separate announcement from the DebConf team. Call for papers Debian invites submissions of proposals for papers, presentations, discussion sessions and tutorials for its DebConf12 conference. Submissions are not limited to raditional talks: you could propose a performance, art installation, debate, or anything else. Official submissions will be accepted until 1 June 2012, 23:59 UTC. This year, we will continue to organize some talks into thematic tracks. If you have a proposal for a DebConf track, please contact <debconf-team@lists.debconf.org>. Previous examples included Debian Science, Debian Infrastructure, and Community Outreach tracks. If you would like to be a track coordinator, please volunteer on that list. Detailed information on how to submit an event are available on the mail from the DebConf team.

---

---

---

I got into a discussion recently about what a Linux distribution actually is. To some people, a distro is just a convenient way of source of binaries from which to pick what you install, either on your own computer or when creating an embedded system, or some other specific kind of system. To others it's a way of life, and the realization of the ideals of software freedom. To yet others, a distro is an outdated concept that will go away in the near-ish future, to be replaced by ... something. Here's my take: a Linux distribution is defined by the following things:

• It's a specific, chosen set of upstream projects. The selection criteria vary wildly between distros, and may be based on things like purpose, quality, licenses, etc. Debian, for example, chooses pretty much everything that is free software, but a distro targeted at providing NAS devices would choose a much smaller set.
• It's a set of changes made to the upstream code, and specific configurations of the upstream code, in order make all the software work well together. For example, if the distro supports multiple mail transport agents, they might all need to be patched and configured to store incoming mail the same way by default.
• It's the tools, processes, policies, and workflows used to develop the distribution.
• It's the tools provided to the people using the distribution to install and manage their systems.
• It's the people and companies who develop the distribution and support its users, and their shared values and purpose. This may be a real community, or just the kind of community that exists as a web page.

Historically, there has been a lot of variation in all aspects mentioned above. Some of this variation may today be unnecessary. For example, it might be helpful for more distributions to share work on common tools, so that not everyone needs to re-invent and re-implement the same kinds of tools. However, my gut feeling is that due to the inherent interconnectedness of all things, it may be very difficult to have one set of tools to support well all workflows, policies, processes, and communities. The most important differentiating factors between the various distros are not the tools, but the community values, the perceived purpose of the distro, and the resultant criteria for choosing what to include and the workflow for developing the distro. Even when two distros use the same tools, there can be a lot of difference in the end result. Fedora and RHEL, for example, use a lot of the same infrastructure, but their purpose is entirely different. Debian and Ubuntu use mostly the same packages, even, and still there's a huge difference between the two distros: one aims at a high quality system based on free software, the other aims to support the business of one particular company. For this reason, there will always be a lot of Linux distributions around, and that is good.

Recently I have spent time with Skolelinux Drift AS on speeding up a Debian Edu / Skolelinux Lenny installation using LTSP diskless workstations, and in the process I discovered something very surprising. The reason the KDE menu was responding slow when using it for the first time, was mostly due to the way KDE find application icons. I discovered that showing the Multimedia menu would cause more than 20 000 IP packages to be passed between the LTSP client and the NFS server. Most of these were NFS LOOKUP calls, resulting in a NFS3ERR_NOENT response. Because the ping times between the client and the server were in the range 2-20 ms, the menus would be very slow. Looking at the strace of kicker in Lenny (or plasma-desktop i Squeeze - same problem there), I see that the source of these NFS calls are access(2) system calls for non-existing files. KDE can do hundreds of access(2) calls to find one icon file. In my example, just finding the mplayer icon required around 230 access(2) calls. The KDE code seem to search for icons using a list of icon directories, and the list of possible directories is large. In (almost) each directory, it look for files ending in .png, .svgz, .svg and .xpm. The result is a very slow KDE menu when /usr/ is NFS mounted. Showing a single sub menu may result in thousands of NFS requests. I am not the first one to discover this. I found a KDE bug report from 2009 about this problem, and it is still unsolved. My solution to speed up the KDE menu was to create a package kde-icon-cache that upon installation will look at all .desktop files used to generate the KDE menu, find their icons, search the icon paths for the file that KDE will end up finding at run time, and copying the icon file to /var/lib/kde-icon-cache/. Finally, I add symlinks to these icon files in one of the first directories where KDE will look for them. This cut down the number of file accesses required to find one icon from several hundred to less than 5, and make the KDE menu almost instantaneous. I'm not quite sure where to make the package publicly available, so for now it is only available on request. The bug report mention that this do not only affect the KDE menu and icon handling, but also the login process. Not quite sure how to speed up that part without replacing NFS with for example NBD, and that is not really an option at the moment. If you got feedback on this issue, please let us know on debian-edu (at) lists.debian.org.

I met Francesca in Debconf 11 in Banja Luka. If I recall correctly, it s Enrico Zini who introduced me to her, because she was the madamezou (her IRC nickname) who started to get involved in the publicity team. Since then and despite having a bachelor thesis to complete she got way more involved and even gained official responsibilities in the project. Before starting with the interview, I wanted to mention that Francesca is drafting a diversity statement for Debian I was expecting the discussions to go nowhere but she listened to all objections and managed to improve the text and build a consensus around it. Thank you for this and keep up the good work, Francesca! Rapha l: Who are you? Francesca: My name is Francesca, I m 30 and I studied Social Sciences. Currently I live in Italy but I m planning to go abroad (not a lot of jobs here for geeky social scientists). Apart for Debian and FLOSS world in general, I have unrestrained passions for chocolate; zombie movies; sci-fi; zombie books; knitting sewing crafting and DIY in general; zombie videogames; bicycles; pulling apart objects to look inside them; splatter B movies, David Foster Wallace s books, playing trumpet, and did I already mentioned zombies? Days are too short for all this stuff, but I try to do my best. Raphael: How did you start contributing to Debian? Francesca: Some years ago I was stuck in bed for literally some months, due to a grave series of migraine attacks. I wasn t able to do anything: no social life, no books or television. So, I decided to turn on the laptop and do something constructive with it: I was already a Debian user and it seemed quite logical to me to try to give back to the community. I am not a coder and I ve not studied Computer Science, so my first step was to join an Italian Debian on-line community (Debianizzati) and help with tutorials, users support, wiki management. In a couple of months I learnt many things: helping other users with their problems forces you to do lots of research! My first contributions to the Debian project were mostly translations of the main website. Translators are the perfect typos spotters: they work so precisely on the text to be translated that they finish to do a great QA job. This is how I ve started to contribute to the Debian website: with very simple things, fixing typos or wrong links or misplaced wml tags. I still remember my first commit to the website: the idea was to undercase some tags, but it ended up that I misplaced some of them and in addition I fixed them only in the English page and not on the translations as well. When after a couple of minutes, K re Thor Olsen a long time contributor of the team and now webmaster reverted my commit, I felt so stupid and full of shame. But, to my great surprise, no one treated me like an idiot for that error: Gerfried Fuchs, one of the guru of the team, replies me in a really helpful and polite way explaining what I did wrong and how to do things correctly. I think this episode was a turning point in my Debian life: there s this idea that Debian Developers are just a bunch of arrogant assholes and maybe it was true in the past, but for my experience they are not. Well, at least the ones I met and work with .

To my great surprise, no one treated me like an idiot for that error.

Since then, I joined the WWW team and helped them apply the shiny new design provided by Kalle S derman. A lot of work was done during the week immediately before the release of the new website. Oh that was a week! We worked night and day to have the new design ready for February 6th, and it was fantastic when we finally published it, simultaneously with the release of Squeeze. At the same time, I started to contribute more actively to the Debian Publicity team, not only translating news but also writing them. It can sound scary for a non native English speaker to write something from scratch in English, but you have to keep in mind that your text will be reviewed by native speakers before being published. And we have some fantastic reviewers in the English localisation team: particularly Justin B Rye, who is tireless in his effort and more recently Moray Allan. I think I m particularly lucky to work with all these people: there s a special mood in both Publicity and WWW team, which makes you feel happy to do things and at the same time pushes you to do more just because it s fun to work with them sharing jokes, ideas, rants, patches and hugs. Rapha l: I believe that you have been trough the new member process very quickly. You re now a Non-Uploading Debian Developer. How was the experience and what does this mean to you? Francesca: Becoming a Debian Developer was not so obvious for me, because I didn t need to be a DD for the work I do in Debian. For instance, I don t maintain packages, so I had no reasons to want to become a DD in order to have uploading rights. For a while I didn t really feel the necessity of being a DD. Luckily, some people started to pester me about it, asking me to apply for the NM process. I remember Martin Zobel-Helas doing this for an entire week every single day, and Gerfried Fuchs doing it as well. Suddenly, I realized that people I worked with felt that I deserved the DD status and that I simply had thought I didn t. As a non coder and a woman, there probably was a bit of impostor syndrome involved. Having people encouraging me, gave me more confidence and the desire to finally become a DD. And so I did. The process for non uploading DD is identical to the one to become an uploading DD, with one exception: in the second part of the process (named Tasks and Skills) instead of questions about how to create and maintain packages, there are questions about the non packaging work you usually do in Debian. The general resolution which created the possibility to become a non uploading DD gave us a chance to recognize the great effort of Debian contributors who work in various area (translations, documentation, artworks, etc.) that were not always considered as important as packaging efforts. And this is great because if you are a regular contributor, if you love Debian and you are committed to the project, there are no reasons to not be an official member of it. With regards to this, I like the metaphor used by Meike Reichle in her recent talk about the Debian Women Project (video recording here):

a Debian Developer status is a lot like a citizenship in a country that you re living in. If you live in a country and you don t have citizenship, you can find a job, buy a house, have a family [...] but if this country at any point in time decides to go into a direction that you don t like, there s nothing you can do about it. You are not in the position to make any change or to make any effect on that country: you just live there, but there s no way that you can excercise influence on the people who run this country.

Rapha l: You recently joined the Debian Press Team. What does it involve and how are you managing this new responsibility? Francesca: The Press Team is basically the armed wing of the Publicity Team: it handles announcements that need to be kept private until the release, moderate the debian-announce and debian-news mailing list and maintain contacts with press people from outside the project. The real job, so, is done within the Publicity Team. The most important part of our work is to write announcements and the newsletter: while the newsletter is published bi-weekly, the announcements need to be write in a shorter timeframe. Localization is really important in spreading Debian word, so we work closely with translators: both announcements and DPN are usually translated in four or five different languages. The publicity work could be stressful, as we have strict deadlines, we need to take quick decisions and often do last-minute changes. Personally, I like it: I work better under pressure. But I know that is sometimes difficult for contributors to accept that we can t debate endlessly on details, we have just to go on and do our best in a given timeframe.

The publicity work could be stressful, as we have strict deadlines, [ ]. Personally, I like it.

Raphael: You re one of the main editor behind the Debian Project News. What s the role and scope of this newsletter? Francesca: Debian Project News is our beloved newsletter, direct successor of the Debian Weekly News founded by Joey Hess in 1999 and later kept alive by Martin Schulze. In 2007, Debian Weekly News was discontinued but in 2008 the project was revived by Alexander Reichle Schmehl. The idea behind DPN is to provide our users an overview of what is happening inside and outside the project. As the core team of editors is formed by three people, the main problem is to be able to collect enough news from various sources: in this sense we are always glad when someone points us to interesting blogposts, mails and articles. DPN is also a good chance for non coders to contribute to Debian: propose news, write paragraphs and review the draft before the publication are quite easy tasks but very useful. English native speakers can do a proofread (as no one of the main editors is a native speaker) while others can always translate DPN in their native language. People who want to help us can take a look at our wiki page.

DPN is also a good chance for non coders to contribute to Debian.

Just yesterday I realized that since January we don t miss or delay an issue: so I d like to thank the fantastic team of editors, reviewers and translators who made it possible. The team is now working on another way of spreading Debian s message: a long-time project is finally becoming real. Stay tuned, surprise arriving! Raphael: You re trying to organize IRC training sessions but that doesn t seem to take off in Debian, while it s quite common in the Ubuntu community. How do you explain that? Francesca: I m not sure about it: both Debian users and contributors seemed to appreciate this initiative in the past. I was quite surprised by the amount of Debian members present during the various sessions and by the amount of interesting questions asked by the users. So the only reason I can think about is that I need to put more enthusiasm in convincing the teams to do it: they need more encouragement (or to be pestered more!). I, for myself, think that IRC training sessions are a great way to promote our work, to share our best practice, to talk about our project to a wider audience. And I ll sure try to organize more of them. Help, suggestions, ideas are really welcome! Raphael: If you could spend all your time on Debian, what would you work on? Francesca: There is a project I d like to give more love, but I always end up without the time to do it: the debian-community.org project. Back in 2007, Holger Levsen founded it with the aim of reducing the gap between Debian contributors and Debian users, giving all an opportunity to contribute, share ideas and more. The project was discontinued and I d really like to revive it: in these years various things have changed, but I think that the core idea of having a node to connect existing local communities is still good and doable. In Debian we don t have the wide and well articulated local infrastructure present in other distributions (Ubuntu, particularly, but also Fedora): even if I don t like too centralized structures, I think that a better connection between the project and local groups of users and on-line communities would be a step forward for the project. Being part of the Events Team, I m aware of how much we need to improve our communication with local groups. An example is the events organization: sometimes, Publicity and Events teams even don t know about regional Debian related events (like booth at conferences, workshops, talks, install parties, etc) and this is a shame because we could offer a lot of help in organizing and promoting local events. What we lack is better communication. And debian-community.org project could give us exactly this. Could be a cluster of local groups, a platform for events organization and even a useful resource for newbies who want to find a local group near them. I started some effort in this sense, sending a proposal about it, working on a census of Debian local groups. Any help is appreciated! I m really curious to see how many Debian communities (from all around the world and the web) are out there, and I d love to have members from these communities better connected with the Debian Project. Raphael: What s the biggest problem of Debian? Probably the bikeshedding feticism of almost all of us. It s the other side of the coin of Debian s commitment to technical excellence and our perfectionism, but sometimes it leads just to endless discussions about details, and it is a blocker for various initiatives. In Debian, you have to be really patient and in a way stubborn to push some changes. This is frustrating sometimes. On the other hand, I really appreciate how people take some times to think to each proposals, give some feedback and discuss about it: the process could be annoying, indeed, but the result is often an improvement of the initial proposal. Raphael: Is there someone in Debian that you admire for their contributions? Most of my teammates are simply brilliant and adorable and hard-working. But I have to admit that I particularly admire David Pr vot: beside being a webmaster he does a lot of things, from French translations to DPN editing. All his contributions have a great quality and he s able to push you always further in doing things and doing them better. He is a good example of how I d like to be as contributor: smart, tireless, friendly.

---

Thank you to Francesca for the time spent answering my questions. I hope you enjoyed reading her answers as I did. Note that older interviews are indexed on wiki.debian.org/PeopleBehindDebian.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook.

3 comments Liked this article? Click here. My blog is Flattr-enabled.

Since installing a whiteboard in our kitchen, conversations at the Acetarium have been moving in new and interesting directions. For example, Mika and I recently noticed that, when rotated correctly, the skeletal formula for 2,3-dimethyl-2-butanol looks pretty friendly!

I look after a lot of systems, and most of them want identical and simple backups taking of their filesystems. Currently I use backup2l which works but suffers from a couple of minor issues. In short I want to take a full filesystem backup (i.e. Backup "/"). I wish to only exclude a few directories and mounted filesystems. So my configuration looks like this:

# List of directories to make backups of.
# All paths MUST be absolute and start with a '/'!
SRCLIST=(  / /boot -xdev )
# The following expression specifies the files not to be archived.
SKIPCOND=( -path '/var/backups/localhost' -o -path '/var/run/' -o \
    -path '/tmp'  -o -path '/var/tmp' \
    -o -path '/dev' -o -path '/spam' \
    -o -path '/var/spool/' )

The only surprising thing here is that I abuse the internals of backup2l because I know that it uses "find" to build up a list of files - so I sneakily add int "-xdev" to the first argument. This means I don't accidentally backup any mounted gluster filesystem, mounted MySQL binary/log mounts, etc. backup2l then goes and does its jobs. It allows me to define things to run before and after the backup runs via code like this:

# This user-defined bash function is executed before a backup is made
PRE_BACKUP ()
 
   if [ -d /etc/backup2l/pre.d/ ]; then
      run-parts /etc/backup2l/pre.d/
   fi
 

So what is my gripe? Well I get a daily email, per-system, which shows lots of detail - but the key thing. The important thing. The thing I care about more than anything else, the actual "success" or "fail" result is only discoverable by reading the mail. If the backup fails, due to out of disk, I won't know unless I read the middle of the mail. If the pre/post-steps fail I won't know unless I examine the output. As I said to a colleague today in my view the success or failure of the backup is the combination of each of three distinct steps:

• pre-backup jobs.
• backup itself
• post-backup jobs.

If any of the three fail I want to know. If they succeed then ideally I don't want a mail at all - but if I get one it should have:

Subject: Backup Success - $(hostname) - $(date)

So I've looked around at programs such as backup-ninja, backup-manager and they seem similar. It is a shame as I mostly like backup2l, but in short I want to do the same thing on about 50-250 hosts:

• Dump mysql, optionally.
• Dump postgresql, optionally.
• Dump the filesystem. Incrementals are great, but full copies are probably tolerable.
• Rsync those local filesystem backups to a remote location.

In my case it is usually the rsync-step that fails. Which is horrific if you don't notice (quota exceeded. connection reset by peer. etc). The local backups are good enough for 95% of recovery times - but if the hardware is fried having the backups be available, albeit slowly, is required. Using GNU Tar incrementally is trivial. If it weren't such a messy program I'd probably be inclined to hack on backup2l - but in 2012 I can't believe I need to. (Yes, backuppc rocks. So does duplicity. So does amanda. But they're not appropriate here. Sadly.) ObQuote: "Oh, I get it. I see now. You've been training for two years to take me out, and now here I am. Whew! " - Blade II - An example of a rare breed, a sequel that doesn't suck. No pun intended.

I still get lot of questions why I wrote Weblate instead of staying with some existing solution. Most of the reasons were already addressed in previous posts, where I looked for Pootle alternative or announced Weblate, but still it might be worth to summarize reasons once more. Git integration None of existing tools integrate with distribute version control, while I find this integration somewhat obvious and necessary. Having translation system as one of remote branches makes things really easy to handle for developers. Merging translations is nothing more than git merge. Attribution to translator I think it's fair to request that translator gets appropriate attribution for his contribution. Once we rely on Git, it is quite obvious that he will be author for Git commit doing changes. This way you can find author purely with Git and don't need to look for other tools. Handling of several branches In phpMyAdmin we heavily use branches to maintain stable releases. On the other side we want to be able to push translations to both stable and testing branches without translators having to do the work twice. Weblate does automatically push changes to all branches. Well this is not limited to branches but to subprojects as they don't have to be only branches, they can be also just related projects like with Gammu and Wammu. Displaying context Most of tools I tried fail to provide way to show related source code even when Gettext po files contain such references. With Weblate this is just matter of configuring URL of viewer, what nicely integrates with existing services like Github. Sane merging of uploaded translations With Pootle, it is possible to upload incomplete po file and it will luckily take it and replace existing one. This is quite dangerous as you would not notice it is incomplete unless you do some translation validation before release. Weblate does merge uploaded translations unit by unit, what makes it slower, but generates minimal diff and works also across branches. Consistency checks There are always things which can go easily wrong in the translation like format strings and translation tool should catch these as early as possible. Weblate semi-enforces checks - if check has failed, you will be redirected back to previous translation to possibly correct it (or mark check for ignoring). At current state checks are not as good as I would like to, but they are completely modular and can be easily extended and improved. Bonus points While working on Weblate I realized that some things could be done even better than I originally anticipated. For example with Github web hooks, it is possible to receive notifications on upstream repository updates and merge changes as they appear, leaving to really live translation process. Git allows to write custom merge drivers and this can be utilized for less merge conflicts on po files (they are usually caused by timestamps in headers), so that's another handy you get with Weblate. Of course there is much more built in, just check the documentation.

Filed under: English Weblate 3 comments Flattr this!

Dunetna reloaded

Welcome to new dunetna website!!! After some years of low maintenance of this site, it's time to update it. It has more or less the same content as it had, but some new features have been added. For example, this blog. The main change is that dunetna website was a home-made application and now it's based in dokuwiki software. You can read more about this site at about page.

Welcome to new dunetna website!!! After some years of low maintenance of this site, it's time to update it. It has more or less the same content as it had, but some new features have been added. For example, this blog. The main change is that dunetna website was a home-made application and now it's based in dokuwiki software.

Some networks have no connection to the outside except that they allow surfing through an HTTP(S) proxy. Sometimes you are happy and the HTTPS port (443) is unrestricted. The following server-side tools allow you to exploit these weaknesses and get you a shell on your server. sslh sslh is an SSH/SSL multiplexor. If a client connects to sslh, it checks if the clients speaks the SSH or the SSL protocol and then passes the connection to the according real port of SSL or some SSL enabled service, e.g. an HTTPS, OpenVPN, Tinc or XMPP server. That way it s possible to connect to one of these services and SSH on the same port. The usual scenario where this daemon is useful are firewalls which block SSH, force HTTP to go through a proxy, but allow HTTPS connections without restriction. In that case you let sslh listen on the HTTPS port (443) and to move the real HTTPS server (e.g. Apache) to listen on either a different port number (e.g. 442, 444 or 8443) or on another IP address, e.g. on localhost, port 443. On an Debian or Ubuntu based Apache HTTPS server, you just have to do the following to run Apache on port 442 and sslh on port 443 instead:

1. apt-get install sslh as root.
2. Edit /etc/default/sslh, change RUN=no to RUN=yes and --ssl 127.0.0.1:443 to --ssl 127.0.0.1:442.
3. Edit /etc/apache2/ports.conf and all files in /etc/apache2/sites-available/ which contain a reference to port 443 (which is only /etc/apache2/sites-available/default-ssl.conf in the default configuration) and change all occurrences of 443 to 442.
4. service apache2 restart
5. service sslh start

Now you should be able to ssh to your server on port 443 (ssh -p 443 your.server.example.org) while still being able to surf to https://your.server.example.org/. sslh works as threaded or as preforking daemon, or via inetd. It also honors tcpwrapper configurations for sshd in /etc/hosts.allow and /etc/hosts.deny. sslh is available as port or package at least in Gentoo, in FreeBSD, in Debian and in Ubuntu. AjaxTerm A completely different approach takes AjaxTerm. It provides a terminal inside a web browser with login and ssh being its server-side backend. Properly safe-guarded by HTTPS plus maybe HTTP based authentication this can be an interesting emergency alternative to the more common but also more often blocked remote login mechanisms. AjaxTerm is available as package at least in Debian and in Ubuntu.

Happily I never were forced to use either of them myself. :-)

In the language introduced yesterday, mutable parameters play the important role of representing the side effects of actions in the type system and thus ensure referential transparency. One of the questions currently unaddressed is how to handle currying in the presence of mutable parameters. In order to visualise this problem, consider a function

    printLn(mutable IO, String line)

If we bind the the first parameter, what should be the type of the function, and especially important how do we get back the mutable parameter? Consider the partially applied form printLn1:

    printLn1(String line)

The mutability parameter would be lost and we could not do any further I/O (and the curried form would appear as a pure function, so a good compiler would not even emit a call to it) An answer might be to put the thing into the result when currying:

    printLn2(String line) -> mutable IO

But how can we explain this? In the end result, do we maybe have to use a signature like:

    printLn(String, mutable in IO io1, mutable out IO io2)

We could then introduce syntax to call that as

    printLn(string, mutable io)

Where as the mutable io argument basically expands to io and io1 for the first call, and for later calls to io1, io2 , and so on. It can also be easily curried by allowing currying to take place on variables not declared as output parameters. We can then curry the function as:

    printLn3(mutable in IO io1, mutable out IO io2)
    printLn4(mutable out IO io2)

If so, we can rename mutable back to unique and make that easier by introducing the unary operator & for two locations, just like Mercury uses ! for it. We could then write calls looking like this:

    printLn("Hello", &io);
    printLn("Hello", io, io1);

How out parameters are explained is a different topic; we could probably say that an out parameter defines a new variable. Another option is to forbid currying of mutable parameters entirely. This would have the advantage of maintaining the somewhat simple one parameter style. The programming language Clean does not provide any special syntactic sugar for having mutable variables. In Clean, the function gets a unique object and returns a unique object (noted by *). For example, the main entry point in a Clean program (with state) looks like this:

    Start:: *World -> *World

In short, the function Start gets a abstract world passed that is unique and at the end returns a new unique world. In Clean syntax, our example function would most likely have the signature:

    printLn :: String *IO -> *IO

You know have to either maintain one additional variable for the new unique object, which gets a bit complicated with time. On the other hand, you can do function composition on this (if you have a function composition operator that preserves uniqueness when available, as should be possible in Clean):

    printTwoLines :: *IO -> *IO
    printTwoLines = (printLn "World") o (printLn "Hello")

Function composition on mutable things however, does not seem like it is needed often enough in a functional programming language with a C syntax. People might also ask why monads are not used instead. Simon L Peyton Jones and Philip Wadler described monadic I/O in their paper Imperative Functional Programming (http://research.microsoft.com/en-us/um/people/simonpj/papers/imperative.ps.Z) in 1993, and it is used in Haskell (the paper was about the implementation in Haskell anyway), one of the world s most popular and successful functional programming languages. While monadic I/O works for the Haskell crowd, and surely some other people, the use of Monads also limits the expressiveness of code, at least as far as I can tell. At least as soon as you want to combine multiple monads, you will have to start lifting stuff from one monad to another (liftIO and friends), or perform all operations in the IO monad, which prevents obvious optimizations (such as parallelizing the creation of two arrays) in short dependencies between actions are more strict than they have to be. For a functional language targeting imperative programmers, the lifting part seems a bit too complicated. One of the big advantages of monads is that they are much easier to implement, as they do not require extensions to the type system and the Hindley-Milner type inference algorithm used by the cool functional programming languages. If you want uniqueness typing however, you need to modify the algorithm or infer the basic types first and then infer uniqueness in a second pass (as Clean seems to do it).
Filed under: General

Germany is a core area for the Debian Edu and Skolelinux user community, and this time I managed to get hold of Wolfgang Schweer, a valuable contributor to the project from Germany. Who are you, and how do you spend your days? I've studied Mathematics at the university 'Ruhr-Universit t' in Bochum, Germany. Since 1981 I'm working as a teacher at the school "Westfalen-Kolleg Dortmund", a second chance school. Here, young adults is given the opportunity to get further education in order to do the school examination 'Abitur', which will allow to study at a university. This second chance is of value for those who want a better job perspective or failed to get a higher school examination being teens. Besides teaching I was involved in developing online courses for a blended learning project called 'abitur-online.nrw' and in some other information technology related projects. For about ten years I've been teacher and coordinator for the 'abitur-online' project at my school. Being now in my early sixties, I've decided to leave school at the end of April this year. How did you get in contact with the Skolelinux/Debian Edu project? The first information about Skolelinux must have come to my attention years ago and somehow related to LTSP (Linux Terminal Server Project). At school, we had set up a network at the beginning of 1997 using Suse Linux on the desktop, replacing a Novell network. Since 2002, we used old machines from the city council of Dortmund as thin clients (LTSP, later Ubuntu/Lessdisks) cause new hardware was out of reach. At home I'm using Debian since years and - subscribed to the Debian news letter - heard from time to time about Skolelinux. About two years ago I proposed to replace the (somehow undocumented and only known to me) system at school by a well known Debian based system: Skolelinux. Students and teachers appreciated the new system because of a better look and feel and an enhanced access to local media on thin clients. The possibility to alter and/or reset passwords using a GUI was welcomed, too. Being able to do administrative tasks using a GUI and to easily set up workstations using PXE was of very high value for the admin teachers. What do you see as the advantages of Skolelinux/Debian Edu? It's open source, easy to set up, stable and flexible due to it's Debian base. It integrates LTSP out-of-the-box. And it is documented! So it was a perfect choice. Being open source, there are no license problems and so it's possible to point teachers and students to programs like OpenOffice.org, ViewYourMind (mind mapping) and The Gimp. It's of high value to be able to adapt parts of the system to special needs of a school and to choose where to get support for this. What do you see as the disadvantages of Skolelinux/Debian Edu? Nothing yet. Which free software do you use daily? At home (Debian Sid with Gnome Desktop): Iceweasel, LibreOffice, Mutt, Gedit, Document Viewer, Midnight Commander, flpsed (PDF Annotator). At school (Skolelinux Lenny): Iceweasel, Gedit, LibreOffice. Which strategy do you believe is the right one to use to get schools to use free software? Some time ago I thought it was enough to tell people about it. But that doesn't seem to work quite well. Now I concentrate on those more interested and hope to get multiplicators that way.

I have been putting a lot of testing into jffnms lately. I have been very lucky to have had someone with the time and patience to try out various sub versions and give me access to their results. The end-result of all this testing is a much, much less buggy JFFNMS. There have been a strack of problems with caching results, for example, where status would not be updated or even worse the status of one device impacted on another. The poller parent scheduler had a problem too where it would almost always sit in the first child starving the others of work which slowed things down. The scheduler now is a lot fairer across the children giving a speed up. I ve heard speed-ups of 15x for this one change alone. I also had a curious bug where if a device was set to not gather state it still did and created events but not alerts. This meant your event table was spammed with down interface alerts even on interface you know are down and you turned state checking off. 0.9.3 now does it the right way. The first RC is now uploaded and can be found at https://sourceforge.net/projects/jffnms/files/jffnms%20RC/ to try out. I m a little worried that the pollers now run too fast and could overwhelm the usually crummy control stack found in network devices for parsing SNMP. I m interested to hear how people find it.

My public radio station is engaged in a most obnoxious spring pledge drive. Good time to listen to podcasts. Here are the ones I'm currently liking.

• Free As In Freedom: The best informed podcast on software licensing issues, and highly idealistic. What keeps me coming back, though is that Karen and Bradley never quite agree on things, and always end up in some lawyerly minutia culdesac that is somehow interesting to listen to. They once did a whole show about a particular IRS tax form, and I listened to it all. (Granted, I often listen to this while cleaning house, but as Bradley would say, at least I'm not listening to it while driving.)
• This Developer's Life: At least the early episodes before it got popular are a unashamed imitation of This American Life, and I have quite enjoyed them. Although I often roll my eyes at the proprietary developer mindsets on display in the show. For example, often they'll have a bug and not root cause it, because well, they don't have the source code for the Windows layers. Still, beneath that it's mostly about the parts of software development that are common to all our lives. A particular episode I can recommend is #10 "Disconnecting" -- the first 20 minutes is a perfect story.
• Off the Hook: This is actually a live radio show, quite well done, with call-ins and everything. So much more polished than your typical podcast. It's hosted by Emmanuel Goldstein! And it's been going on for over 20 years, so why did I never hear about it before? Probably I'm not quite in the right hacker circles. Since it's out of NYC and very anti-authoritarian, I've mostly been enjoying it as a view into the Occupy protests.
• StarShipSofa: The best science fiction podcast around. Probably not news to anyone who ever looked for such a podcast. Long, and tends to be frontloaded with a lot of administrivia, which I fast-forward to get to the stories.
• Spider on the Web: The best music and science fiction podcast around. Mostly on hiatus since Jeanne died, but I hope Spider picks it back up. A good examplar is "Bianca's Hands"
• Long Now Seminars: Consistently interesting. I visited their space last time I was in SF only to learn they'd had a talk the night before, which would have been a bummer, except they ran the bits of the Clock for us.
• Linux Outlaws: After 18 years using Linux, I find the level of discourse in most Linux podcasts typically rather annoying. Including this one, but when Fab gets on a rant, it's all worth it. Sometimes some interesting guests.
• This Week In Debian: Sadly no new episodes lately, and I've been too lame to respond to repeated interview requests. Probably it needs to move away from being an interview show if it is to continue; there are only so many DD's who can give excellent interviews like liw did.