Computer security is generally defined as the following:

1. The data should remain available to those who need it.
2. The data should not be available to those who don t have the right to.

The critical software component to the first item is your well-known nightmare: the backup software. Backups are easy! In many corporate environments, here is how you ensure you have backups:

• you install the backup agent,
• you call/mail the backup guy, telling him the IP of the machine and what to backup,
• you wait for his call/mail telling you it s ok.

Isn t that cool? So little work to have all your data completely safe! Wait if you look more closely, there is something wrong here. Nowhere in this process did you specify credentials to the backup guy, nor did you add anything to identify the backup server. Or to say it otherwise, once you have installed the backup agent, the backup server has full access to your data. But you didn t even specify anywhere the IP of the backup server. So this means that anyone on the network has full access to your data. Which, while trying to make your data secure, actually brings a gaping security issue on your system. Security by nullity When you install a well-designed backup agent, like Bacula, you have to specify a password for each client and make this password known to the backup server. A simple authentication protocol ensures that only the backup server is able to backup your data. However, when you install, for example, the HP Data protector agent, it starts listening on a TCP socket and (no kidding!) binds it to a restricted shell which has access to a small list of commands. The backup server only needs to connect to this TCP socket and issue commands. While this has the great advantage of simplifying the development process of the backup server, such a software has a name: a rootkit. Several other proprietary backup software have different implementations, like RPC-based or proprietary protocols, but the basis remains the same: you connect to the TCP port, and you have a way to read absolutely any file on the system. Of course, there are so-called security options, that you can buy besides the disk agent or the nifty web interface. Yes, when you buy software that is critical for your security, the very lowest level of security that you d expect from any software not turning your box into a self-service is a paying option. In the end, what prevents your data from being available to anyone on the intarweb is the ultimate solution to everything(tm): the firewall. As people are not stupid enough to use the same backup system in the DMZ, you can t simply bounce from it after having used a hole in the lousy PHP code that is never updated. (Well, that will still work in many cases since people actually are stupid enough.) No, what makes it very fun is the backup network. The hackup network A tape library is expensive hardware, therefore you only want to buy one. However, when you have several departments or, in outsourced environments, several clients, you can t simply access to the backup agents on all these servers, because of all these evil routers and firewalls that are blocking you. The solution is trivial: use the second network card on all servers to connect to a backup network. The backup network is entirely managed by backup people who have no idea of network security, and is routed all the way down to the backup server. If you are lucky, some ACLs set up by the network guy who plugged the switch will prevent some parts of the network to access some others. In all cases, it s very likely that many people in the company (or from other companies!) have full IP access to all your private servers where sensitive data is stored. That includes access to your databases setup without a password, and of course to your highly secure backup agent. That generally also includes full IP access to the lousy maintained backup server, on which security updates are never installed. But don t worry. Your data is safe. Note: no, this is not a worst-case scenario. Such badly-designed software is widespread. Such practice is widespread in several companies I know.

I had a problem where the email address circsales@washpost.com spammed a Request Tracker (RT) [1] installation (one of the rules for running a vaction program is that you never respond twice to the same address, another rule is that you never respond to automatically generated messages). Deleting these tickets was not easy, the RT web interface only supports deleting 50 tickets at a time. To delete them I first had to find the account ID in RT, the following query does that:
select id from Users where EmailAddress='circsales@washpost.com'; Then to mark the tickets as deleted I ran the following SQL command (where X was the ID):
update Tickets set Status='deleted' where Creator=X; Finally to purge the deleted entries from the database (which was growing overly large) I used the RTx-Shredder [2] tool. RTx-Shredder doesn’t seem to support deleting tickets based on submitter, which is why I had to delete them first. I am currently using the following command to purge the tickets. The “limit,500” directive tells rtx-shredder to remove 500 tickets at one time (the default is to only remove 10 tickets).
./sbin/rtx-shredder --force --plugin 'Tickets=status,deleted;limit,500' There are currently over 34,000 deleted tickets to remove, and rtx-shredder is currently proceeding at a rate of 9 tickets per minute, so it seems that it will take almost three days of database activity to clear the tickets out. I also need to purge some tickets that have been resolved for a long time, I’m running the following command to remove them:
./sbin/rtx-shredder --force --plugin 'Tickets=status,resolved;updated_before,2008-03-01 01:01:34;limit,500' With both the rtx-shredder commands running at once I’m getting a rate of 15 tickets per minute, so it seems that the bottleneck is more related to rtx-shredder than MySQL (which is what I expected). Although with two copies running at once I have mysqld listed as taking about 190% of CPU (two CPUs running two capacity). The machine in question has two P4 CPUs with hyper-threading enabled, so maybe running two copies of rtx-shredder causes mysqld to become CPU bottlenecked. I’m not sure how to match up CPU use as reported via top to actual CPU power in a system with hyper-threading (the hyper-threaded virtual CPUs do not double the CPU power). I wonder if this means that the indexes on the RT tables are inadequate to the task. I tried adding the following indexes (as suggested in the rtx-shredder documentation), but it didn’t seem to do any good - it might have improved performance by 10% but that could be due to sampling error. CREATE INDEX SHREDDER_CGM1 ON CachedGroupMembers(MemberId, GroupId, Disabled);
CREATE INDEX SHREDDER_CGM2 ON CachedGroupMembers(ImmediateParentId, MemberId);
CREATE UNIQUE INDEX SHREDDER_GM1 ON GroupMembers(MemberId, GroupId);
CREATE INDEX SHREDDER_TXN1 ON Transactions(ReferenceType, OldReference);
CREATE INDEX SHREDDER_TXN2 ON Transactions(ReferenceType, NewReference);
CREATE INDEX SHREDDER_TXN3 ON Transactions(Type, OldValue);
CREATE INDEX SHREDDER_TXN4 ON Transactions(Type, NewValue);

• [1] http://bestpractical.com/rt


• [2] http://search.cpan.org/~ruz/RTx-Shredder-0.07/

Share This

I’ve been installing free software for library cataloguing onto Apples recently. It’s not been great (GNU/Linux is still easier and faster, in my opinion) but it has worked much better than older Macs. As well as making servers that are finally getting easier, Apple lead the way in locked-down portable players, with their iPod and iPhone devices and iTunes service. Quite rightly, free software users campaign against Apple’s use of DRM/TPM (Technical Protection Measures). One of the rallying points has been Defective by Design (DbD), an FSF initiative since 2006. Long-time readers may remember that I took part in a DbD protest at an Apple Store in September 2006:-

“I noticed the huge till queue, which was attractive for three reasons: it had no security (I guess shoplifters rarely queue for the till), most of them are standing around waiting for something to read and they are almost certainly Apple customers, so who better to inform? After I rejoined the back of the queue a few times, a security man was watching it, so I moved off and chatted to people waiting to do hands-on tests on the upper floor for a bit [...] then I moved back to leaflet the till queue again. [...] Then, I got a bit bored and started putting the leaflets into Apple’s product leaflet dispensers (in front of Apple’s - no leaflets were removed), which soon got me rumbled.”

To me, that was fine: I dislike that type of store, which tries to look like public space, but is actually privatised and controlled. My resource use was minimal (one security man escorting me out and some Saturday kid probably “cleaned” the leaflet racks) but I passed information to many Apple customers, who are one of only two groups Apple must keep happy, ultimately. Popey is a bit upset by the latest protest idea, which he describes as:-

“urging people to book slots at the Apple in-store “Genius Bar” to ask probing questions which they already know the answer to about their company policy on DRM and Free software. The ‘Genius Bar’ is an official Apple support avenue for their customers, and is a service provided inside many of their stores.”

It’s also been criticised by neuro and Pete along similar lines. I agree. It’s wasting both side’s resources. Apple don’t really need to keep their staff happy (they can hire new staff) and it offends Apple customers and shareholders. It’s not even efficient: each unit of Apple employee time wasted costs a unit of DbyD worker time and who’s got the most resources there? What are FSF thinking? It’s stupid, even without the bad press it’s had. So, this brings me in a roundabout way to answering

“I am aware of the Linux Foundation and the FSF and others, but how does SPI differentiate itself and its goals from those organizations?”

from why people don’t join SPI. In my opinion, a key difference of Software in the Public Interest is that it is managed by free software developers in a more-or-less democratic fashion. Even though I’ve failed in two elections, I still think it’s better than the alternatives. No project as unpopular as wasting Apple Store helpdesk time would ever get backed by SPI, would it?

I got back home from DebConf8 about 11pm last night, having left Mar del Plata at 11am on Sunday morning. 4 hours time difference makes for a 32 hour trip time covering minibus, 2 planes, a coach and a car, 3 time zones (I flew via Madrid) and 4 countries. Not fun, and I'm glad I booked today off to recover. However I'm also exceptionally glad I went to what's probably my best DebConf so far.

Marga and Mart n, along with the rest of the local team, did a fantastic job. The venue was great - I really like it when the talks and accomodation are in the same building. It helps convince me to roll out of bed to the first talk of the day after a late night playing Mao if I know I only have to walk upstairs to get to it! ;) Bonus points for the hotel being actually in the middle of the city so it was possible to go out and walk about during the breaks or in the evening. Special mention to zer0 (website looks down at present, but given that he gave his machine up to act as the schedule announce by the restaurant it should hopefully be back soonish) who arranged for the hotel to supply us with Antares beer, as well as taking us to their pub/restaurant several times.

Mark, Steve and Joerg did the usual great job of running the network. They were hampered by a dodgy 3com switch (don't think they'll be using one of those again), but wireless that coped with roaming between the ground, 1st, 3rd and 7th floors without dropping connections is pretty impressive. Phil Hands' VOIP setup was much appreciated too.

I've got a few notes about the talks I attended that I'll try to write up this week (it might help explain to my workmates just exactly why I went as well), as well as a few todo things I should really post so I actually do some of them. For now I'm going back to relaxing on my sofa and trying not to think about having to go to work tomorrow.

I haven't posted for a long while about ctioga, but that does not mean that development has stopped. It goes on, and features get added basically every single time I need them. Recent additions make it possible to tune a little more the look of the legends and add pure text lines in them. I've however realized that now, what ctioga is missing the most is real flexibility over the coordinates. For now, all coordinate positions are either relative to the frame (such as for --inset) or to the graph (for --draw). What I really wish is a way to mix all kind of coordinates: real size (1cm), frame position and graph position. That will require quite a fair deal of work, of course, but that shouldn't be too difficult... I'll go on posting about it here.

A large part of the disagreement about the way to manage the policy seems to be based on who will be the primary “owner” of the policy on the machine. This isn’t a problem that only applies to SE Linux, the same issue applies for various types of configuration files and scripts throughout the process of distribution development. Having a range of modules which can be considered configuration data that come from a single source seems to make SE Linux policy unique among other packages. The reasons for packaging all Apache modules in the main package seem a lot clearer. One idea that keeps cropping up is that as the policy is modular it should be included in daemon packages and the person maintaining the distribution package of the policy should maintain it. The reason for this request seems to usually be based on the idea that the person who packages a daemon for a distribution knows more about how it works than anyone else, I believe that this is false in most cases. When I started working on SE Linux I had a reasonable amount of experience in maintaining Debian packages of daemons and server processes, but I had to learn a lot about how things REALLY work to be able to write good policy. Also if we were to have policy modules included in the daemon packages, then those packages would need to be updated whenever there were serious changes to the SE Linux policy. For example Debian/Unstable flip-flopped on MCS support recently, changing the policy packages to re-enable MCS was enough pain, getting 50 daemon packages updated would have been unreasonably painful. Then of course there is the case where two daemons need to communicate, if the interface which is provided with one policy module has to be updated before another module can be updated and they are in separate packages then synchronised updates to two separate packages might be required for a single change to the upstream policy. I believe that the idea of having policy modules owned by the maintainers of the various daemon packages is not viable. I also believe that most people who package daemons would violently oppose the idea of having to package SE Linux policy if they realised what would be required of them. Caleb Case seems to believe that ownership of policy can either be based on the distribution developer or the local sys-admin with apparently little middle-ground [1]. In the section titled “The Evils of Single Policy Packages” he suggests that if an application is upgraded for a security fix, and that upgrade requires a new policy, then it requires a new policy for the entire system if all the policy is in the same package. However the way things currently work is that upgrading a Debian SE Linux policy package does not install any of the new modules. They are stored under /usr/share/selinux/default but the active modules are under /etc/selinux/default/modules/active. An example of just such an upgrade is the Debian Security Advisory DSA-1617-1 for the SE Linux policy for Etch to address the recent BIND issue [2]. In summary the new version of BIND didn’t work well with the SE Linux policy, so an update was released to fix it. When the updated SE Linux policy package is installed it will upgrade the bind.pp module if the previous version of the package was known to have the version of bind.pp that didn’t allow named to bind() to most UDP ports - the other policy modules are not touched. I think that this is great evidence to show that the way things currently work in Debian work well. For the hypothetical case where a user had made local modifications to the bind.pp policy module, they could simply put the policy package on hold - I think it’s safe to assume that anyone who cares about security will read the changelogs for all updates to released versions of Debian, so they would realise the need to do this. Part of Caleb’s argument rests on the supposed need for end users to modify policy packages (IE to build their own packages from modified source). I run many SE Linux machines, and since the release of the “modular” policy (which first appeared in Fedora Core 5, Debian/Etch, and Red Hat Enterprise Linux 5) I have never needed to make such a modification. I modify policy regularly for the benefit of Debian users and have a number of test machines to try it out. But for the machines where I am a sysadmin I just create a local module that permits the access that is needed. The only reason why someone would need to modify an existing module is to remove privileges or to change automatic domain transition rules. Changing automatic domain transitions is a serious change to the policy which is not something that a typical user would want to do - if they were to do such things then they would probably grab the policy source and rebuild all the policy packages. Removing privileges is not something that a typical sysadmin desires, the reference policy is reasonably strict and users generally don’t look for ways to tighten up the policy. In almost all cases it seems best to consider that the policy modules which are shipped by the distribution are owned by the distribution not the sysadmin. The sysadmin will decide which policy modules to load, what roles and levels to assign to users with the semanage tool, and what local additions to add to the policy. For the CentOS systems I run I use the Red Hat policy, I don’t believe that there is a benefit for me to change the policy that Red Hat ships, and I think that for people who have less knowledge about SE Linux policy than me there are more reasons not to change such policy and less reasons to do so. Finally Caleb provides a suggestion for managing policy modules by having sym-links to the modules that you desire. Of course there is nothing preventing the existence of a postfix.pp file on the system provided by a package while there is a local postfix.pp file which is the target of the sym-link (so the sym-link idea does not support the idea of having multiple policy packages). With the way that policy modules can be loaded from any location, the only need for sym-links is if you want to have an automatic upgrade script that can be overridden for some modules. I have no objection to adding such a feature to the Debian policy packages if someone sends me a patch. Caleb also failed to discuss how policy would be initially loaded if packaged on a per-module basis. If for example I had a package selinux-policy-default-postfix which contains the file postfix.pp, how would this package get installed? I am not aware of the Debian package dependencies (or those of any other distribution) being about to represent that the postfix package depends on selinux-policy-default-postfix if and only if the selinux-policy-default package is installed. Please note that I am not suggesting that we add support for such things, a package management system that can solve Sudoku based on package dependency rules is not something that I think would be useful or worth having. As I noted in my previous post about how to package SE Linux policy for distributions [3] the current Debian policy packages have code in the postinst (which I believe originated with Erich Schubert) to load policy modules that match the Debian packages on the system. This means that initially setting up the policy merely requires installing the selinux-policy-default package and rebooting. I am inclined to reject any proposed change which makes the initial install of of the policy more difficult than this. After Debian/Lenny is released I plan to make some changes to the policy. One thing that I want to do is to have a Debconf option to allow users to choose to automatically upgrade their running policy whenever they upgrade the Debian policy package, this would probably only apply to changes within one release (IE it wouldn’t cause an automatic upgrade from Lenny+1 policy to Lenny+2). Another thing I would like to do is to have the policy modules which are currently copied to /etc/selinux/default/modules/active instead be hard linked when the source is a system directory. That would save about 12M of disk space on some of my systems. I’ve taken the unusual step of writing two blog posts in response to Caleb’s post not because I want to criticise him (he has done a lot of good work), but because he is important in the SE Linux community and his post deserves the two hours I have spent writing responses to it. While writing these posts I have noticed a number of issues that can be improved, I invite suggestions from Caleb and others on how to make such improvements.

• [1] http://www.calebcase.com/node/2


• [2] http://lists.debian.org/debian-security-announce/2008/msg00201.html


• [3] http://etbe.coker.com.au/2008/08/18/se-linux-policy-packaging-distribution/

Share This

What I liked

• Excellent organization, orga team fully deserved all the praise they got.
• Meeting people - it was the first DebConf I attended in person, so it was a pleasure to meet lots of folks whom I’ve only known online before.
• Quite a few interesting talks (videos are available, or will be available shortly, in DebConf8 video archive). Particularly recommended: quality assurance talks/BOFs by Lucas Nussbaum, BTS development progress by Don Armstrong, method diffusion in large volunteer projects by Martin Krafft, talk and debate on OpenSSL security issue by Luciano Bello, various lightning talks.
• Deserving a special mention, a talk by Mark Shuttleworth on Debian and Ubuntu relationship. I thought that he said all the right words and, judging by reactions, attitude towards Ubuntu has changed significantly during the last year, turning from hostility to peaceful cooperation.
• Argentinian food: steaks, empanadas, alfajores.

What I did not like

• People pay much less attention to fixing RC bugs than they used to, the two BSPs organized during DebConf8 attracted only 3-5 people.
• Some nice ideas being thrown around without anyone volunteering to work on them. Not really surprising, given that most of the core teams are short on manpower.
• Almost complete lack of events targeted at engaging users and beginner contributors.
• Quite a few specialized talks, interesting (IMHO) to only a minor percentage of the attendees.
• Lots of discussions with few or no specific decisions or action items.
• Pretty long trips to and from Mar del Plata (about 30 hours door-to-door travel time for me), luggage was lost for 2 days on the way there.

What I did

• Attended about 75% of talks, tried to contribute to discussion in constructive, non-flamy ways .
• Participated in BSPs, contributed debugging information, patches or fixes for about a dozen of release critical bugs, started triaging sparc-specific bugs.
• Started working on updating the Debian kernel handbook for Lenny, about 1/3 done.
• Gave a “Google and Open Source” lightning talk.

15 years ago today, Ian Murdock announced the Debian Linux Release. I've been involved in Debian since 2002 and am amazed at how the project has grown during my involvement, let alone its entire history. Here's to continued success for the Universal Operating System!

Me and Ail arrived at Buenos Aires on saturday. A member of the local team was waiting for people at Ezeiza, and we gave away our 2130 bus reservations in order to take a combi to Mar del Plata with Clint, four fellow german debianites and some venezuelans. The trip was quite long, around six hours IIRC. We arrived in the early morning, and slept past noon. I have no words to describe my particular experience with this year’s conference, but I do agree with our DPL: this has been the best DebConf I’ve ever attended. In general, the Annual Debian Conference is one of the most attractive events for me, since it provides us with the opportunity to share and define a little bit of the future of the universal operating system. I use, develop for and work with this operating system, so there’s the importance of this conference for me. That said, DebConf is a unique event. Some people that I know back in Venezuela are used to a different, more academic style of event, which features a tight schedule and no specific provision of group networking. They are not used to an event like DebConf, and when they attend it they feel they’ve lost their time in an event they don’t understand. And they ask for a participation certificate :) In that respect, besides the very interesting talks and events in the conference, DebConf offers an excellent opportunity for group development and general networking. On the other hand, both the local team and the venue have striked me in many good ways. I’m aware of LatAm limitations for some of the particularities of DebConf such as connectivity, provision for special food needs, special usage of the venue and general receptivity for a bunch of 24/7 geeks. The local team has made a huge effort putting this together in such a special and organized way. I don’t think the Venezuelan DC8 Bid would have made this much success in the same timeframe. Congratulations, Argentina! We will leave Mar del Plata on saturday (at 2200) and then we will spend some more days in Buenos Aires, particularly to attend Debian Day, where I’ll be giving a talk in spanish. Therefore, we’ll be writing a little more (in spanish) about Argentina later, since we loved our time in this country.

This keynote reports on the current state of collaboration between Debian and Ubuntu, reports back on progress made since previous DebConf's, and looks forward to new opportunities for collaboration and development.

Debian's derivatives are a powerful part of the Debian family. They bring Debian technology and, usually, Debian values to a broader audience than Debian itself will reach. At the moment, Ubuntu is the derivative with the largest reach, bringing millions of new users to GNU/Linux and Debian specifically. Ubuntu has shifted the commercial Linux landscape from one focused on RPM-based distributions to one where DEB-based distributions have much greater commercial and consumer acceptance. In this keynote, the leader of the Ubuntu project will discuss the state of current collaboration between Debian and Ubuntu developers, articulate the roadmap of Ubuntu in the coming years, and explore ways in which Ubuntu's work can improve the state of Debian.

The talk will cover:

* Current areas of collaboration
* Toolchain, Java, Python, Debian Installer,
* OpenOffice, X.org
* GNOME and KDE
* Security
* Roadmaps and transitions

* Challenges and obstacles to collaboration
* Cultural and organisational differences
* Divergent goals and priorities
* Release management and coordination

* Frameworks for collaboration
* Patch management and submission processes
* Package review, merging and delta management
* Version control interchange
* Leadership meetings and dialogue
* Bug coordination and collaboration
* Translation coordination and management

Ubuntu is only one of many derivatives - and others are achieving tremendous success in their own ways, too. The eeePC ships a derivative of Debian, and Debian is at the heart of many excellent free software solutions. The goal of this talk will be to celebrate some of those achievements and also to look for ways to make the "family" feel more cohesive. In short, to ensure that innovation which happens in derivatives has a natural pathway back to Debian. If the leaders of other derivatives are present at DebConf, this would also be an opportunity to seek comment from them, perhaps in the form of a panel for Q&A at the end of the keynote.
Full event details

This talk will briefly explain what Debian Edu is, how we develop our distribution, how we differ from Debian, how we work on bridging the gaps that still exist and what our plans for the future are.

In July 2007 the Debian Edu project released its third release, codename "Skolelinux 3.0 Terra". Debian Edu is probably the most widely known and most widely used Custom Debian Distribution - at least if you follow the official criteria for a CDD, which we aim to finally fulfill 100% for Lenny: all packages are taken unmodified from Debian main.

This talk will explain the development process of Debian Edu and the various differences to a default Debian install. An overview of the planned features for Lenny will be given as well as a list of current showstoppers will be presented. To explain to the audience, what challenges have been overcome since Debian Edu was started, the history of how 100% inclusion in Debian was achieved, will also be explained. (Debian Edu used to be a seperate distribution! Now we have become a part of Debian!)

Using wiki.debian.org to write a fine manual will be explained briefly too: our documentation is written in a wiki, then exported as docbook and translated using .po files, then transfered back into docbook, to generate html and PDFs versions, which are made available in the debian-edu-doc package, which is installed on a webserver again.

In the end, releasing with Lenny will not be the end of the process of becoming part of Debian. Currently it looks like we might release with Lenny, but then do point releases on our own. Whether this is a sensible strategy or if there are others, will (hopefully) become clearer in future. Until & after the talk - it's a long and winding road!Full event details

One goal of Custom Debian Distributions is to form a group of Debian
developers who care for a specific set of packages that are used in
the day to day work of a certain user group. The fact that Debian has
grown to the largest pool of ready to install packages on the net has
led to the side effect that it is hard to maintain for
beginners. A Custom Debian Distribution adds some substructure
to the currently flat pool of 15000+X packages without a real
structure. These substructures are intended to put a focus on
special user interest. These substructures are not oriented on
technical matters like Debian installer team, porting teams or teams
that are focussing to implement programming language policies.

There are some similarities to Debian-i18n which also has the pure
goal to serve the needs of certain end user groups with the difference
that the users are grouped not according to their field of work but
according to their language. In fact it makes even sense to create
CDDs for languages that require certain technical means to optimally
support the language regarding direction of writing, special fonts
etc. It is known that some countries in Asia builded Debian
derivatives for this purpose but in principle it is not necessary to
derive - the better solution is to make Debian more flexible by
starting a CDD effort inside Debian.

The talks will give some examples from the success of CDDs like
Debian-Edu and Debian-Med. One very important outcome of the CDD
effort is the ongoing reunification of Linex - the Debian derived
distribution that is used in all schools in Extremadura - with
Debian-Edu. This step means that Debian gets a very large
implementation in all schools of Extremadura while on the other hand
the effort of development for the people who invented Linex will be
reduced. Debian featuring Debian-Edu now has a very good chance to
become a really good international school distribution because it has
roots in five countries (Norway, Spain, France, Germany and Japan) and
might become attractive for many more.

The success stories of CDDs would not have been possible outside
Debian and thus leaving the path to build Zillions of Debian
derivatives that reach a very small user base and working together
inside Debian is the main idea of the talk. To make this idea more
attractive in the second part of the talk a description of tools that
were developed in the CDD effort will be presented. Especially the
newly developed web tools that give a good overview about the
packages that are useful for a certain field of work and the QA tools
that enable the CDD team members to easily get an overview about
packages that need some action. So if people are not yet convinced
that a CDD for their purpose makes sense we will catch them by the
tools they might get for free if they follow the proposed strategy.
Full event details

The Debian-Med CDD was started in January 2002 and has evolved a lot
since this time. Debian now becomes the distribution of choice for
people who are working in the field of medicine. On the one hand this
can be obtained from the increase of pure numbers of packages in the
field of medicine.

On the other hand - which is even more interesting for general Debian
development - a group was formed that has developed a certain way of
cooperation which might serve as an example for other teams. Debian
knows a lot of technical teams focussing onto technical issues
(pkg-perl, pkg-python, d-i, etc.). An important part of the project is
to involve people who call themselves "pure users" or "non-developers"
(whatever this might mean) as it is done at a certain amount by the
i18n and Debian Edu team.

So this talk is not about boring the audience by mentioning medical
Free Software - it exists at a certain amount out there as other
specific software does as well - but rather how to attract people to a
topic where FLOSS is not common but should. It will give examples for
our way to advertise Free Software in medicine which is heavily
dominated by proprietary software, which tools we developed inside the
CDD framework to be always up to date with our web pages that are
intended to attract users to Debian and last but not least the lessons
we have learned in the project that might be of high interest for
other CDDs (or potential CDDs). Examples are given for project
management, attracting people by working techniques and making sure
that medical software finds a good home in Debian.
Full event details

Statically detecting FTBFS with edos-builddebcheck Here at DebConf8, after my talk about Mancoosi, I got in touch with several people interested in the QA implication of the EDOS/Mancoosi tools. Me and Ralf have for example been contacted by Ubuntu people which are setting up theirs equivalent of http://edos.debian.net. A guy working on a mirroring tool made an interesting hack using edos-debcheck: out of a mirror with broken dependencies he can mirrors a subset of packages which do not suffer of the same problem. But now to the more interesting development, following a bug report by Atomo and discussions with Lucas (the RC-bug-filing-master), me and Ralf implemented edos-builddebcheck. You can invoke it on a pair of Packages/Sources package list, and it will check whether some packages will FTBFS due to unsatisfiable dependencies. The results on Lenny main/amd64 as of today are, ... erm ..., scary : 49 packages will FTBFS. The RC-bug-filing-master and Ralf are working on filing them, "don't worry". You can find edos-builddebcheck in the edos-debcheck package, starting from version 1.0-7 (in incoming at the time of writing). Update yes, ATM edos-builddebcheck does not know anything about P-A-S packages, so there are some false positive. Still, 33 source packages are proper hits.

Some time ago, when I packaged treeline, I wanted to make the package configure the environment so that .trl files would be automatically open with the program. That meant that the package had to configure the MIME types in the desktop to handle that, and assign the new MIME type the selected program. I tried a dozen of things at the time, based on a lot of howto’s that I found, without any success, so I decided to forget about it. This week I started packaging a new program: L ve, a 2D game development framework based in Lua and OpenGL. Many of the games developed for this platform are distributed as a zip file with the extension .love, so I decided I wanted to try again the MIME stuff. After many unsuccessful tries, I found a package that gave me some hints on how to do it: chemical-mime-data. After having a look at it, I was able to prepare a proof of concept that did what I wanted. It is supposed to work, at least, in GNOME, KDE and ROX. In case someone else is interested, here is the code:

• mime-example_0.1-1.dsc
• mime-example_0.1-1.diff.gz
• mime-example_0.1.orig.tar.gz

po4a - or PO for anything. PO is the de facto standard for translations and po4a tries to help maintainers prepare packages for translation.

1. I18n (internationalisation) is making a package accessible for translators and is an upstream/maintainer action. i18n is done once for all translation support and updated when the content changes (typically during the build).


2. l10n (localisation) is using i18n support to create an individual translation for a specific locale. l10n is done many times per package, with individual translations being updated each time the i18n content changes. (Tdebs will support this area but are a separate topic from po4a.) PO files that exist when the POT is updated are also updated, tagging modified strings as "fuzzy".

The natural burden, therefore, falls within l10n but without i18n, l10n is all but impossible (it means having to do the i18n "outside" the package).
I'm using po4a and I've been doing various translation-support stuff now for a few years, so I'm going to add po4a into my blog entries as a series to help maintainers make their own packages i18n aware.
This is particularly important for:

1. all Debian native packages, particularly scripted packages. My example will be based on perlpod documentation often embedded wthin the perl scripts themselves as well as DocBook XML for manpage generation. po4a supports many other formats.


2. All manpages (or other documentation) created by Debian maintainers

To make things easier, I'll be creating a new category in my blog ("i18n") for such entries. This is helpful for me too because the first time I used po4a (current versions of emdebian-tools) I basically got things wrong and made more work for myself than was necessary. This entry primarily concerns providing i18n support from manpage sources and perl scripts. i18n support in compiled packages is a separate issue. The fixes described here will be included in the next version of emdebian-tools (which will be uploaded to Emdebian but not into Debian until after Lenny).
Terminology

• PO - (Portable Object) files contain the POSIX C locale strings abstracted from the "master document" which in many cases is the source file in the relevant package(s), alongside the translated version of that string (if any). Multiple master documents are typically combined into a single collection of translatable strings. PO files are translated into individual locales, one PO file per locale.


• POT - PO Template - the definitive collection of translatable strings from which all PO files are prepared and updated. The POT file is never translated directly, it is maintained, updated and packaged by the maintainer.


• gettext - the binary application that handles the translatable strings. gettext abstracts the strings out of the source files during the package build and inserts the translated strings back into the text actually displayed to the user at runtime. If no suitable PO file exists, the user gets the untranslated string, exactly as it appears in the POT file. If a suitable PO file does exist but the strings have been modified since translation, gettext drops "fuzzy" translations that do not match precisely and the user gets a "mixed" translation containing some translated strings and some untranslated. Most translators try to maintain a 100% translation because the ugliness of the "mixed" output is a partial incentive to keep up with the updates.

Using po4a
po4a needs a config file where the maintainer keeps all metadata relating to the master documents and the relevant formats. po4a then converts the master documents into a collection of strings for translation and puts those strings into a POT file (a PO template file). The maintainer packages the POT file (and optionally sends the POT file to relevant mailing lists seeking translations to be returned as PO files) and keeps the POT file up to date during the build process. With a config file (e.g. called po4a.config), all the maintainer needs to do is call:

po4a po4a.config

po4a then parses the list of files defined in the config file, generates the lists of strings, collates the lists and identifies where the same string is used in multiple files and writes out a POT file containing references to where the strings are used and the unique strings.
po4a config file
Comments start with '#' in the usual way. Instructions are enclosed in square brackets.
po4a_paths defines where the POT file will be put (it can be extended later but I'm keeping this simple). The name of the POT file is important for compiled packages but can just be the name of the source package for scripted packages and/or manpages.

[po4a_paths] po/emdebian-tools.pot

type identifies *each* master document to be processed by po4a. The syntax is well suited to being generated from some other list within the package but I'll give just a snippet of the final output here:

[type: xml] doc/xml/em_autobuild.1.xml

[type: xml] doc/xml/embug.1.xml

[type: pod] emtargetcmp

[type: pod] emdebcheck

That's it. The complexity of my current script was entirely unnecessary.

Also, po4a is more than capable of handling multiple document types within the same source package and the same POT file which is something that I had missed so far. One more hack that can be removed.

All I do now is use xmllint to "clean-up" the DocBook XML and pod2html and some XSLT to generate HTML documentation for the website. The loops to do those tasks already contain the list of files to be processed by po4a, so a few echo commands is all that is needed to create po4a.config at build time and run po4a to generate a combined POT file for all strings in the package.

#!/bin/sh

CONFIG="po4a.config"

echo "[po4a_paths] po/emdebian-tools.pot" > $CONFIG

for file in  ls doc/xml/*.xml ; do

	xmllint --format "$file" --output "$file"

	echo "[type:xml] $file" >> $CONFIG

done

for file in emdebcheck emtargetcmp emprunecross Emdebian/Tools.pm \

 em_installtdeb emrecent emcache; do

	if [ "$file" = "Emdebian/Tools.pm" ]; then

		pod2html -outfile html/EmdebianTools.html -title $file  ../$file

	else

		pod2html -outfile html/$file.html -title $file  ../$file

	fi

	if [ "$file" = "Emdebian/Tools.pm" ]; then

		pod2man ../$file > man/Emdebian::Tools.3

	else

		pod2man ../$file > man/$file.1

	fi

	echo "[type: pod] $file" >> $CONFIG

done

po4a $CONFIG

Nameservers are part of the basic infrastructure of the internet, translating names like www.software.coop into numbers like 86.7.8.9. You really don’t want them to be completely unreachable because it breaks lots of things, so as well as the primary nameserver, it’s good to have secondary ones, both in the same hosting facility (for fast replacement if the primary fails) and a different city (for fallback if the hosting facility or its network fails). Since moving one of our customer’s secondary nameservers to an OpenVZ virtual server in another city last year, it’s been having serious stability problems. That server ran MyDNS as its main brain, but the primary servers are an unholy mishmash of BIND, MyDNS and I don’t know what else (they’re not all ours), so domain updates are done by periodic AXFR requests. Every so often, mydns would quit with an “out of memory” error, which was narrowed down to a few possibilities on the mailing list. I tried pretty much everything that was suggested (mostly off-list), but we still didn’t have a stable nameserver. Since then, MyDNS-NG has taken up development and Howard Wilkinson has worked to cure some of the memory leaks. I’ve upgraded to 1.2.8.3 and this time, the nameserver is still running so far. The out of memory problem was random, so I’ll leave it a few more days before sending reports and thank-yous upstream, but I’m pretty damn happy about this.

Hi folks! We’re almost done for the official Summer of Code program and I thought that I couldn’t let it end without another update, so here we are (you may thank Daniel Burrows for additional poking). The Gtk+ interface for Aptitude is making great progress. The product is not final yet but already implements many of the planned ideas, with others to come. Here are the screenies: So, let’s see the tasks presented in my previous update:

• A real user interface <- much progress here
• Being reliable <- check! we don’t crash all the time anymore
• Packages dependencies, pre-dependencies, etc. <- check!
• Hyperlinks between packages <- check!
• Advanced package search <- check!
• Tags support (tag clouds ?) <- well, no clouds yet, but we have tags
• Linking with interesting data sources like popcon <- not done yet
• A lot of interesting things <- well..

In some more details: we are now much closer to the final interface. Some things are not done yet, for example, notification bars la Iceweasel (or is it Firefox ?) are in the works for unobtrusive notifications about important things and a little guidance. We are much more reliable now. We eliminated lots of memory management problems that resulted from, mmh, imperfect coding discipline. We also lock the interface when important backend stuff reload to prevent the user from shooting himself in the foot when triggering an install when the APT cache is reloading for example.. The backend refactoring took a sizable part of this second term but it was worth it. The code is now much cleaner and uses no more (at least much fewer) ugly unscalable hacks. One very important feature of Aptitude is the ability to easily display all kinds of dependencies and navigate through them by hyperlinking. It’s now working in the Gtk+ interface as well. Feel free to click on whatever looks like a package or a package version. We now support the powerful Aptitude package searching syntax. Daniel Burrows is talking with Enrico Zini to bring the power of apt-xapian-index to Aptitude: autocompletion! lightning fast full-text searches! tag clouds! ponies! There is still a lot to be done. I’m not going away after the GSoC program. I’m here to stay, for Aptitude and Debian. Daniel Burrows has been very helpful and supportive. The Debian community is as awesome as I knew it before participating in the program. Daniel Burrows is planning to merge the Gtk+ interface into the main branch in September and packages may appear into Experimental. With the very interesting evolutions of the upcoming Adept-3, Lenny+1 will show very interesting changes in the area of packages managers. Now for you. I need you to test my interface. All necessary instructions are on the development trac : http://dev.graffit.net/aptitude/trac.

[ Taken and modified from a post to the hecl-devel mailing list ] I've been talking with a company about a consulting job regarding a programming language for mobile apps. Sounds familiar, right? There is, however, a bit of discussion regarding the syntax - my contact at said company maintains that Hecl's syntax is unfamiliar to many programmers, and I think on the face of it, I'd have to agree. Of course, Hecl's syntax is like it is because when I started the project, being very small was a necessity, and a very simple syntax facilitated that goal nicely. Additionally, the syntax allows for extreme flexibility, letting you create new control commands, and "DSL's" (domain specific languages) easily. Also, sticking with small/simple means that the interpreter and parser can both reside on the phone, instead of doing some kind of solution where the development/host computer compiles the scripting language to byte code, and the interpreter on the phone runs the byte code. That sort of strategy has some advantages, but has disadvantages too: if you don't have an on-board parser, you don't get "eval".

---

A quick Hecl syntax refresher: In Hecl, everything is a command, seperated by spaces, and grouped with :

if   ...     ...  

Is a command that takes two arguments. It evaluates the first one, and if it's true, evaluates the second one. This is different from languages like C, Python or Ruby, where 'if' is part of the language. For those unfamiliar with languages like Lisp, where this looks strangest is probably in conditions or other expressions:

if   = $foo 10     puts "foo is ten"  

Or

set foo [+ [* $bar 10] [- $beebop $doowah]]

Which is certainly a different take on things than most 'mainstream' languages. That radical simplification is, however, one of the ways that I used to save space in the interpreter.

---

Personally, I'm pretty happy with how Hecl looks, but I am sensitive to the 'marketing' argument: popularity helps, and this isn't the first time I've heard complaints about Hecl looking "funny" to people. What do you think? A) "I like Hecl just the way it is and wouldn't want to change it!" B) "Hecl is quite useful, but I just want a mobile scripting language, the syntax doesn't matter much." C) "I think the syntax is ugly, but I use it because I need an interpreter." Also related: 1) "I don't care too much about code size. My current apps target N kb as an acceptable size." 2) "Small is really important. The smaller the better." And: I) "I don't care about eval, or being able to run human readable code on the phone." II) "I really like the fact that you can download or even write code directly on the phone." And finally: X) "I don't care about midp1.0" Y) "The fact that Hecl runs on midp1.0/cldc1.0 is important to me." Now, I am not going to run off and change Hecl! Code that currently runs ought to continue to run, modulo some tweaks here and there, api changes, and the normal stuff. If I do something 'new', it would probably be a clean break (although of course code that could be reused would be), and will be in addition to Hecl, rather than instead of Hecl, because I would be working on the 'new' language as a day job. I do that with Hecl too, but not all the time, as I often have consulting/contracting work.