Hello To check the validity of Debian dependencies in Debian package, Config::Model queries a remote web server to get the list of package version known to Debian. The first version of this check did sequential requests: when the cache was not very fresh, I had to wait more than 60s before getting results for complex package like padre. That was very frustrating (but less frustrating than checking package names manually) For the following version, I addedhacked AnyEvent in Dpkg model to run parrallel queries. This went much faster but gaves weird results: before getting a response from Debian server, the packages were flagged as unknown. To get consistent results, running twice cme check dpkg was required. So, at new year, I ve decided to bite the bullet and implement correctly value checks with asynchronous queries to remote server. This new feature is now ready and will be delivered in Config::Model 2.030. cme will now return consistent results. This new release is mostly backward compatible. You may notice some quirks with some other modules based on Config::Model:

• Current Config::Model::Dpkg will returns unconsistent results as usual
• Tk UI will not show correct package status (i.e. packages may be wrongly shown as unknown)
• config-model-edit process must be killed if it refuses to exit.
• Tk UI non-regression tests will block

These quirks will disappear once these modules are updated. This should not be long since all the updates are ready in github. All the best
Tagged: Config::Model, debian, dpkg, Perl

The Standalone Sysadmin asks, How do you manage database server instances? :

Do you have one (or a few) centralized database servers, either standalone or clustered, or do you spread the load like we are currently?

His argument for centralisation is one of easing the management burden of configuration and backups, whereas the distributed approach eliminates a central point of failure and performance degradation. I go for distributed, all the way. For a start, we run so many databases for so many customers that there s no way on earth we could stand up a small number of database servers and handle all the load (hell, we ve got single customers who consume a cluster of machines with 384GB of RAM and all the SSDs you can eat). Security and permissions is a whole other kettle of fish; the contortions we d have to do to allow customers the level of management they need with a centralised database system would be immense. Then there s the need of some customers for MySQL, some for PgSQL, different performance tuning for different workloads nope, centralised DBs don t work for us. Given this, we ve bitten the bullet and solved pretty much all of the management problems. Installation and configuration is all handled via Puppet, and backups are trivial the same system that installs the DB server itself also drops a hook script that the backup agent uses to know that it has to dump a database server. Monitoring that this backup is taking place successfully is also automatically provisioned, so we know that we re not missing anything. Ultimately, this same approach applies to practically anything that you re tossing up between centralised and distributed. At scale, you can never rely on centralisation, so you may as well bite the bullet and learn how to do it distributed pretty much from the start. That saves some serious system shock when you discover what your hardware vendor wants for the next step up in big iron hardware

The following is what I usually do when I want to debianize a Haskell stuff from Hackage. This is a personal recipe, by no means official procedure or such. I may be wrong, badly at that.

1. Check pkg-haskell repository whether your intended package already exists or not. Somebody might work on it already.
2. Find the webpage of your intended package at HackageDB, then download Cabal source package . Untar it. The tarball filename should be changed to fit the Debian source package convention. For example, if the original name is something like foobar-1.0.tar.gz, then it should be haskell-foobar_1.0.orig.tar.gz.
3. File ITP. reportbug is your friend.
4. Run cabal-debian in untared dir:

    $ cabal-debian --debianize --quilt
   

   We do have the upstream tarball, so use quilt. See http://wiki.debian.org/UsingQuilt. It might give you warnings about changelog. debian/changelog doesn t exist yet, so ignore it.
5. Now read http://wiki.debian.org/Haskell/CollabMaint/Processes and http://wiki.debian.org/Haskell/CollabMaint/PackageTemplate carefully. Then do things as they say. These two are short, but well-written instructions. Edit the generated debian/control appropriately. Do not forget to make your debian/copyright machine-friendly. See http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/.
6. Test whether your package is flawless and really buildable. Do lintian. Use pbuilder.

    $ lintian haskell-foobar_1.0-1_amd64.changes
    # pbuilder --build haskell-foobar_1.0-1.dsc
   

7. Read http://wiki.debian.org/Haskell/CollabMaint/DarcsBasic. Darcs is quite easy to use, and you don t have to be the master of Darcs to merely debianize things anyway. Basically, what I usually do is

    $ darcs record -a; darcs push --repo=debian
   

   Darcs is well-suited to Haskell-related development, but seems you can use Git, too.
8. Change debian/changelog. UNRELEASED in the first line should be unstable or experimental . Do not forget to close ITP.
9. Finally, do

    $ debuild -tc; debcommit -r; debrelease; darcs push --repo=debian
   

   That s all folks!

Also this year the Debian project was present at Chemnitzer Linuxtage, this time right next the debianforum.de booth. The booth folks arriving on friday organized a flashmob at Expitas after booth setup. Unfortunatly our second planned flashmob at the mensa was boycotted by much more students, so we ended up in the Turm-Brauhaus, which is a great location with good drinks but the service was very harshly. On the next two days at the booth we chatted and discussed with visitors and other exhibitors a wide variety of questions, including 'When will be (the next Debian version) released?' and 'Are there installation disks available?'. The answers was as always 'When we are ready and we will have reached the quality-level we defined', 'No we don't have installation medias, as they are always outdated. Do you have an USB-dongle with you?'. Merchandising was requested by visitors as always, but we just had some leftovers of fosdem, brought by Axel. The demonstration was as usual a small box running Babelbox and xpenguins which worked out the last years too. This year there were three lectures held by Debian related people, about Debian GIS, Aptitude - known but even unknown and SSH and unreliable network connections. The organisation team did a really great job. The social event at saturday night was very exciting and we left it early in the morning. The whole event was indeed fun and a pleasure to find new friends and meet old ones of the Free Software community. Many thanks to Florian Baumann, Jan Dittberner, Andreas Tille, Christian Hoffmann, Axel Beckert, Markus Rekkenbeil, Daniel Schier, Jonas Genannt, Jan H rsch and kurio for taking care and running the booth, which worked out this year extreme smoothly from my point of view. Likewise as the last years a special thanks to TMT GmbH & Co. KG, which kindly donated additional boothtickets, the equipment, its transportation and accommodation for almost half of the booth staff.

It is customary during Debian Project Leader elections to write a rebuttal of the other candidate's platform. However, this is very hard to do this year, as there are a lot of overlaps between platforms, goals and ideas this time around. It is hard to rebute another platform, without also rebuting one's own. So instead of trying to highlight where I disagree (there's very little of that, if at all), I'd like to point out where I found each platform lacking.Mind you, these notes are primarily based off of the platforms, because I'm sadly still behind on my debian-vote@ reading list. Furthermore, due to the high level of similarity between platforms, I will not always single out one or the other, but mention what I found lacking, or troublesome in general instead. So unless stated otherwise, my thoughts below apply to both Moray's and Lucas' platforms.The thing I miss most from both platforms is the curious lack of plans about non-packaging project members (something which I strongly emphasize in my own platform). Similarly, even though Moray does mention encouraging more local events, I find those few paragraphs a little bit lacking; and Lucas does not even mention local teams in his platform, even though they are a very, very important part of the wider Debian community, and a terrific tool in our asset. They can further our reach by long miles! Considering their importance, little's said about either local teams or local events, even when the candidate does explicitly recognise their importance.On the other hand, Lucas writes about innovation within Debian (which I found lacking in Moray's platform) - but while he does bring up the topic, I sadly found that section of his platform vague, a mistake I made last year myself, and which I tried to avoid this time, by explicitly listing a few of my ideas on how I'd imagine inspiring & motivating people (which, in turn, would foster innovation within the project).From what I see - and I may be wrong here, and I am obviously biased - both platforms are reasonably explicit when it comes to easier topics, things that we know to work, and which we'd all continue doing in the very same spirit as our predecessor. But when it comes to experimenting, to new ideas, I found both platforms too vague. There are exceptions, of course, Moray's intern idea for example is sufficiently well explained, as far as it is possible within the limits of a platform. I found no similar thing in Lucas' platform, however, most of the ideas he explained were technical things, and those are the areas where we're fairly good at, where we already have a reasonably chance of improving - and thus, an easier topic. I was suprised by Lucas' platform, because even though he lists the need for experimentation important, the lack of truly experimental ideas was lacking - I found more in Moray's platform.All in all, I found the discussions on debian-vote@ much more useful so far, than the platforms, a lot of things were asked and answered there that the platforms did not touch.

Via twitter I just discovered that Pcwizz have done a video review on Youtube of Skolelinux / Debian Edu version 6. He installed the standalone profile and the video show a walk-through of of the menu content, demonstration of a few programs and his view of our distribution. There is also some really nice quotes (transcribed by me, might have heard wrong). While looking thought the Graphics menu:

"Basically everything you ever need in a school environment."

And as a general evaluation of the entire distribution:

"So, yeah, a bit bloated. It kept all the Debian stuff in there, just to keep it nice and GNU. So, I do not want to go on about it, but lets give it 7 out of 10. I am not going to use it. That is because I am not deploying a school network. There may be some mythical feature to help you deploy Skolelinux on a school network."

To bad he did not test the server profile, and discovered the PXE installation option. It make it possible to install only the main server from CD, and the rest of the machines via the net, and might be considered the mythical feature he talk about. :) While looking through the menus, there is also this funny comment about the part of the K menu generated from the Debian menu subsystem:

"[The K menu] have a special Debian section for software that no-one is going to look at, because it contain lots of junky stuff that you actually don't need in the education distribution, but have just been included because it isn't stripped out for some reason."

I guess it is yet another argument for merging the Debian menu and Gnome/KDE desktop menu entries into one consistent menu system instead of two incomplete and partly inconsistent menu systems. The entire video is available below for those accepting iframe embedding: <iframe allowfullscreen="allowfullscreen" frameborder="0" height="315" src="http://www.youtube.com/embed/wPzTZ61Pcuc" width="560"></iframe>

I probably should have updated more of my other packages before kicking out the 2.0 release, but I wanted to get something out last night. The change to try to find test executables without any extension as well as with the -t or .t extensions broke backward compatibility. In some of my test suites, I use a helper program with no extension that's run by the actual test program with a -t extension. If the helper program is compiled and hence in the build directory, and the actual test is a shell script and hence in the source directory, then the new runtests would find and try to run the wrong program. Fix this problem by instead searching all possible locations for the test program with a particular suffix before moving on to try the next suffix on the list. This should provide the same functionality while being backward-compatible. You can get the latest release from the C TAP Harness distribution page.

The XDG standard says user-specific data files should be written [in] $XDG_DATA_HOME which defaults to $HOME/.local/share.

Appdirs explicitly breaks XDG_DATA_HOME specification and returns a path based on $XDG_CONFIG_HOME, ~/.config/<appname> instead of ~/.local/share/<appname> for user_data_dir. Although the spec is as clear as day, the authors of appdirs say this is 'subject to some interpretation'.

No, it's not, read the damn spec!

http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html



The irony is that initially they got it right, but decided to break the standard some two years ago.

Thank you appdirs authors for ignoring a perfectly clear standard just because you *think* 'in practice, Linux apps tend to store their data in ~/.config/<appname>'.


Dear appdirs authors, if some people break the law it doesn't mean the law is broken. And if *you*think* some apps do not conform to a standard, it does mean *at*all* that you should not, either. More than that, if some applications decide on their own they consider some things config data and you'd classify those as 'application data', that's their business, and is not your place to decide to break the standard for everybody. Standards exist for a reason.

BTW, your software is worthless to me now and I decided NOT to use it because of this idiotic decision of yours.

On Tuesday, there was a memorial for Aaron Swartz held at the MIT Media Lab. Unfortunately, I am traveling this week and was unable to attend. As I wrote recently, I was close to Aaron. I am also, more obviously, close to MIT and the lab. It was important to me to participate in the memorial and I found a way to give a short talk with a video. I think the lab plans to post a recording of the whole event but I have put the video of my own remarks below (and online in WebM). If you prefer, you can also read the text of the remarks. <iframe allowfullscreen="" frameborder="0" height="315" src="http://www.youtube.com/embed/uYGoAJdE8jg" width="420"></iframe>

This is basically a link-post to the Debian Project Leader election email discussions on GMANE s blog-style interface to debian-vote. After only 3 days of the 21, there s already a pageful, so if I don t start collecting links now, I ll probably miss some. Right or wrong, I ve grouped these into three topics: The Job

1. Why do you think you are a good candidate for DPL (10 Mar 2013)
2. How do you plan to represent Debian externally? (10 Mar 2013)
3. about a DPL board (12 Mar 2013)
4. DPL term duration (12 Mar 2013)
5. Work balance and traveling (12 Mar 2013)
6. trying to do awesome and risking to fail (11 Mar 2013)
7. To Lucas: how do you plan to push your ideas (12 Mar 2013)
8. All candidates quotes for the press if you win (13 Mar 2013)

Money

1. using debian funds for Debian s hardware infrastru (12 Mar 2013)
2. Usage of Debian s Money (12 Mar 2013)
3. Debian s relationship with money and the economy (12 Mar 2013)

Project Management

1. getting new people to Debian (10 Mar 2013)
2. Free Software challenges and Debian role (11 Mar 2013)
3. Development and technical issues and challenges (10 Mar 2013)
4. Are there problematic infrastructure or processes in Debian? (12 Mar 2013)
5. to Moray: encourage teams to take interns (11 Mar 2013)

So, what do you think are the key points or differences? Leave me a comment, or get involved in the discussions. Campaigning ends and voting begins 30/31 March.

...we stopped the singing. This may be irony:

No More Google Reader
from Google Operating System by Alex Chitu
via What's Hot in Google Reader

but the topic at hand remains serious. This is a stark reminder of why it's important to be able to get your data out of cloud services at a moment's notice and why you should always keep local backups of important data. Thankfully, Google is quite fair about this, as opposed to a lot of other large players. (As a side note, export of full data should be doable, but come with another layer of security like a grace period, password confirmation, verification email, or other means. Imagine someone gains temporary access to your account and exports all your digital life at once. And yes, I know how that relates to IMAP.) Google Reader has felt like a second class citizen for quite some time now, the removal of features to push Google+ and the slow updates for Android were two signs that didn't forebode well (Vim's spelling correction does not know "forbode", how weird is that)(turns out I'm simply stupid, see comments; thanks Jim). I can see how this move may make sense from Google's POV even though I am sad to see an old companion go. Anyway, the stats page of two of my accounts show how sadly I will miss Google Reader in two simple lines:

Since January 24, 2010 you have read a total of 59,253 items.
Since February 20, 2011 you have read a total of 215,285 items.

Farewell, my friend. PS: Maybe those petitions will sway them, but I won't get my hopes up. PS: Speaking of Google, I found this gem today; TMI.

While working on fixing a few encoding problems that I managed to introduce to the BTS almost half a year ago, I took a side bit of coding, and introduced libravatar support to the BTS. Every e-mail now has an avatar to the right which should correspond to the sender. Libravatar is a federated service, which means that if you control your domain, you can serve your own icons. It also automatically falls back to gravatar, so if you're using that service, things should "just work". Hopefully this will be primarily amusing, and people won't abuse it. More importantly, but much less fun, the double encoding problems (where mails would get double-encoded if any of the headers contained non-us-ascii text), and mojibake wontfix icon ( ) should be fixed now. If you see any additional cases of this, please report them to owner@bugs.debian.org.

sage@guug This January, I held an updated version of my "Gitify your life" talk at sage@guug, which is the Munich chapter of the German Unix User Group. Once again, it was quite successful and turnout was about three times that of normal attendance. Afterwards, I was asked if I would be willing to hold the same talk at the German Unix User Group's Spring Meeting in 2014 as the programme for 2013 has been finalized for quite some time, already. Obviously, I said I would be willing to do that and will submit the talk once registration opens. Open Source Days By the way, I am writing this post from a hotel room in Copenhagen. Tomorrow, there will be a world first... "Gitify your life" will not only be held as a talk, it will be expanded into a full three hour workshop (with the option of extending it if there's interest as the room will be free after my workshop). Having never held a workshop of this scope, I am really looking forward to this experience. Besides enabling me to get people up and running with their personal data, this will allow me to see where new users are having issues with vcsh. I keep hearing of people not getting their heads around the concept easily, but no matter how many promises of updates to my docs I get while helping them, no one really follows through with that promise once they know how to use vcsh. Maybe I will find someone willing to follow through and/or simply gather enough material for docs 2.0. Your place? As an aside, and I will probably try to put a real announcement together as well, if you know of a conference, user group or other place where a lightning talk, a full talk, or a workshop about one or all of these topics would fit, please let me know. I will gladly submit proposals to or accept invitations from most places as long as I can spread this message. Especially if the venue happens to be reasonably close to Southern Germany, covers travel cost, or if they can guarantee video recording of the talk. Joey Hess has speculated if the talk isn't really called "git naked" as I seem to keep on giving that particular talk at venues which don't have video recordings. Please help me make Joey happy... ...with a video recording, I mean.

Hi everyone! Today I make an announcement I thought I would never ever make. But things changed. Discussion about this has a long history, starting as a non-serious suggestion at DesktopSummit 2011, continued with people on IRC, but it was decided back then that it wouldn t be worth the effort. This has changed too, and a small team has formed to work on it. We hereby announce Tanglu, a new Debian-based-Linux distribution. Why do we need another one? Let me explain the concepts of that distro: Tanglu will be based on Debian Testing and follow the Debian development closely. It will have a 6-months release-cycle and it s target audience are Linux desktop users. We will make installing and setting up the distro as easy as possible. Tanglu will be usable for both developers of upstream software and the average Linux user and Linux newbie. This is possible because in our opinion developers and users don t have different needs for a desktop system. Both kinds of users like a polished desktop which just works . We will, hwever, not apply any kind of fancy modification on upstream software, we will basically just distribute what upstream created, so users can get an almost pure GNOME and KDE experience. Tanglu is designed to be able to solve the issue that Debian is frozen for a long time and Debian Developers can t make new upstream versions available for testing easily. During a Debian freeze, DDs can upload their software to the current Tanglu development version and later start the new Debian cycle with already tested packages from Tanglu. The delta between Tanglu and Debian should be kept as minimal as possible. However, Tanglu is not meant as experimental distribution for Debian, so please upload experimental stuff to Experimental. Only packages good enough for a release should go into Tanglu. Ideally, Tanglu and Debian should be working well together in mixed environments, where you for example have Debian servers and multiple Tanglu desktops with the new software, targeted at desktop user. Since the differences between Tanglu and Debian should not be very high, administering both systems should be very easy (if you know Debian). Tanglu will be an open project, driven by community. At the beginning of each cycle, people can make suggestions for release goals they want to implement (similar to Fedora, but without FESCo). These proposals are discussed in public and are rejected if there are major technical concerns. If consensus about a certain proposal is lacking, a vote is done for it. The proposal can be accepted with absolute majority. If this does not happen, the proposal is postponed for the next release, where people can vote for it again. If nobody wants that function, it is rejected. In general, decisions made by Debian are superior and have to be followed. We don t think we know every package and every software better than the original upstream. That s why it makes much sense to rely on feedback from others and to have a community-based and peer-reviewed distribution, instead of secretly developing stuff and dumping it on the community. Tanglu will have a highly predictable set of features, defined at the beginning of a cycle, so you will know what you can expect from the next release as soon as possible and plan for it. Tanglu will make it easy to deploy applications for it. It will contain a software-center, similar to what Ubuntu has. We will also try to establish a solution for a Linux-AppCenter , a place for Linux applications, which will be open not only for Tanglu, but can be implemented in any other distribution too. Possible income will flow back into development of the platform. Now, let s answer the FQA (Future Questions Asked): Why don t you contribute to Debian directly and create yet another distribution? First of all, we contribute to Debian And for me, I can say that I will contribute to Debian even more. The point is that Debian can not cover all possible use-cases, and with Tanglu we want to make a distro which solves this. You might ask why we have to create a new distro for that, instead of creating improvements inside Debian? Creating a new distro allows us to do stuff we can never do in Debian. For example, we will include proprietary firmware in that distro, we will make installations of proprietary stuff possible easily (but don t ship with it by default) and we will have a time-based release cycle. These are already things which are a no-go for Debian, and that s fine. We don t want Debian to support these cases, as it is already a great distribution. We want to offer a distro as close to Debian as possible, but with a few modifications for use-cases which are not covered by Debian itself. Of course we will participate in DEX. If Debian Developers contribute to Tanglu, freezes will take even longer! This is an often-heard concern, it comes up on every mailinglist discussion about continuing development while freeze. I would disagree here, packaging new upstream stuff is not slowing down testing and improving of packages in Testing. Also, Tanglu is an offer for Debian developers to participate (we will sync privileges for their packages) we don t expect anyone to work on it, but as we think DDs know their packages best, we will make it possible for them to participate without extra barriers. We hope that Tanglu can add value to Debian and that Debian cycles can start with better-tested packages. You said you are a small team you cannot develop a whole distribution with it! Let s put that to the test! All people working on this are well aware of the issue that the project can not survive without much community-involvement on the long run. But we see a chance that many people are interested in it and that there is a high demand for it. At the beginning, we will just start with a small set of packages. We will also sync many packages from Ubuntu, to reduce workload. For example, it is planned to use the Ubuntu-Kernel and KDE packaging. By doing this, we keep the workload at the beginning low. We also reduce duplicate work with that. We even have some possible sponsors for the new distribution. But nothing is set in stone yet, so just wait for it to happen. Why not participate in Arch, OpenSUSE $other_distro? These are not Debian . I know, it sounds odd, but if you like the Debian way of doing things, you want to use a Debian-based distribution. There is nothing wrong with OpenSUSE. And Debian has issues too. But we want to be close to Debian and use it s tools and way of doing things. I hate you!!! You are doing it wrong!! The project is useless!
Well, that s fine. But there is no reason for hating us. If you dislike our idea, there are basically two options: First, you hate us but the project is successful. In that case, you have been wrong with hate, as there are definitely people who liked the project and contributed to it. Second, you hate us and we fail. In this case, there is no reason for hate, as the project will just vanish and you don t have to worry about it. So hating it would ve been just a big waste of energy. Also keep in mind that forking is a way to keep development healthy and to adapt software to new use-cases which it didn t target before. And we are not introducing incompatibilities here (like e.g. writing our own display server could). Instead, we want to stay close to Debian and reuse as much code as possible. Which desktop will you use? Everyone can add a new desktop to Tanglu, as long as the desktop-environment is present in Debian. Long term, we will have to offer Linux-newbies a default flavour, probably by setting a default download on the website. But as long as there is a community for a given desktop-environment, the desktop is considered as supported. At the beginning, we will focus on KDE, as many people have experience with it. But adding vanilla GNOME is planned too. Can you say something about the software used in Tanglu? Yes, but this is still in flow, so I can t promise final decisions here. On the server, side, we will try to use the Dak for repository management, as soon as we have enough server capacity. We will also use the standard Debian repository GUI and basically reuse most of the software there, to diverge less from Debian. The distribution itself will use a Linux Kernel from Ubuntu and systemd as the primary init system, as well as the login manager logind. It will be based on current Debian Testing with some fresh packages from Unstable and Experimental. We might also use the Ubuntu driver packages and KDE packaging. We expect to have a very rough start with the first release, but there will be enough time to polish Tanglu. Nice idea! How can I help? Well, you can help with basically anything at time from writing manuals, over designing logos and pages to administering a webserver and create packages. We are at an early stage of development at the moment, but we wanted to go public with it as soon as possible, to include the community and receive feedback so we can make that distro community-centric from the beginning. Most of the infrastructure is currently in progress too. So, if you want to get started with Tanglu, subscribe to our mailinglist tanglu-devel and write a mail to it, intruducing yourself. We can then include you in the loop. Generally, if you want to get access to our machines, a trusted GPG-signature will help a lot. If you want to talk to us, join #tanglu-devel or Freenode! Most discussions are currently happening there. And that s it! Tanglu will be awesome!

Some other projects of mine will develop a bit slower because I am now involved in Tanglu. But nothing will stop, and there is some pretty cool stuff coming for both GNOME and KDE (and I still have to implement DEP-11 for Debian).

Normally when using Subversion s SSH authentication facility you ll use your own SSH generated key-pair that svn will read from the proper location, usually $HOME/.ssh. However, there could be situations when you ll need to use a different key-pair. In such case you can use a nice trick to make svn+ssh authentication work smoothly. Let s say you have an external key-pair and the public key it s already set on the svn server. You store the private key somewhere in your home directory. Now when doing a svn checkout you ll find that you have to use SSH s -i parameter. However, when used together with svn it won t work as expected as you ll normally do when using ssh with an external key-pair. Subversion makes your life easier by using the $SVN_SSH environment variable, so you can put the ssh command and modifiers that fit your needs. In the external key case you can do something like:

export SVN_SSH="ssh -i </path/to/external-key>"

Next time you use svn s ssh auth it will read $SVN_SSH then use it. So now you can use svn commands such checkout, etc in the same fashion you normally do.

svn co svn+ssh://rudy@somesvnserver.at.somwhere.org/repo/for/software

Update: Jeff Epler offered great advice for using .ssh/config and key-pairs based on hostname. See the comments below.

Review: Higher Order Perl, by Mark Jason Dominus

Publisher:	Morgan Kaufmann
Copyright:	2005
ISBN:	1-55860-701-3
Format:	Trade paperback
Pages:	563

Disclaimer: The author and I both used to be (technically still are, but the newsgroup is dead) moderators of comp.lang.perl.moderated, and I knew him electronically from answering questions on the Perl Usenet newsgroups many years ago. Perl is, in general construction, a typical imperative language in the style of C, with object-oriented capabilities bolted (somewhat awkwardly) on. Most Perl code looks remarkably like C code with more powerful built-in data types and no memory management. A minority of Perl programs embrace object-oriented techniques and remind one of C++ or Java (or Python). But Perl also has powerful capabilities borrowed from functional programming languages and the Lisp tradition, even though many Perl programmers rarely use them and aren't familiar with them. Higher Order Perl focuses on those capabilities and how to use them effectively, starting with callbacks and uses of function pointers and moving into recursion and analysis of recursive functions, iterators, currying, and parsers. It concludes with a fully-worked example of constructing a declarative programming system using the techniques developed earlier in the book. Higher Order Perl is a programming book for an intermediate to advanced Perl programmer, already a rare topic. This is not a look at how to apply Perl to another area of programming, or a cookbook of techniques. It's an attempt to help a Perl programmer think about and use the language differently, and it follows through on that. It's refreshing and rare to read a programming technique book that's targeted at the practicing expert. (I think the size of the audience is often too small for publishers to target it.) Even the most experienced Perl programmer is probably going to learn something fundamental from this book, not just interesting trivia around the edges. On the negative side, though, I found the book a bit too focused on computer science and mathematics problems, particularly in the choice of examples and sample scripts. (And I say this as someone who has a master's degree in computer science with a software theory focus.) This is a bit hard to avoid for topics like recursion, where problems like computing Fibonacci numbers are classic, but throughout the book I struggled to focus past feelings of "but why would I ever do that"? The extended discussion of the Newton-Raphson method is the most memorable; I'm not sure that's a problem many Perl programmers would have and need a higher-order technique for. There's also a lot of discussion of recursion analysis and transformations between recursive and iterative expressions of problems, which is ground I remember well from my degree but which I've rarely had any practical use for in day-to-day programming. This is not a uniform problem, though, just a tendency. There are some great examples that I think are more in the mainstream of Perl problems, including a reinvention of File::Find that shows how to add more flexibility, a web spider, and a great discussion on how to construct a conventional queryable database on top of a flat file (a topic that's near and dear to my heart). And then there's the chapter on infinite streams. Dominus presents a method of using closures to create a version of an interator: a sort of infinite linked list that can keep generating additional elements. He presents this in several contexts, but one of them is log parsing, and that turned out to be exactly the solution that I needed for a problem I was working on while reading this book. I've written about this elsewhere, but this was a wonderful idea that helped me think about both Perl and a major application area in a completely new way, and I wrote an application using that knowledge that would have taken me much longer using different techniques and would have been much less fun. So, for me, this chapter was more than worth the entire book, and blankets the rest of it in a delighted feeling. Other people may or may not have that experience. I think it will depend on whether one of these techniques hits home for you the way that one did for me. This is a book with some idiosyncracies, and some sections that may drag. Having lots of fully-worked examples is a major plus, but some of those are so comprehensive that one can get a bit lost in the details. That particularly hit me with the last couple of chapters on parsing and on the example declarative programming application. Quite a bit of that text involved reinventing a recursive descent parser (another very computer science example), when I'm not quite sure why one wouldn't use one of the existing parser generators on CPAN for practical purposes, and inventing a lot of new syntax to try to make parser Perl code more readable. But one certainly can't complain that Dominus omits necessary details, and there is some appeal in watching an experienced programmer work through a problem from analysis to implementation. But, despite the idiosyncracies, I recommend this book to any experienced Perl programmer who wants to expand their view of the language. The techniques here closures, higher-order functions, iterators and streams, and formal parsers are powerful and underused in the Perl community. You may want to pick and choose which sections you pay close attention to, but I think everyone will find something of interest here. In the weeks since I read this book, my opinion of it has only grown. And I can't tell you just how much I loved the infinite stream concept. Rating: 8 out of 10

Today, during Blackhat Europe, Offensive Security announced the availability of Kali Linux 1.0, which aims to be the most advanced, robust, and stable penetration testing distribution to date. It is the successor of Backtrack Linux. Kali s choice of Debian Kali s release is a significant event in the security auditing and penetration testing field, and I m proud to see that Debian was retained as the best distribution to create this new product. Here s what Mati Aharoni of Offensive Security told me:

Debian provides a reliable base to build a new distribution and yet can easily be customized to add bleeding edge features, thanks to the unstable and experimental distributions.

Kali s development policies Even though Kali was prepared in secret, from now on Kali s development happens in the open in public git repositories. There are repositories for all the packages that have been created (or forked) as well as for the ISO images creation script. Debian packages are maintained with git-buildpackage, pristine-tar and the associated helper tools, making it easy to integrate the latest changes of Debian. Kali packaged several hundreds tools that relate to their field and they intend to contribute those which are DFSG-free back to Debian. Kali s technical infrastructure In the last year, I have been working within the Kali team to setup large parts of their infrastructure as a proper Debian derivative. Kali s main ISO images are built with live-build. All the bugfixes that I contributed to Debian Live were the direct result of my work for Kali. The git repositories are managed with gitolite. The package repositories are built with reprepro. The build daemons use rebuildd and sbuild. The (push) mirrors are synchronized with the same tools than Debian (based on rsync), but there s also a central server which redirects to a mirror close to you (and which is used by default everywhere). This one runs mirrorbrain (and not Rapha l Geissert s redirector). The ARM build daemons (armel/armhf) run on machines powered by Calxeda s Highbank (4 cores, 4 GB RAM) that work pretty well. Even better, Offensive Security is willing to dedicate one node of this cluster for Debian s own usage. The future This first release is not an end. It s only the start of a journey. Not all applications have been packaged yet and there s lot of work left to integrate everything in Debian. I m really looking forward to continue my collaboration with the Kali team as this has been one of the most interesting project I ever had as a Debian consultant. And also one of the few where I could really contribute something back to Debian.

9 comments Liked this article? Click here. My blog is Flattr-enabled.

Assigning a value to a variable when executing a command is a way to populate the command's environment, without the variable assignment persisting after the command completes. This is not true, however, when a special built-in is the command being executed.

POSIX:2001 states that "Variable assignments specified with special built-in utilities remain in effect after the built-in completes".

Not only this is tricky because it depends on whether a utility is a special built-in or not, but the bash interpreter does not respect that behaviour of the POSIX standard. That is, special built-ins are not so "special" to the bash interpreter.

This leaves two things to take into account when assigning to a variable when executing a command: whether the command is a special built-in, and whether bash is interpreting the script.

Now, the list of special built-ins is rather short and it would be a bit unusual to perform variable assignments when calling them, except for some cases: "exec", "eval", "." (dot), and ":" (colon).

It is important to note that ":" and "true" differ in this regard; the former is a special built-in, the latter is just a utility. Watch out for this kind of differences when using ":" or "true" to nullify a command. E.g.

Compare

$ dash -c '
method=sed
# some condition or user setting ends up making:
method=true
# later:
foo=bar $method
echo foo: $foo'
foo: 

To (redacted for brevity):

$ dash -c '
method=:
foo=bar $method
echo foo: $foo'
foo: bar

WebAuth is the site-wide web authentication system that we use at Stanford. After a lot of time focusing on things at work other than coding, it's been my primary job for the past nine months or so, which has been a lovely change. I wasn't planning on doing another 4.4 release and instead focusing on 4.5.0 (which is well underway and adds significant new features to multifactor support), but I kept finding bugs, including several that were quite embarassing. So this is another bug-fix release, and hopefully the last one before 4.5.0. WebAuthTrustAuthzIdentity enabled WebAuthDoLogout in the same scope (ever since it was added in 4.4.0). It's horribly embarassing to have been writing C for more than 20 years and still manage to omit a break in a switch statement. Benjamin Coddington found another bug in authorization identity handling where if the user changed their authorization identity to match their authentication identity, their authentication would be rejected. We now detect that case and just discard the authorization identity if it's the same as the authentication identity. There are multiple fixes in the mod_webauth logging of bad app tokens: expired app tokens are now logged with a proper message (and at info level rather than error), empty app tokens (created internally and therefore seen by subqueries before fixup to delete expired app tokens) are now just ignored, and invalid app tokens don't result in spewing binary strings into the Apache error log. mod_webauthldap has, since the beginning, refused to add more than 127 values of a multivalued LDAP attribute to the environment. We ran into that limit with entitlements, so it's now been removed. This runs the risk of overflowing the environnment, but I did some calculations, and it looks like you're going to have to try really hard and have impressively large multivalued attributes to create that problem. If it actually becomes a problem for someone, I'll add a configuration option. There's also a syntax fix to the default WebLogin error template and a variety of minor bug fixes, mostly around error handling conditions, to correct problems caught by clang --analyze. (The current master branch gets a completely clean bill of health from clang --analyze using clang 3.0.) You can get the latest release from the official WebAuth distribution site or from my WebAuth distribution pages.