Follow me on Identi.ca or Twitter or Diaspora* Sysadmins usually know ServerSpec, a tool allowing to check if your servers are correctly configured. The use of automated deployment tools makes deployments easier and easier, but lot of unexpected situations can come up using this tool, especially when more and more servers are implied. ServerSpec allows to verify that the result of your deployments are what you were expecting for and that no exception or unexpected event goes undetected. Backup Checker (Github stars appreciated :)) offers the same kind of controls, but for your backups. Once you have backups, you don t know if they contain what you are expecting for. And this is especially important because having broken backups mean losing your data after a major outage! Moreover if you think you are well protected with backups but they are eventually broken (corrupted archives, archives being filled with empty files, saving useless files ), you will perform your server or database upgrades without being really protected, leading to terrible situations. Backup Checker offers lots of controls over backups:

• check if an archive is corrupted
• control if some files or directories exist inside an archive exist
• check if a file size inside an archive is not null or has a specific size (operators smaller than, equals, greater than supported)
• owner, group, uid, gid of files/directories inside an archive
• and a lot more. Check the official documentation!

Use Backup Checker with Backup-Manager or Rsnapshot Starting from Backup Checker 1.7,if you use a backup tool like Backup-Manager or Rsnapshot, you ll be glad to know that Backup Checker perfectly works with them. The documentation is available on the Backup Checker Community page with the howtos to install and configure Backup Checker with Backup-Manager or with Rsnapshot. What about you? Let us know in the comments what you think of Backup Checker and its features to connect with other backup tools. We would be happy to get your feedbacks about how you use Backup Checker or what you expect from a backup checking solution.

Follow me on Identi.ca or Twitter or Diaspora* Backup-manager is a tool creating backups and storing them locally. It s really usefult to keep a regular backup of a quickly-changing trees of files (like a development environment) or for traditional backups if you have a NFS mount on your server. Backup-managers is also able to send backup itself to another server by FTP. In order to verify the backups created by backup-manager, we will use also Backup Checker (stars appreciated :) ), the automated tool to verify backups. For each newly-created backup we want to control that:

• the directory wip/data exists
• the file wip/dump/db.sql exists and has a size greater than 100MB
• the files wip/config/accounts did not change and has a specific md5 hash sum.

Installing what we need We install backup-manager and backup checker. If you use Debian Wheezy, just use the following command:

apt-key adv --keyserver pgp.mit.edu --recv-keys 2B24481A \
&& echo "deb http://debian.mytux.fr wheezy main" > /etc/apt/sources.list.d/mytux.list \
&& apt-get update \
&& apt-get install backupchecker backup-manager

Backup Checker is also available for Debian Squeeze, Debian Sid, FreeBSD. Check out the documentation to install it from PyPi or from sources. Configuring Backup-Manager Backup-manager will ask what directory you want to store backups, in our case we choose /home/joe/dev/wip In the configuration file /etc/backup-manager.conf, you need to have the following lines:

export BM_BURNING_METHOD="none"

export BM_UPLOAD_METHOD="none"

export BM_POST_BACKUP_COMMAND="backupchecker -c /etc/backupchecker -l /var/log/backupchecker.log"

Configuring Backup Checker In order to configure Backup Checker, use the following commands:

# mkdir /etc/backupchecker && touch /var/log/backupchecker.log

Then write the following in /etc/backupchecker/backupmanager.conf:

[main]
name=backupmanager
type=archive
path=/var/archives/laptop-home-joe-dev-wip.%Y%m%d.master.tar.gz
files_list=/etc/backupchecker/backupmanager.list

You can see we re using placeholders for the path value, in order to match each time the latest archive. More information about Backup Checker placeholders in the official documentation. Last step, the description of your controls on the backup:

[files]
wip/data  type d
wip/config/accounts  md5 27c9d75ba5a755288dbbf32f35712338
wip/dump/dump.sql  >100mb

Launch Backup Manager Just launch the following command:

# backup-manager

After Backup Manager is launched, Backup Checker is automatically launched and verify the new backup of the day where Backup Manager stores the backups. Possible control failures Lets say the dump does not have the expected size. It means someone may have messed up with the database! Backup Checker will warn you with the following message in /var/log/backupchecker.log:

$ cat /var/log/backupchecker.log
WARNING:root:1 file smaller than expected while checking /var/archives/laptop-home-joe-dev-wip-20150328.tar.gz: 
WARNING:root:wip/dump/dump.sql size is 18. Should have been bigger than 104857600.

Other possible failures : someone created an account without asking anyone. The hash sum of the file will change. Here is the alert generated by Backup Checker:

$ cat /var/log/backupchecker.log
WARNING:root:1 file with unexpected hash while checking /var/archives/laptop-home-joe-dev-wip-20150328.tar.gz:
WARNING:root:wip/config/accounts hash is 27c9d75ba5a755288dbbf32f35712338. Should have been 27c9d75ba3a755288dbbf32f35712338.

Another possible failure: someone accidentally (or not) removed the data directory! Backup Checker will detect the missing directory and warn you:

$ cat /var/log/backupchecker.log
WARNING:root:1 file missing in /var/archives/laptop-home-joe-dev-wip-20150328.tar.gz: 
WARNING:root:wip/data

Awesome isn t it? The power of a backup tool combined with an automated backup checker. No more surprise when you need your backups. Moreover you spare the waste of time and efforts to control the backup by yourself. What about you? Let us know what you think of it. We would be happy to get your feedbacks. The project cares about our users and the outdated feature was a awesome idea in a feature request by one of the Backup Checker user, thanks Laurent!

A new release of Backup Manager has been published. This release closes a bunch of bugs reported in Debian and in the official Bugzilla. The release notes are available here. Here is the announcement made on the user mailing list. I ve also uploaded a new Debian package to sid, in order to provide 0.7.8 for the unstable distribution of Debian. Reports are welcome (but bug-related ones are prefered, if possible in the official Bugzilla rather than in the Debian BTS).

It is sad to see how people complain about how packaging Ruby gems is painful. It seems like it is the Java packaging game all over again where any application ships its dependencies in a lib/ directory (or in the case of Rails, vendor/). Mac OS X applications seem to do some of the same thing by shipping lots of libraries in their application bundle, which is really just a directory with some magic files in it. This is of course just like static linking, which we made away with for most software many years ago, mostly due to the pain associated with any kind of security updates. Update: What I find sad is that people keep making the same mistakes we made and corrected years ago, not that people are complaining about those mistakes.

Hey planet, it s been a very long time since I took my e-pen to write something down here, my work is taking pretty much all my time nowdays. Anyway, it seems that there is sort of a polemic rising here and there about RubyGems and its packaging philosophy. I d like to take that opportunity to contribute to the debate with the experience I had at Yoolink. Yes, we provide Ruby on Rails applications and are defacto dealing (or trying to) with Ruby and its gems. Wouter enlightened the main reason why it s such a mess to put into production a Rails app and to maintain it : the Ruby on Rails ecosystem is unstable. If you start writing your app, there are very big chances that when it gets mature for production mode, one or more of the Ruby libs you depend on will have a new version. And sadly, chances are there too that one of those libs will provide an API change. This can happen in a lot of places in a Rails app, because of its modular structure: plugins, gems or even Rails core upgrades (try to upgrade a real-working Rails app from 1.x to 2.x if you like tricky games). I agree with Joss when he said that Debian packages and Ruby gems are not designed for the same purpose, the latter are for a developer ecosystem and the first suitable for long-working production ecosystems. That s why setting up a complex Rails app in production is a nightmare : no Debian packages for all your Ruby needs on the first hand and API-unstable and hard-to-upgrade gems on the other hand. At Yoolink, the only working way we ve found to maintain in production our Rails apps is to freeze pretty much every Ruby-thing we can. Our production servers are then Rails/Ruby free (nothing installed system-wide) and the apps provides all their Ruby dependencies in their vendor/ directory. You can t do that for native gems but that works for 90% of a random Rails app s dependencies. To conclude, I d say that most of these issues come from the philosophy of the Ruby/Rails people : they don t fear to change anything they want from one release to another, just take a look at the gettext support in Rails 2.2 and you ll understand what I mean . It s like they don t even take in mind that people are now using their stuff, in real world, with customers relying on their apps. It s not a pet project anymore guys, if Rails want to remain the killer app it is, it will have to be someting stable and maintainable.

It’s been a looong time since I blogged here. Perhaps I should write more frequently so you won’t start thinking I died silently ;). I’m not dead. It’s just that I’ve started a company a couple of months ago and that does take a lot of my time. It’s pretty exciting, - as you can imagine - we’re using Debian everywhere and the team is great. I’ll write another blog-entry about that later. In the meanwhile, I’m on holydays for a week, and I plan to squash my TODO list:

• Backup Manager (a couple of bugs have to be fixed)
• Electricsheep (a new upstream release is available)
• My Perl modules deserve a couple of maintenance too, like Coat::Persistent

I’ve uploaded a new major release of TinyMCE into sid, the package jumps from the 2.x branch to the new 3.x one. For the record, packages that use TinyMCE should rather depend on that packge instead of shipping the sources itself.

Yes I know, it’s like a year since the last development release of Backup Manager has been published, I should be ashamed of such a huge idleness. But actually, If you ask me, I’m not. Instead of writing long lines of haiku to apologize for being so late at releasing, here is an illustrated reason why it was so loooong.

Yes I know, it’s like a year since the last development release of Backup Manager has been published, I should be ashamed of such a huge idleness. But actually, If you ask me, I’m not. Instead of writing long lines of haiku to apologize for being so late at releasing, here is an illustrated reason why it was so loooong.

Dear lazyweb, this sunday afternoon, I opened BackupManager’s Bugzilla and started squashing bugs. There was 54 bugs opened that requested a review. 15 of them are now tagged “pending” (meaning they’re closed in SVN and will be shiped with the next release). All of the patches submited - that made sense - were applied, thanks to all the reporters for their help, by the way. So, fear a new release soon…

If you’re like me and want to build the ieee80211 driver for your debian box with the last kernel available in sid (2.6.24), you’ll have to apply that patch to the sources. As you can see, a couple of changes occured in the Kernel API and that blocks the build. Hope that can help.

Sam suggested in his lat blog entry to create a team whose role would be to handle the promotional stuff. I find that idea pretty good, having a central point where to take decisions regarding how Debian should communicate with the outside world makes sense. I totally support that idea.

Some days ago, my RSS reader found Mark Jason Dominus' Help.pm - Yes, the module is (so far, at least - I could not find it on CPAN) only published as a blog post. But don't let that fool you - It's a beautiful (and simple!) Perl module that can help developers that are too lazy to go look up methods in the man pages. Perl's introspection capabilities are not behind other dynamic languages' (i.e. Python's or Ruby's, speaking only about what I'm familiar with). However, it's used much more seldom, partly because Perl does not ship by default with an interactive console (such as Ruby's irb or Python's regular behaviour when called without an input script). Of course, writing a Perl console is an easy task, and good Perl consoles exist, although its use is not part of the Perl culture. But of course, just glancing over MJD's code made me come up with a simple, yet useful, way to use introspection in Perl, usable as a simple one-liner. Say you want to look at all of the methods provided by IO::File:

gwolf_at_mosca[25]/tmp$ perl -e 'use IO::File; print join(", ", grep defined & "IO::File::$_" sort keys % "IO::File::" ), "\n"'
binmode, carp, confess, croak, gensym, new, new_tmpfile, open, qualify, qualify_to_ref, ungensym

Want the scalar variables? Of course:

gwolf_at_mosca[26]/tmp$ perl -e 'use IO::File; print join(", ", grep defined $ "IO::File::$_" sort keys % "IO::File::" ), "\n"'
VERSION

Same goes for arrays and hashes. And, of course, leaving out the grep gives you anything. Yup, it's the magic package-name hash trick. Main difference between this and MJD's Help.pm? That Help.pm goes up the inheritance chain, and is thus much more correct. Of course, I'll be uploading Help.pm to Debian very soon - And, why not, I think I'll add a way for it to query on different symbols, not just on methods. And the simple binary to call from the command line. Sounds very much worth it ;-) Thanks, MJD!

At work, I was voluntold to deal with an old AIX server, and today, part of my job was to get OpenSSH up and running… Erf. How the hell could someone work with the tool “smitty”? Do they call that a package maintenance system? Do people really pay for that ? This makes me sick, why didn’t they choose Debian? I have the strange feeling of being 10 years back in time, dealing with an old interface, user-unfriendly, terminal intolerant and anti-productive. Sometimes, life is crual.

There is going to be a Perl event in Lyon, France during November : The French Perl Workshop 2007 . I’ve submitted two talks, that have been approved today:

• Perl Console - Votre compagnon pour d velopper en Perl (Fr)
• Red couvrir le Perl Objet avec Moose (Fr)

I was thinking of a talk about the Debian packaging of Perl modules. That could be interesting to enlight how Perl libraries are packaged, how one can build his own package for a random module and so on.
This would be a talk designed for Perl developers under Debian. Would be great to do that with a member of the Debian Perl Group. If you’re interested, feel free to contact me so we can coordinate our efforts. Other talks are listed on that page.

A new version of PerlConsole is available. As you can see in the changelog, lots of goodness with this release comes to you (~/.perlconsolerc rules, thanks to S. Zachariah Sprackett for his patch). After reading the source code of Devel::REPL and learning from it (this is really a great library, congrats to mst) I decided not to orphan Perl Console. Yes, that was a possibility as Devel::REPL provides anything you need to build a console (and comes with “re.pl” which actually is a console itself). I decided not to give it up for one reason: Devel::REPL is great, and better written than Perl Console (it supports plugins and profiles and is Moose-based) but, it’s slow. When you launch it, you have to wait a couple of seconds before the console prompts you. This is because of Moose I suppose. I know that’s not blocker as a console is a development tool, but I feel like some of PerlConsole’s users would like to have the choice between a full featured, extensible and Moose-base console that’s slow, and a light one that is basic in design and implementation. I backported one of the most interesting plugin of Devel::REPL to PerlConsole, being the console’s lexical environement. Devel::REPL did a great thing there, using Lexical::Persistence to simulate a namespace in the console. Feel free to give your feedback about this, I wonder if there’s still an audience for PerlConsole when Devel::REPL is in the town. I think so, maybe I’m wrong, then tell me.

A good read eval print loop for perl has been on my Wanted list for a long time. Matt S Trout’s Devel::REPL has been looking promissing but I didn’t want to fiddle with installing the dependencies. Thanks to Alexis Sukrieh and the Debian Perl Group Devel::REPL will enter unstable. Yeahhh… Another thing on my wanted list was Debian packages of Perl::Critic. Couldn’t find a ITP, but looking around at the Debian Perl Group’s website to find out how to package it I discovered that Joey Hess allready did the work. I’ve been using the morning installing the above from the subversion repository on pkg-perl.alioth.debian.org. Only problem was that I needed a newer package of libppi-perl which wasn’t maintained in subversion. What’s next? Looking at my ~/.perl Perl::Tidy seems to be a candidate.

Thanks to Florian Ragwitz, who packaged libpadwalker-perl 1.5-1, there is no blocker anymore that prevents libdevel-repl-perl from entering sid: By the way, the author of Devel::REPL, Matt S Trout, looks pretty happy to see his module entering Debian. I’ve just uploaded libdevel-repl-perl, this upload closes the exciting work session we did during all the weekend with Damyan Ivanov, in order to get the module into debian. All its dependencies are now in the Perl group’s hands. That was fun. Team maintenance rocks!

I decided to play with Devel::REPL which looks to be exactly what I need to enhance my Perl Console. That module is not packaged in Debian, then I started packaging it with the help of the Debian Perl Group (big thanks go to Damyan Ivanov for his help). The day was pretty productive and we’re almost done now, as you can see in this tomboy note: PS MadCoder: I’m sorry dude, I’m still speaking about Perl

The first version of the debian package of Perl Console has been uploaded to the NEW queue. For those who are waiting for it, I’ve also uploaded the package here. Thanks to the patch sent by Antonio Terceiro, the version 0.3 will be properly packaged ala Perl (namely with the famous Makefile.PL, MANIFEST and friends). I plan to adress the multi-line issue for 0.3 (mainly handling code with loops or conditional structures), as Florian Ragwitz underlined, it could be worth using Devel::REPL instead of rewriting the wheel.