Search Results: "will"

13 September 2013

Joey Hess: a special announcement from RMS

A little grep has told me that RMS is going to announce something special on olduse.net in two weeks time, on 27 September 1983. Best not to speculate about the details, but this announcement will be something people are still talking about thirty years later. So it's an event you and your news reader won't want to miss. I have set up an IRC channel #olduse.net on FreeNode, and we'll see who first spots the post!

Miriam Ruiz: Pink Pony: a 3D multiplayer Tron-like racing game, with ponies

The first time I found this cute game, was around 2009, but for a bunch of reasons I haven t been able to get it into the archive until now. Pink Pony is a cute game in which you have to control a pink pony who s main goal is to last in the game more time than the other ponies. You might remember the game Tron, or the film, in which there were light cycles who left a trace behind, with which the other cycles crashed. Well, this is the similar game concept, only with cute ponies instead of light cycles. You can see a video of the game in action, if you want to get a feeling of how it is. Thomas Weber (Ginko), has published a newer version of the game (1.3.1) a couple of days ago, and that is the one that has entered Debian repositories (sid) today. Thanks, Ginko, for this lovely game! Even though the game might not seem too appealing for some of the adult users of Debian or Ubuntu, maybe some of their kids will like it as much as I do. The game is quite quick, though, so very young kids might have trouble controlling the pony. I would suspect it might be all right for kids above 9 years old, but I haven t been able to test my hypothesis, so if you find out, please tell me :)

Justus Winter: cgroupfs is as cgroupy as it gets...

... at least until the cgroup interface is fixed. So, what can it do? So, what's missing? So, what's wrong with Linux cgroup API? Well for one thing the whole API is underspecified. Yes, there is Documentation/cgroups/cgroups.txt, but that is not a specification, that's a howto at best. Second, the notification API is not particularly nice:
To register a new notification handler you need to:
 - create a file descriptor for event notification using eventfd(2);
 - open a control file to be monitored (e.g. memory.usage_in_bytes);
 - write "<event_fd> <control_fd> <args>" to cgroup.event_control.
   Interpretation of args is defined by control file implementation;
Seriously? There is a POSIXly way to pass file-descriptors around, but smashing the decimal representation of it into a string is not the way to do that. Linux gets away with this hack because the kernel knows the process who wrote(2) that string in the first place, parse the string into an integer and look it up in the table of file descriptors for that process. Now the trouble for cgroupfs is, that it is not the kernel and even if it were, it wouldn't solve the problem because on Hurd there are no file descriptors (well there are, but that's only to appease all the POSIX programs out there). Instead Hurd has ports, and you can send messages to ports, and this is pretty much everything that you can do on a Mach system. Reading a file works roughly like this:
  1. You open a file and get a port X.
  2. You send a message like "I'm like really interested in the first Y bytes of that file" to X.
  3. Whoever has the receiving end of X (probably the one who gave you X in the first place) answers your request.
Ports look pretty much like file descriptors, they are (usually small) integers, you can make them, destroy them, pass them around easily (yes, ports are first class objects in the Mach messaging system). Everything is implemented atop of this mechanism. It is transport-agnostic, the other end could be on another machine and you wouldn't even know. You can create proxies or filters (in fact, that is exactly how the firewall eth-filter is implemented). It's beautiful and extensible at it's heart, like Lego bricks. So if X were a port to e.g. memory.usage_in_bytes and the cgroups interface would be less braindead^W^Wmore carefully designed so that on Hurd it could be transported like ports usually are, then cgroupfs could in fact use port X' to look up which file the caller is interested in (this is possible because cgroupfs was the one handing out the port in the first place) and generate notifications for that file. This is not possible when X is "serialized for transport" using sprintf because port names are specific for each process, so X != X'. The kernel would do the translation while sending the message, but it obviously cannot do that if the number is carried in a character array. I'm not sure what I'm going to do next week. The gsoc timeline suggests a soft-pencils-down, time to scrub code and write documentation, not sure that this is applicable to me as I have pushed most of my work upstream as early as possible. I guess I will nag Samuel so that he merges the outstanding patches and continue working on my gnumach patch.

Daniel Pocock: Calendar and Contact data with free software in the Smartphone era

Since the rise of smartphones, first Blackberry, then iPhone and now Android, the world at large has had a convenient option to fall back on for management of contact and calendar data. The data itself seems somewhat trivial: many of us software developers have worked with a plethora of obscenely more obscure schemas. It is unlikely that most people would ever have to perform an operation on their address book that requires a statistical toolkit like the R project or any of the other advanced tools we now have for data processing. In terms of data, addresses and phone numbers can appear quite boring. So why hasn't the free software community mastered it and cloud providers are currently running rampant hoovering up the data of billions of users? Ruling out LDAP One problem that people have faced is the existence of different standards. LDAP has quite a high profile and appears to be supported by default in many desktop applications. For this reason, it tends to grab people's attention but after a close look, many people find it is not the right solution. LDAP tends to work well for organisations where the data is centrally managed and in situations where fine-grained access control is not required. LDAP servers often need a capable UNIX administer to manage effectively. To top it off, LDAP itself does not provide an effective solution for two-way synchronization of data with remote and mobile users. The emergence of SyncML, CalDAV, GroupDAV and CardDAV Addressing this final problem (two-way mobile/portable device sync) has brought on the development of further standards. Initially, SyncML (now known as Open Mobile Alliance Data Synchronization and Device Management) grabbed significant attention, partly because it had been included in some of the first generation of smartphones from once-prominent vendors like Nokia. Open source server solutions for SyncML did not gain traction though. The most prominent is Funambol, however, it is not packaged in any major Linux distribution and it requires some basic Java knowledge to maintain it effectively. Like LDAP it appears to be most comfortable in enterprise use cases and doesn't appeal to the people who run a private mail and web server. SyncML has further suffered with the lack of support in more recent generations of smartphones. WebDAV was formalised in 1999 and there are now various derivative standards such as CalDAV and CardDAV which use WebDAV as a foundation. However, standards didn't arrive at this place directly: CalDAV only appeared as RFC 4791 in 2007 and CardDAV as RFC 6352 in 2011. In the meantime, some vendors experiemented with an alternative called GroupDAV. GroupDAV is largely obsolete now and GroupDAV v2 is a subset of CalDAV and CardDAV. Finally gaining traction The good news is, CardDAV and CalDAV are gaining traction, in no small part due to support from Apple and the highly proprietary iPhone. Some free software enthusiasts may find that surprising. It appears to be a strategic move from Apple, using open standards to compete with the dominance of Microsoft Exchange in large corporate networks. Both protocols are supported natively in iPhones and Mac OS X. More exciting, Apple releases their Contacts Server and Calendar Server as open source software, making it easier to look under the hood and observe any deviations from the standards. Apple's adoption of CardDAV and CalDAV has been accompanied by a similar effort in free software, however, support is more patchy. The Mozilla Lightning calendar plug-in for Thunderbird (known as the iceowl-extension plugin for icedove on Debian) has supported CalDAV for some time. In contrast, Thunderbird's address book lacks native support for CardDAV and has other limitations (discussed in detail below). Limited CardDAV support in Thunderbird can be achieved using the third-party SOGo Connector, which is not just for ScalableOGo server. The Evolution PIM appears to support both CardDAV and CalDAV relatively smoothly and appears to be more functional and easier to get started with. Free software servers are also emerging. DAViCal, despite the name, supports both CardDAV and CalDAV. It is a good solution for people who have an existing mail server and don't want to transition to an all-in-one groupware suite. Packages available in Debian make it easy to deploy and get started. It is a PHP solution, using a PostgreSQL database underneath. For people who do want a groupware suite, CardDAV and CalDAV are supported in products like ScalableOGo and Zimbra. For people who don't want a server at all and just want to sync between a desktop and a mobile device, there are options like Calypso. Mobile support: the missing link A number of people have blogged about using the Android apps CardDAV-Sync and CalDAV-Sync to complete the picture. I've tested these myself and verified that they work. However, there is one vital thing missing: the source code. Nobody knows what is inside these apps except the developer. That makes it impossible for other developers to troubleshoot them and improve them and it also makes it impossible to be certain that they do not leak information. An alternative is the aCal app for Android. Despite the name, aCal can synchronize the address book as well as the calendar. However, in my tests against DAViCal, I found that the addressbook can only be synced once and that on subsequent syncs, it fails to discover any new server-side contacts. aCal source code is available on gitorious.
aCal appears to be a compelling solution for Android users, hopefully the contact sync will be fixed soon. It only syncs once after installation and then fails to detect any new contacts. Thunderbird's limitations It has already been mentioned that the Thunderbird address book can be synchronized to a CardDAV server by adding the SOGo Connector plugin. However, this is not a perfect solution. The plugin itself is fine: the SOGo web site describes the connector (as opposed to the SOGo Integrator plugin) as a portable solution: in other words, it is vendor neutral, compatible with any standards-compliant server. The first limitation is that it is necessary to duplicate the setup process for both the calendar sync and the address book sync. This is just a question of convenience.
The SOGo Connector plugin configuration window is very trivial. I recommend using read-only mode to avoid the problems with multiple email addresses or telephone numbers. More serious limitations exist in the address book itself: if the CardDAV server has more than 2 email addresses, they will not appear in the Thunderbird address book. Changing an email address on the client side will result in loss of those additional email addresses on the server-side. The Mozilla team initially committed to rewrite the address book from the ground up but it is not clear whether that work is still in progress. DAViCal confusion I had been using DAViCal for quite some time as a CalDAV server before I recently decided to try and use it for CardDAV as well. As a calendar server, it had been working fine. I have a small number of users with some private calendars and also some shared calendars. When I installed the SOGo Connector plugin and went to setup the client-side configuration, it asked me for the server URL. Having used CalDAV already, I simply tried copying the existing CalDAV URL, which is of the form https://myserver.example.org/davical/caldav.php/daniel/home The URL was accepted and I could create address book entries on the client side. No errors appeared. I then tried setting up the same URL as an address book in Evolution. This time, I found I was able to create contacts and sync them to the server and access them from another copy of Evolution. It appeared DAViCal was working fine and I felt that Thunderbird or the SOGo Connector was somehow broken, because the Evolution clients were syncing but Thunderbird was not. The lack of any errors from the Thunderbird GUI further undermined my confidence in the SOGo Connector. Nonetheless, I decided to dig a little deeper, looking through the DAViCal source code. This brought me to the realisation that each URL could be flagged as either a calendar resource or an address book resource. This had not been obvious to me before. Evolution had effectively been storing the contact entries into a calendar URL and DAViCal had been happily accepting them. SOGo Connector, querying the URL, appears to have received both calendar and contact records from DAViCal and given up.
Here is where I configured the extra resource for an address book store under my user account in DAViCal In the DAViCal web management console, I created an extra resource for my user ID and set the appropriate checkbox to mark the resource as an address book. Then I put this resource URL into all the clients and found that they were all able to sync with each other. DAViCal's maintainer has explained that this is a limitation in the CardDAV/CalDAV protocols: technically, it is valid for the client to store the wrong type of objects in each URL. Clients need to be more resilient when they stumble across objects they don't expect in a particular resource. Conclusion A complete free software based contact-management solution can be easily deployed using any of the servers mentioned above, particularly if Evolution is the preferred client software. Using Thunderbird as a client, it can also work, but attention is needed to avoid data loss on sync. Thunderbird could also be used in a read-only mode to access address books maintained with Evolution. Mobile support is still patchy on Android. The closed source solution (CardDAV-Sync and CalDAV-Sync) appears to work and a promising open source close, aCal, appears to be "nearly there". Some people would obviously prefer to use a standalone contact/calendar server, while others may prefer an all-in-one groupware suite. Both approaches appear to be feasible, but the standalone contact/calendar server appears to offer a more compelling way to start quickly and have minimal administrative overhead, particularly due to the fact that DAViCal is available in a supported package.
This is the contact server configuration window in Evolution. It offers slightly more options than SOGo Connector in Thunderbird and appears to be a more finished solution overall.

MJ Ray: Upgrading to Debian 7 wheezy

So I have upgraded my main workstation at last, but there are a few things I wanted to figure out. Some of these I found an answer to, but others I haven t and some answers open more questions: Where did debian-menu.menu go and why doesn t lxlauncher have the Debian menu by default anyway? Answer: It seems this is a combination of debian bug 333848 and bug 722563 which I reported yesterday in that lxlauncher doesn t use the debian menus. Workaround for now: install menu-xdg and hack the lxlauncher menu file. Why don t my xterm shells load .profile any more? Did I bodge that before without making a note? How do I stop kernel or initrd or whatever it is configuring the network with the dhcp settings at boot time instead of the better wicd ones that are available once the system is up? Continuing with networking, how did network-manager (which doesn t cope with my network configuration) get installed again? Answer: debian bug 645656 explains that debian is simply following a silly decision taken upstream and refers to attempts to make network-manager optional as the Crusade . Workaround: remove gnome and install its dependencies like gnome-core directly instead. Drawback: any new dependencies of gnome will have to be installed manually. Why is the audio capture volume at 10% by default and what s the best way to change that? I suspect this might be 682731 but that bug is untouched in over a year. I m tempted to remove pulseaudio, but gnome-core Depends pulseaudio so this is yet more hardware-breaking caused by gnome dependency changes and the only way out of this one would be to remove gnome-core and install its dependencies. Or maybe I should just give up and finally remove gnome because its packages have jumped the shark. I installed gnome to make the system easier to use and it s broken it in at least two ways now. One question that makes me hesitate is: why is gdm3 faster to start than lightdm?

12 September 2013

Gunnar Wolf: What is to be an author? How much code does grant you authorship? #encirc13

I signed up to take part of an online, massive course Arte y cultura en circulaci n: crear y compartir en tiempos digitales (The circulation of arts and culture: Creating and sharing in digital times). As the activities of us the attendees are to be published by each on their personal blog/space, I will be publishing them here. I hope they are interesting to some of my other regular readers. I am late in joining, and I should already be posting the second activity. Anyway, this text goes towards the first week's lesson: Qu es un autor: la (de)construcci n hist rica del concepto de autor a (What is an author? Historic (de)construction of the concept of authorship). Our homework is to find an example of a current discussion where the notion of authorship is discussed, and share it this way. And I'm somewhat in a hurry right now, so this will be hasty. But I didn't want to delay my (late) submission anymore! In the Debian Ruby mailing list, Hleb Valoshka asks basically how deep should copyright recognition be. Because, yes, while copyright attribution is simple (simpler, at least) in literary / artistic works, computer programs (and even more those developed following the Free Software distributed fashion) are harder to properly attribute. Is a two-line patch to a tens-of-thousands-of-lines project enough to warrant mention in the copyright file? Most of us would agree it is not. Few would contend this were the amount of changes to make up, say, even 1% of that scale of project. In the coding projects where I am the main author (which, yes, are usually very small I cannot recognize myself as a good programmer, and the size of my successful works proves me), I try to take care to mention explicitly each of the contributors, even if their contribution is quite small. But were I to lead a larger development work, with enough following to generate on the order of one submitted patch a day, would it make sense for me to follow up with such detail the authorship information? And as a minor side note: How much does the law require me? A very small patch can fix important functionality issues (think security or performance). If somebody sends a very small patch fixing an intellectually hard to grasp problem... I am sure it should be properly acknowledged! Anyway, I have to flee now. I am just dropping some ideas on the table. Of course, if anybody is willing to discuss them further, I'm interested in any debate that springs off it! (Meanwhile... I expect to spam you with this topic a couple of times. But it should not bug you much, as it's one of my usual blog topics!)

11 September 2013

Steve Kemp: I've always relied upon the kindness of strangers

Many thanks to Vincent Meurisse who solved my node.js callback woe. Some history of the blogspam service:
Back in 2008 I was annoyed by the many spam-comments that were being submitted to my Debian Administration website. I added some simple anti-spam measures, which reduced the flow, but it was a losing battle. In the end I decided I should test comments, as the users submitted them, via some kind of external service. The intention being that any improvements to that central service would benefit all users. (So I could move to testing comments on my personal blog too, for example). Ultimately I registered the domain-name "blogspam.net", and set up a simple service on it which would test comments and judge them to be "SPAM" or "OK". The current statistics show that this service has stopped 20 million spam comments, since then. (We have to pretend I didn't wipe the counters once or twice.)
I've spent a while now re-implementing most of the old plugins in node.js, and I think I'll be ready to deploy the new service over the weekend. The new service will have to handle two different kinds of requests:
New Requests
These will be submitted via HTTP POSTed JSON data, and will be handled by node.js. These should be nice and fast.
Legacy Requests
These will come in via XML-RPC, and be proxied through the new node.js implementation. Hopefully this will mean existing clients won't even notice the transition.
I've not yet deployed the new code, but it is just a matter of time. Hopefully being node.js based and significantly easier to install, update, and tweak, I'll get more contributions too. The dependencies are happily very minimal: The only significant outstanding issue is that I need to pick a node.js library for performing CIDR lookups - "Does 10.11.12.23 lie within 10.11.12.0/24?" - I'm surprised that functionality isn't available out of the box, but it is the only omission I've missed. I've been keeping load & RAM graphs, so it will be interesting to see how the node.js service competes. I expect that if clients were using it, in preference to the XML-RPC version, then I'd get a hell of a lot more throughput, but with it hidden behind the XML-RPC proxy I'm less sure what will happen. I guess I also need to write documentation for the new/preferred JSON-based API... https://github.com/skx/blogspam.js

Olivier Berger: Publishing RDF views for tastypie/django resources

Here s some documentation about a hack I ve been working on to allow publication of RDF views for Tastypie resources (in Django applications). While working on implementing support for RDF meta-data for descriptions of Debian packages for the Debian Package Tracking System rewrite, I ve tried and idetify which libraries/frameworks would allow to create some RDF views for Django model objects with a minimal effort. Ideally, this would save the hassle of writing code, and could just be a matter of mapping some Django model fields to proper ontology attributes. I haven t found an existing tool to do so, but it seemed to me that Tastypie could offer a nice starting point. Tastypie (I focused on v. 0.9.15 which is currently in Debian testing) offers some REST content-negociation support, and other niceties. This post is an attempt at documenting an initial implementation for my problem. I ve implemented it as some code in the example blog application described in the Tastypie documentation. Unfortunately, it s not yet a patch that could be applied to Tastypie, to add this as a standard feature. The code is available in my git clone of Tastypie, based on 0.9.15, and lies in the commit(s) between the example_myapp and rdfviews branches. It needs an up to date RDFLib to work (Debian s is too old, btw). It basically relies on the addition of a _rdf_mapping dict in the Django model objects, and an RdfModelResource as a base class for Tastypie ModelResources. To provide an RDF view for a ModelResource, it also requires to declare a particular MetaClass, and adding a few bits to its Meta subclass. The principle is that the dehydratation steps of the Tastypie system will replace fields values by their RDF objects counterparts, as either RDFLib Literals or URIRefs. Then the resuting Bundle will be converted to some RDFLib Graph. It is then just a matter of serialization of that Graph to turtle (I ve not added another format, but it should be pretty straightforward). There s a bit of hackish drak magic in the code, in particular when introspecting the Tastypie / Django resource / model objects to be able to dynamically add some dehydrate_FOO methods to perform the conversion to Literals or URIRefs. Custom dehydrateFoo methods can still be written to process the fields in particular ways. I ve added some LDP style paging along the way, so I hope it will be quite usable by LDP compatible tools. Here s the result : First, listing all resources : $ curl -H 'Accept: text/turtle' "http://localhost:8000/api/v1/entry/"
@prefix dcterms: <http://purl.org/dc/terms/> .
@prefix ldp: <http://www.w3.org/ns/ldp#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix sioc: <http://rdfs.org/sioc/ns#> .
@prefix xml: <http://www.w3.org/XML/1998/namespace> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
</api/v1/entry/#list> dcterms:hasPart </api/v1/entry/1/#post>,
</api/v1/entry/2/#post>,
</api/v1/entry/3/#post> .
Then, details of one particular post : $ curl -L -H 'Accept: text/turtle' "http://localhost:8000/api/v1/entry/1"
@prefix dcterms: <http://purl.org/dc/terms/> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix sioc: <http://rdfs.org/sioc/ns#> .
@prefix xml: <http://www.w3.org/XML/1998/namespace> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
</api/v1/entry/1/#post> a sioc:Post ;
dcterms:created "2011-05-21T22:46:38+00:00"^^xsd:dateTime ;
dcterms:title "Another Post" ;
sioc:content "MESSAGE: This will prbbly be my lst post. /MESSAGE" ;
sioc:has_creator </api/v1/user/1/> ;
owl:sameAs </posts/another-post#post> .

I hope this is usable, and welcome any feedback.

10 September 2013

Gergely Nagy: How to monitor a directory of logfiles?

One feature of syslog-ng PE is support for wildcard file sources, meaning you can give it a wildcard, like /var/log/apache/*.log, and it will automatically notice new files being created or old ones disappearing. This is a very useful feature which has been missing from the open source edition, and one that is being worked on, but is far from complete, and will not make it into syslog-ng 3.5. However, at least on GNU/Linux, there is a way to implement something similar, although in a fairly crude manner. Yet, it works surprisingly well in most cases. You only need to abuse a few things...The trick itself is pretty easy, and is based on the same underlying kernel feature that syslog-ng PE uses under the hood: inotify(7). What we will do, is use incron to monitor a directory, and create syslog-ng config file snippets whenever a file creation event occurs. We also set up syslog-ng to include these snippets, and reload the configuration after each event received. This way, we immediately notice newly created logfiles without the need to poll the directory, and with config snippets created on the fly, we do not modify the main syslog-ng configuration file, either.To do this, we first need to create an appropriate syslog-ng configuration:
@version: 3.5
destination d_all_logs  
  file("/var/log/all.log");
 ;
include "/etc/syslog-ng/conf.d/notify-*.conf";
The config itself is a dummy: it has no sources, no logpaths, just a destination, which we will use in the snippets. Next, we create the script that will create these for us! It takes a single command-line argument: the filename of the newly created logfile.
#! /bin/sh set -e CONFDIR="/etc/syslog-ng/conf.d" SNGCTL=/usr/sbin/syslog-ng-ctl id="$(uuidgen tr '-' '_')" conffn="$ CONFDIR /notify-$ id .conf" fn="$1" cat >$ conffn <<EOF source s_$ id file("$ fn "); ; log source(s_$ id ); destination(d_all_logs); ; EOF $ SNGCTL reload
This creates the config snippet with a unique filename and source name, adds a new logpath that binds it to the d_all_logs destination defined in the main configuration file, and reloads the configuration afterwards.Now, all we need is to tell incron to call this script. Assuming we saved it as /usr/local/sbin/syslog-ng-wildcard-notify, this is how the incrontab entry would look like:
/var/log/remote.d/*.log  IN_CREATE,IN_MOVED_TO /usr/local/sbin/syslog-ng-wildcard-notify $@/$#
And that's about it! Now whenever a new file gets dropped into the directory, syslog-ng will get notified. There is a downside, however: when a file gets deleted, the configuration is not removed. That part is not so simple, though: we want syslog-ng to finish reading the file before reloading the config, so that makes it a bit trickier. But if we don't care about that, we could just add an IN_DELETE event, and write another script that looks up which snippet is responsible for the file, delete it and reload. Doing that is left as an exercise for the reader!

Julien Danjou: OpenStack Ceilometer Havana-3 milestone released

Last week, the third and last milestone of the Havana development branch of Ceilometer has been released and is now available for testing and download. This means the end of the OpenStack Havana development time is coming, and that the features are now frozen. New features
Eleven blueprints have been implemented as you can see on the release page. That's one more than during Havana-2, but it's less than was planned initially, though we had a pretty high score considering the size of our contributors team. I'm going to talk through some of them here, that are the most interesting for users. Bug fixes Fifty-six bugs were fixed, though most of them might not interest you so I won't elaborate too much on that. Go read the list if you are curious. Toward our final Havana release With the feature freeze in place, we're now focusing on fixing bugs and improving documentation. I'll try to make sure we'll get there without too much trouble for the 17th October 2013. Stay tuned!

Petter Reinholdtsen: Recipe to test the Freedombox project on amd64 or Raspberry Pi

I was introduced to the Freedombox project in 2010, when Eben Moglen presented his vision about serving the need of non-technical people to keep their personal information private and within the legal protection of their own homes. The idea is to give people back the power over their network and machines, and return Internet back to its intended peer-to-peer architecture. Instead of depending on a central service, the Freedombox will give everyone control over their own basic infrastructure. I've intended to join the effort since then, but other tasks have taken priority. But this summers nasty news about the misuse of trust and privilege exercised by the "western" intelligence gathering communities increased my eagerness to contribute to a point where I actually started working on the project a while back. The initial Debian initiative based on the vision from Eben Moglen, is to create a simple and cheap Debian based appliance that anyone can hook up in their home and get access to secure and private services and communication. The initial deployment platform have been the Dreamplug, which is a piece of hardware I do not own. So to be able to test what the current Freedombox setup look like, I had to come up with a way to install it on some hardware I do have access to. I have rewritten the freedom-maker image build framework to use .deb packages instead of only copying setup into the boot images, and thanks to this rewrite I am able to set up any machine supported by Debian Wheezy as a Freedombox, using the previously mentioned deb (and a few support debs for packages missing in Debian). The current Freedombox setup consist of a set of bootstrapping scripts (freedombox-setup), and a administrative web interface (plinth + exmachina + withsqlite), as well as a privacy enhancing proxy based on privoxy (freedombox-privoxy). There is also a web/javascript based XMPP client (jwchat) trying (unsuccessfully so far) to talk to the XMPP server (ejabberd). The web interface is pluggable, and the goal is to use it to enable OpenID services, mesh network connectivity, use of TOR, etc, etc. Not much of this is really working yet, see the project TODO for links to GIT repositories. Most of the code is on github at the moment. The HTTP proxy is operational out of the box, and the admin web interface can be used to add/remove plinth users. I've not been able to do anything else with it so far, but know there are several branches spread around github and other places with lots of half baked features. Anyway, if you want to have a look at the current state, the following recipes should work to give you a test machine to poke at. Debian Wheezy amd64
  1. Fetch normal Debian Wheezy installation ISO.
  2. Boot from it, either as CD or USB stick.
  3. Press [tab] on the boot prompt and add this as a boot argument to the Debian installer:
    url=http://www.reinholdtsen.name/freedombox/preseed-wheezy.dat
  4. Answer the few language/region/password questions and pick disk to install on.
  5. When the installation is finished and the machine have rebooted a few times, your Freedombox is ready for testing.
Raspberry Pi Raspbian
  1. Fetch a Raspbian SD card image, create SD card.
  2. Boot from SD card, extend file system to fill the card completely.
  3. Log in and add this to /etc/sources.list:
    deb http://www.reinholdtsen.name/freedombox wheezy main
    
  4. Run this as root:
    wget -O - http://www.reinholdtsen.name/freedombox/BE1A583D.asc   \
       apt-key add -
    apt-get update
    apt-get install freedombox-setup
    /usr/lib/freedombox/setup
    
  5. Reboot into your freshly created Freedombox.
You can test it on other architectures too, but because the freedombox-privoxy package is binary, it will only work as intended on the architectures where I have had time to build the binary and put it in my APT repository. But do not let this stop you. It is only a short "apt-get source -b freedombox-privoxy" away. :) Note that by default Freedombox is a DHCP server on the 192.168.1.0/24 subnet, so if this is your subnet be careful and turn off the DHCP server by running "update-rc.d isc-dhcp-server disable" as root. Please let me know if this works for you, or if you have any problems. We gather on the IRC channel #freedombox on irc.debian.org and the project mailing list. Once you get your freedombox operational, you can visit http://your-host-name:8001/ to see the state of the plint welcome screen (dead end - do not be surprised if you are unable to get past it), and next visit http://your-host-name:8001/help/ to look at the rest of plinth. The default user is 'admin' and the default password is 'secret'.

Daniel Kahn Gillmor: Support privacy-respecting network services!

Support privacy-respecting network services! Donate to Riseup.net! There's a lot of news recently about some downright orwellian surveillance executed across the globe by my own government with the assistance of major American corporations. The scope is huge, and the implications are depressing. It's scary and frustrating for anyone who cares about civil society, freedom of speech, cultural autonomy, or data sovereignty. As bad as the situation is, though, there are groups like Riseup and May First/People Link who actively resist the data dragnet. The good birds at Riseup have been tireless advocates for information autonomy for people and groups working for liberatory social change for years. They have provided (and continue to provide) impressive, well-administered infrastructure using free software to help further these goals, and they have a strong political committment to making a better world for all of us and to resisting strongarm attempts to turn over sensitive data. And they provide all this expertise and infrastructure and support on a crazy shoestring of a budget. So if the news has got you down, or frustrated, or upset, and you want to do something to help improve the situation, you could do a lot worse than sending some much-needed funds to help Riseup maintain an expanding infrastructure. This fundraising campaign will only last a few more days, so give now if you can! (note: i have worked with some of the riseup birds in the past, and hope to continue to do so in the future. I consider it critically important to have them as active allies in our collective work toward a better world, which is why i'm doing the unusual thing of asking for donations for them on my blog.)

9 September 2013

Kurt Roeckx: State of encryption

With all the articles about the NSA going around it seems to be hard to follow what is still safe. After reading various things it boils down to:
I don't think this should really surprise anybody. There is also speculation that maybe the NSA has better algorithms for reducing the complexity of public key cryptography or that maybe RC4 might be broken. Those things clearly are possible, but it's not clear. Nobody has suggested that AES has any problems, and I think that's still safe to use. One of the question becomes what does properly done encryption mean. There are various factors to this and many applications using various protocols. SSL/TLS is the most used, so I'm going to concentrate on the parts in that. Bit sizes As far as I understand it, 80 bit of security is around what is currently the upper practical limit of a brute force attack. In 2003 NIST said to stop using 80 bit by 2015, in 2005 they said to stop using it by 2010. When using something like SSL/TLS you using various things each having it's own size. So it's important to known which is the weakest part. Public key encryption (RSA, DSA, DH) doesn't offer the same amount of security per bit than symmetric keys. The equivalent sizes I find are:
Symmetric Public
80 1024
112 2048
128 3072
256 15360
There are other sources that have other numbers, but they're similar So this basically means you really should stop using 80 bit symmetric and 1024 bit public keys and that your public key really should be at least 2048 bit, and should maybe even consider going to 3072 bit. There seems to be a push to moving Diffie-Hellman (DH) to provide Perfect Forward Secrecy (PFS). There are many variants to this. We want to use ephemeral (temporary) keys. But it basically boils down to 2 variants:
With DHE you first negotiate a temporary DH key for this session (hopefully) using a random number on both sides, and then using that DH key to exchange a normal symmetric key for the session. This temporary DH key is what provides the PFS, assuming that this temporary key is thrown away. The drawback of this is that creating a sessions takes a lot more CPU time. The standard DH has the same security as other public key encryption. So we really also want to stop using 1024 keys for that. With ECDHE the key size needs to be the double of the symmetric size. This is also faster than DHE, so people want to move to this. There are various curves that can be used for this, and some of them are generated the the NSA and people have no trust in them. Some people even don't trust any of the curves. Most people seem to think that ECDH is the way forward, but then limit the curves they support. There are also hashes used for various things including signatures and MACs. Depending on how it's used the properties of the hash are important. They have 2 important properties collision resistance and preimage resistance. For a collision attack the upper limit is a little more than the output size over 2. But there are attacks that are known to give worst results. I found those numbers:
MD5 2^20
SHA-1 2^60
SHA-2 No known attack (so SHA-256 would have 1.2 * 2^128)
For preimage attacks I found:
MD5 2^123
SHA-1 No known attack (2^160)
SHA-256 No known attack (2^256)
So I think that MD5 is still safe when a collision attack isn't possible, but should really be avoided for anything else and SHA-1 should probably be considered the same. So you want to avoid them in things like certificates. SSL/TLS also use MD5 / SHA-1, but as part of a HMAC. I understand that those are not vulnerable to the collision attack but preimage resistance is important then and so are still considered safe. Randomness A lot of the algorithms depend on good random numbers. That is that the attacker can't guess what a (likely) random number you've selected. There have been many cases of bad RNG that then resulted in things getting broken. It's hard to tell from the output of most random number generators that they are secure or not. One important thing is that the RNGs gets seeded with random information (entropy) to begin with. If it gets no random information, very limited amount of possible inputs or information that is guessable as input it can appear to give random numbers, but they end up being predictable There have been many cases where this was broken. Ciphers There are various encryption algorithms. But since the public key algorithms take a lot of time we usually want to limit the amount we do with them and do most with a symmetric key encryption. There are basically 2 types of symmetric ciphers: stream and block ciphers. For block ciphers there are various ways of combining the blocks. The most popular was CBC. But in combination with TLS 1.0 it was vulnerable to the BEAST attack, but there is a workaround for this. An other mode is GCM but it's not yet widely deployed. This resulted in the stream cipher RC4 suddenly getting popular again. Some ciphers have known weaknesses. For instance triple DES has 3*56=168 bits, but only provides for 2*56=112 bits of security. Other attacks make this even less. The algorithms them self might be secure, but they might be attacked by other mechanism known as sidechannel attacks. The most important of that is timing attacks. If the algorithm has branches they might result in different amount of time taking in each branch. This might result in an attacker finding out some information about the secret key that is being used. It's important that the difference should be reduced as much as possible. This is usually not a problem if it's implemented in hardware. SSL/TLS Session A typical SSL/TLS session uses those things:
Important aspects of using certificates:
The client sends it's list of supported ciphers to the server and the server will then pick one. This is usually the first from that list that is supported by the server, but it might also be the order that is configured on the server that is used. Current status in software If you go looking at all of those things, you find problems in most if not all software. Some of that is to be compatible with old clients or servers, but most of it is not. If you want to test web servers for various of those properties you can do that at ssllabs. Based on that there are stats available at SSL-pulse. Apache only supports 1024 bit DHE keys and they don't want to apply a patch to support other bit size. Apache 2.2 doesn't offer ECDHE but 2.4 does. I don't know of any good site has lists which browser supports which ciphers, but you can test your own browser here. It lacks information about EC curves that your client offers. You can of course as look at this with wireshark. Mozilla currently has a proposal to change it's list of supported ciphers here. They currently don't have TLS 1.1 or 1.2 enabled yet, but as far as I understand the NSS library should support it. They should now have support for GCM, but they still have an open bug for a constant time patch. In general Mozilla seems to be moving very slowly with adopting things. Last I heard Apple still didn't patch Safari for the BEAST attack. For XMPP see client to server, server to server, clients.

For TOR see here.

Bastian Blank: Setting up Ceph the hard way

Components Ceph consists of two main daemons. One is the monitoring daemon, which monitors the health of the cluster and provides location information. The second is the storage daemon, which maintains the actual storage. Both are needed in a minimal setup.
Monitor The monitor daemons are the heart of the cluster. They maintain quorum within the cluster to check if everything can be used. They provide referrals to clients, to allow them to find the data they seek. Without a majority of monitors nothing will work within the cluster.
Storage The storage daemons maintain the actual storage. One daemon maintains one backend storage device.
Configuration The default config is understandable, but several things will just not work with it.
Monitor on localhost By default the monitor daemon will not work on localhost. There is an (undocumented) override to force it to work on localhost:
[mon.noname-admin]
 mon addr = [::1]:6789
The monitor will be renamed to mon.admin internaly.
IPv6 Ceph supports IP (IPv6) or legacy-IP (IPv4), but never both. I don't really use legacy-IP any longer, so I have to configure Ceph accordingly:
[global]
  ms bind ipv6 = true
One-OSD clusters For testing purposes I wanted to create a cluster with exactly one OSD. It never got into a clean state. So I asked and found the answer in #ceph:
[global]
 osd crush chooseleaf type = 0
Disable authentication While deprecated, the following seems to work:
[global]
 auth supported = none
Complete configuration
[global]
 auth supported = none
 log file = $name.log
 run dir =  
 osd pool default size = 1
 osd crush chooseleaf type = 0
 ms bind ipv6 = true
[mon]
 mon data =  /$name
[mon.noname-admin]
 mon addr = [::1]:6789
[osd]
 osd data =  /$name
 osd journal =  /$name/journal
 osd journal size = 100
[osd.0]
 host = devel
Installation This is currently based on my updated packages. And they are still pretty unclean from my point of view.
Setup All the documentation tells only about ceph-deploy and ceph-disk. This tools are abstractions that need root to mount stuff and do all the work. Here I show how to do a minimal setup without needing root.
Keyring setup For some reason even with no authentication the monitor setup wants a keyring. So just set one up:
$ ceph-authtool --create-keyring keyring --gen-key -n mon.
$ ceph-authtool keyring --gen-key -n client.admin
Monitor setup Monitor setup by hand is easy:
$ mkdir $mon_data
$ ceph-mon -c ceph.conf --mkfs --fsid $(uuidgen) --keyring keyring
After that just start it:
$ ceph-mon -c ceph.conf
$
OSD setup First properly add the new OSD to the internal state:
$ ceph -c ceph.conf osd create
$ ceph -c ceph.conf osd crush set osd.0 1.0 root=default
Then setup the OSD itself:
$ mkdir $osd_data
$ ceph-osd -c ceph.conf -i 0 --mkfs --mkkey --keyring keyring
And start it:
$ ceph-osd -c ceph.conf -i 0
starting osd.0 at :/0 osd_data $osd_data $osd_data/journal
$
Health check The health check should return ok after some time:
$ ceph -c ceph.conf health
HEALTH_OK
$

8 September 2013

Joachim Breitner: You-Say-First has dice

A while ago I wrote a small web application called You-Say-First. It allows multiple users to join a chat room where they can, besides chatting, enter moves that are only shown to everyone once everyone has submitted one. This way you can play games like Rock-Stone-Scissors or Diplomacy online, even if you want to play by your own strange rules that none of the standard, game-specific web sites support. Or even make up many kind of games on the spot!
I was approached by Jason Worley who said he liked the site, but would have a need for dice rolling to play the games he wants to play. So he gets dice rolling: Just say, in the chat, what kind of dice you want and the server will roll them for you and tell everyone about them. I tried to make it understand many variations of the command, so you can say roll a die, please , toss five coins or throw 3D20 . If the server does not react, try modifying your command. As always I m happy about feedback, bugreports and patches, and I hope that it is a useful tool for you.

7 September 2013

Daniel Pocock: Evacuating Australia

With the news that Australia's 100mbit (and potentially gigabit) fibre to the home is to be abolished and replaced with a cheap 25mbps service using Europe's ADSL hand-me-downs (which will be obsolete in 5 years when the deployment is finished), I've had quite a few emails from people asking just how easy it is to move abroad.
The acoustic modem has a nice retro look Word has got out that I've been spending some time in Switzerland enjoying the world-beating rate of IPv6 deployment. That is not the only reason to leave Australia of course, nor is it the only criteria for people choosing which country will give them a better deal than either of the two non-choices we had in Australia's elections. A free $20,000 for leaving Australia The first good news is that any recent graduate with a big bad ugly HECS bill is only obliged to pay this extra tax when they earn a salary from an Australian employer and file an Australian tax return. Australians who live abroad are not automatically obliged to file tax returns and are not slogged with this extra tax for having attended university. If somebody just graduated and has this hanging over their head, or if they've just started working and seen how much is being deducted on the payslip, moving abroad could make a lot of sense financially. If you have any moral qualms about this, just reflect on the fact that while the Government intimidates students with the feeling that they have to "pay" for fulfilling an education that makes them into a productive member of society, thousands of would-be athletes are given money with no obligation to repay it. The easiest route: a second passport One in four Australians has a parent born abroad. Most of those people qualify for a passport from their parent's country of birth. In some cases this also applies to people with grandparents born abroad. With the European Union now spanning 28 countries and guaranteeing the rights of any citizen from any country to live, work and study in any of the other 27 countries, a passport from any of the EU countries gives a lot of choices, including places where I have lived myself such as the UK, Ireland, France and Switzerland. Europe's citizen-friendly immigration In other words, an Australian with an Italian parent can go up to the Italian embassy in Canberra, collect an Italian passport and immediately fly into London and start working there with no bureaucratic delays, invasive and degrading medical checks or other Government interference in their life.
London's nightlife is popular for Australians. The UK started rolling out their high speed 21CN many years before Australia started the NBN. The rights also transfer automatically to all family members (spouse and children) - an Australian with a second passport is not restricted to living in Australia simply because they are married to somebody who does not have a second passport. As long as they arrive together at a European airport, they can simply show a marriage/birth certificate (in any language) and cross the border immediately, enjoy the same rights as a citizen, including work, study, healthcare and better broadband. No paperwork is necessary, no cottage industry for otherwise unemployable immigration agents, these are basic human rights for those living in Europe.
Eiffel tower: potentially part of the wireless network for France's new 4G internet, potentially two to four times the fixed-line speed planned for Australia's crippled broadband Uncle Sam wants more Australians Although it is not widely known, the US has a fast lane for immigration from Australia. The regular programs for skilled visas in the US are usually exhausted every year, whereas 10,000 "E3" class visas for Australians are never fully taken in any year. Surely it's not that easy to go to the US? It is. To qualify, an Australian only needs a university degree and a suitable job offer. Although the visa is for 2 years, it can be renewed and Australians who meet and marry an American have in various cases have gained full residence or citizenship. Anything is possible, you could even end up becoming a NASA Astronaut.
San Francisco, home to leading technology professionals and innovators from many countries. Some US cities such as Austin and Kansas City have let Google install fibre to the home, San Francisco could follow Working holiday schemes are not just for Britain Most Australians know somebody who went to London on a "working holiday" visa. These visas are not just for the UK and Australia has similar reciprocal programs with many other countries too. These rules for these schemes are rooted in the industrial age concept of workers with punch cards. Participants are typically limited to work 20 hours per week. Of particular interest for those in IT, such rules are hard to quantify for freelancers, web designers or creative people who earn their living charging fees per-project or undertaking management work without specified working hours. Although the visas are limited to 4 years, I've heard of Australians benefiting from the open borders of mainland Europe, spending 4 years in one country and then obtaining another 4 year visa for an adjacent country and travelling freely back and forth, building up a business and lifestyle of their own design. Singapore and Hong Kong Singapore and Hong Kong have become enormously popular destinations for Australians, particularly those with special skills in IT, finance, engineering and international business.
Singapore is respected as one of the cleanest and most orderly cities in Asia while embracing significant diversity in dining, shopping, leisure and lifestyle possibilities. They have recently boosted speeds to help people get a full 100MBit over fibre. A SIM card with high speed broadband charges SGD 7 (about $5) for 1 Gigabyte. These are not necessarily destinations that a graduate can easily move to. People are measured by their salary and the Government simply assumes that anybody earning a high salary is going to make a valuable contribution to society and they are given a visa in 1 or 2 weeks. Usually 3-5 years employment experience is required before making a move to one of these countries. The Not Ordinarily Resident (NOR) scheme gives some hints about the minimum salary level that ex-pats aim for, many earn significantly more with bonuses, stock programs and properly engineered tax-free offshore pension arrangements. These countries offer particularly low taxes (around 10%) and great technology. They have some rough edges too: like Europe, spouses are automatically given a visa too, but in contrast, their visa does not permit them to work unless they also work in a profession that is highly paid. Visas are issued quickly but they are also canceled very quickly if the job is terminated or the company goes bust. There have been various stories of people paying the typical 2-month deposit on an appartment in Singapore and then finding that it is forfeited when their job and visa are canceled and they have 4 weeks to leave. The bottom line is that people moving to these locations usually have a wide professional network and confidence to change their job quickly if necessary. Freestyle immigration and the visa run Not too long ago I was at a house party in Prague, Czech Republic, which is like a Paris of eastern Europe. Somebody introduced me to some Americans who I was told had similar interests in technology. They were operating an offshore web site and living very well. They had spent 10 years in Prague living on tourist visas. Authorities seemed quite happy for them to remain as long as they left every 3 months and spent 1 night abroad, a typical visa run. That could be as simple as a train journey to Vienna, Austria and an overnight stay.
Making a visa run in comfort and style with European railways. The Railjet train pictured includes free wifi, a lot cheaper than the $20 billion Tony Abbott plans to spend rolling out yesterday's technology in Australia When European countries abolished border controls, this system became much more convoluted for those who depend on the visa run. However, it remains common in many Asian and Latin American countries and many ex-pats live this way. Officials in many of these medium sized economies recognise the value of people bringing in money from offshore business (such as web sites) and using it to pay rent, food and local services. To make this effective, the ex-pats typically need to have an international private health insurance policy (high quality offshore policies can be found quickly with a search engine) and not aspire to work in a normal 9-5 job. Conclusion Young Australians might not want to just consider travel, it may even become unavoidable. The reality is, by 2020, one in three Australians will be a retired baby boomer and one way or another, whichever Government is in office will be unable to avoid taxing those of working age to pay for the health care and pension promises made to older generations. Serious expenditure on public infrastructure such as broadband or high speed rail appears to remain a pipe dream. Moving abroad, however, is just so much fun. Look at the way multinational companies make countries compete to attract their new factories and other projects: why should any individual citizen be any less demanding in choosing where they work, play and get the best value for whatever taxes they have to pay? Some people raise questions of patriotism or responsibility to the nation. Think again. The 200,000 Australians who live abroad or travel extensively make a significant contribution to Australia's international profile, engage in new trade initiatives, bring skills back to Australia and generally represent what our country is really about much better than some of those who are paid to do so.

Sylvestre Ledru: Include What You Use - Yet a new cool tool in Debian.

iwyu (Include What You Use) has just been accepted in Debian. This tool, based on LLVM/Clang, checks the headers of a C/C++ file. It will detect which include directives are useless, what could be improve and add comments to explain why #include are there. Let's take this C file:

#include <stdio.h>
#include <malloc.h>
#include <limits.h>

int foo(void)
fprintf(stderr, "an error\n");
return INT_MAX;

$ iwyu foo.c will return:

foo.c should remove these lines:
- #include <malloc.h> // lines 2-2

The full include-list for foo.c:
#include <limits.h> // for INT_MAX
#include <stdio.h> // for fprintf, stderr
---
Package on Debian

Uwe Hermann: Using mdadm to recover from a dead disk in a Linux RAID-1 array

2.5
Yes, it's that time of the year again. A disk in my desktop-replacement laptop with 2 disks and a RAID-1 has died. Time for recovery. This laptop has been running 24/7 for the last 3 years or such, so it's not too surprising that a disk dies. Surprisingly though, for the first time in a long series of dead disks, smartctl -a does indeed show errors for this disk. Here's a short snippet of those:
  $ smartctl -a /dev/sda
  [...]
  Error 1341 occurred at disk power-on lifetime: 17614 hours (733 days + 22 hours)
   When the command that caused the error occurred, the device was active or idle.
   After command completion occurred, registers were:
   ER ST SC SN CL CH DH
   -- -- -- -- -- -- --
   40 41 02 1f c0 9c 40  Error: UNC at LBA = 0x009cc01f = 10272799
   Commands leading to the command that caused the error were:
   CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
   -- -- -- -- -- -- -- --  ----------------  --------------------
   60 f8 08 20 c0 9c 40 00  41d+01:51:50.974  READ FPDMA QUEUED
   60 08 00 18 c0 9c 40 00  41d+01:51:50.972  READ FPDMA QUEUED
   ef 10 02 00 00 00 a0 00  41d+01:51:50.972  SET FEATURES [Reserved for Serial ATA]
   ec 00 00 00 00 00 a0 00  41d+01:51:50.971  IDENTIFY DEVICE
   ef 03 45 00 00 00 a0 00  41d+01:51:50.971  SET FEATURES [Set transfer mode]
  SMART Self-test log structure revision number 1
  Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
  # 1  Short offline       Completed: read failure       90%     20511         156170102
  [...]
The status of the degraded RAID array looks like this:
  $ cat /proc/mdstat
  Personalities : [raid1] 
  md1 : active raid1 sdb7[1]
       409845696 blocks [2/1] [_U]
  md0 : active raid1 sda6[0] sdb6[1]
       291776 blocks [2/2] [UU]
The [_U] means that one of two disks has failed, it should normally be [UU]. There are two RAID-1s actually, a small md0 (sda6 + sdb6) for /boot and the main md1 (sda7 + sdb7) which holds the OS and my data. Apparently (at first at least), only sda7 was faulty and got kicked out of the array:
  $ dmesg   grep kick
  md: kicking non-fresh sda7 from array!
Anyway, so I ordered a replacement disk, removed the dead disk (I checked the serial number and brand before, so I don't accidentally remove the wrong one), inserted the new disk and rebooted. Note: In order for this to work you have to have (previously) installed the bootloader (usually GRUB) onto both disks, otherwise you won't be able to boot from either of them (which you'll want to do if one of them dies, of course). In my case, sda was now dead, so I put sdb into its place (physically, by using the other SATA connector/port) and the new replacement disk would become the new sdb. After the reboot, the new disk needs to be partitioned like the other RAID disk. This can be done easily by copying the partition layout of the "good" disk (now sda after the reboot) onto the empty disk (sdb):
  $ sfdisk -d /dev/sda   sfdisk /dev/sdb
Specifically, the RAID disks/partitions need to have the type/ID "fd" ("Linux raid autodetect"), check if that is the case. Then, you can add the new disk to the RAIDs:
  $ mdadm /dev/md0 --add /dev/sdb6
  $ mdadm /dev/md1 --add /dev/sdb7
After a few hours the RAID will be re-synced properly and all is good again. You can check the progress via:
  $ watch -n 1 cat /proc/mdstat
You should probably not reboot during the resync (though I'm not 100% sure if that would be an issue in practice; please leave a comment if you know). Also, don't forget to install GRUB on the new disk so you can still boot when the next disk dies:
  $ grub-mkdevicemap
  $ grub-install /dev/sdb
And it might be a good idea to use S.M.A.R.T. to check the new disk, just in case. I did a quick run for the new disk via:
  $ smartctl -t short /dev/sdb # Wait a few minutes after this.
  $ smartctl -a /dev/sdb
  [...]
  SMART Self-test log structure revision number 1
  Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
  # 1  Short offline       Completed without error       00%        22         -
  [...]
Looks good. So far.

Ben Armstrong: Wifi roaming on the move redux

It has been nearly six years with a netbook and five since I last wrote about wifi roaming from the bus to stay on irc without a costly celluar link during the daily commute. Since then, some readers have asked me to share my refinements to the method in a followup post. So here it is. The software On the server: On the client: Putting it together: on the client Make sure if you have a wireless manager installed (such as NetworkManager) it is configured to skip your wireless interface, disabled entirely, or if possible, removed. Set up /etc/wpa_supplicant/wpa_supplicant.conf and /etc/network/interfaces for roaming, as per the instructions in /usr/share/doc/wpasupplicant/README.modes.gz. Don t forget to add yourself to the netdev group if you are not in it already. In /etc/wpa_supplicant/wpa_supplicant.conf, list common names of open networks. Normally the catch-all network that associates with any essid, i.e. the first stanza below, works well. However, occasionally the strongest signal is neither one of the common networks nor an easily accessible network (e.g. web portals), so having a list of common open networks helps to quickly select from among those instead. The more you travel, the more of these will discover and add. Just use reconfigure from wpa_cli to reload your edited list each time you add a new one.
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
network= 
        key_mgmt=NONE
 
network= 
        ssid="default"
        key_mgmt=NONE
 
network= 
        ssid="linksys"
        key_mgmt=NONE
 
...
Since you ll be using ssh repeatedly to connect and it has to be fast, make sure your server is set up to accept your key and use ssh-add so that you only have to enter your ssh key password once. You can tweak isc-dhcp-client to make connections faster. In /etc/dhcp/dhclient.conf, use:
backoff-cutoff 1;
initial-interval 1;
Here are a few scripts I wrote to facilitate quick roaming from one open AP to another and reconnect to irssi running in screen, to break a connection and try the next one, and to recover from occasional lockups (more about that later). ~/bin/screen_reconnect This is a script to reconnect continuously via ssh to a screen session:
#!/bin/sh
reset
while ! ssh -t 10.9.8.7 'screen -UDr' 2>/dev/null ; do echo -n "." ; sleep .1 ; done
Just substitute the IP of your own server here. Using an IP instead of domain name makes the connection faster because a DNS lookup is not required. ~/bin/wifi_reassociate This script closes any open ssh sessions and informs wpa_supplicant to attempt to connect again.
#/bin/sh
/sbin/wpa_cli rea
killall ssh >/dev/null 2>&1
~/bin/wifi_killall This optional, somewhat ugly script addresses an issue I hope you never have. On my ASUS Eee PC 1001PX, occasionally scanning stops. When this happens, and I have never figured out why, apparently ACPI events are blocked. At this point wifi becomes unusable and ACPI sleep is inhibited. By trial and error I found that if you bring down the interface, kill all network-related processes, and bring it up again, ACPI events are unblocked and wifi is usable once more (and any pending request to sleep will finally happen). The script requires sudo, and to use the openbox key binding, gksudo.
#/bin/sh
sudo ifdown wlan0
# in case any of these are hung
sudo killall dhclient3
sudo killall wpa_cli
sudo killall wpa_action
sudo killall wpa_supplicant
# in case any of these are *really* hung
sleep 1
sudo killall -9 dhclient3
sudo killall -9 wpa_cli
sudo killall -9 wpa_action
sudo killall -9 wpa_supplicant
sudo dhclient -r
sudo ifup wlan0
Openbox Since certain actions need to be performed repeatedly and quickly, it is useful to have hotkeys bound in your window manager to the scripts. In ~/.config/openbox/rc.xml, key bindings for <alt>-r to reassociate and <alt>-d to disconnect a hung connection would look like:
  
<keyboard>
  <!-- My keybindings -->
  <keybind key="A-R">
    <action name="Execute">
        <execute>/home/synrg/bin/wifi_reassociate</execute>
    </action>
  </keybind>
  <keybind key="A-D">
    <action name="Execute">
        <execute>gksudo /home/synrg/bin/wifi_killall</execute>
    </action>
  </keybind>
</keyboard>
Putting it together: on the server There is very little to do here. Just start screen, and start irssi in screen. Running screen on the client as well as the server means you should either bind the screen meta keys to a different key sequence on each system, or else learn to press meta twice to pass through meta to the server screen as needed. I use the latter approach. Alternatively, you could use a tabbed terminal on the client, or separate terminals per client process instead of screen. This is a matter of personal taste. Ready to roam Here is a typical setup for roaming on the bus: In a terminal (I use urxvt), first ssh-add, then start screen with these three processes running in separate virtual terminals: March of the dots Most of the commute, just enjoy watching the dots march by, waiting for a new connection. If you estimate a connection is unusable, press <alt>-r to reassociate immediately, giving the next network a chance. If the connection is already firmly established, this might not work on the first try. If the dots don t resume immediately, wait a bit and press it again. This might take a few tries. Changing selected networks on the fly Use wpa_cli when you need to do some fine-tuning of network selections on the fly. While normally you can just watch the march of the dots until a connection is acquired, sometimes you can improve your chances of connecting to a good network by manually controlling the selected candidate networks here. For example, by watching the speed of the bus relative to known good APs, you can predict which networks are more likely to succeed. Rather than connect to any arbitrary network, you might select a specific one by id, and then later when it goes out of range, revert to the original configuration, e.g.
> select_network 5
...
> reconfigure
You can use tab-completion in wpa_cli to type these commands quickly or else just abbreviate the commands. Another common scenario is when you pass through a business area with many captive portal hotspots. These rarely make good choices because they either require a password not known to you or else you can t click through I agree in time before the bus moves on. In this case, you might just disable the catch-all stanza and let the common open network stanzas you listed ( default , linksys , etc.) do the work:
> disable_network 1
Become a type ahead wizard While running, a continuous stream of periods fills the screen, which provides you with a highly visible cue that no available APs are in range. When the movement stops, you know a connection is being attempted. While waiting to connect, you can type ahead any comments you want to make in the current irssi window (taking care to remember which one you are in!) While having periods interspersed in what you type may be disorienting at first, you get used to it. There is a point when a connection is first established and ssh is accepting input, but anything you type can no longer be seen while you re typing. Depending on whether the connection was completely successful or not, what you type now may or may not finally be sent. For best results, only type ahead before the dots stop moving. Eventually you can become skilled enough at this to type ahead a comment in one channel, switch channels with /win # and continue typing ahead in the new channel, all buffered until the next few seconds (or even fraction of a second) of connection time. Fine-tune antenna direction with wavemon When the bus has come to a standstill, you may find wavemon useful to pull in a weak signal. Because wavemon has continuously updated signal level and link quality bars, you can use it to fine-tune the antenna position. Just turn your laptop until the bars are at their maximum. Captive portals I have not figured out how to do any automation for this, so it really is a crapshoot, as it is likely the bus has moved on by the time you ve managed to manually navigate the login through a captive portal. But in rush hour, you may have the luxury of time to connect to these as you pass them. I have recently learned about the CoovaFX Firefox plugin which automates logins to captive portals. I m going to give it a try to see if it helps. Update: I can t recommend this plugin, as it is not compatible with Iceweasel >= 23.0. Also, the standard it is based on, WISPr, appears to have an uncertain future. That, coupled with the fact that the plugin appears to not be open source means I m still looking for alternatives. Summary If all of this sounds a bit nuts to you, well, it probably is. But after half a decade enjoying free access to irc from the bus, it all seems perfectly natural to me! If you try this method and like it, please let me know in the comments. Likewise, if you have any improvements to the process or scripts, please share them!

Wouter Verhelst: Debconf Mechelen: first meeting (service announcement)

To all who might be interested: The first meeting for organizing Debconf in Mechelen will happen next sunday, at 3:00 PM, in the NixSys offices in the Battelsesteenweg 455e in Mechelen, Belgium. It looks like some of those who were interested might not be able to make it at that that time. If you're one of them, and you would like your availability to be considered for the next meeting, please send me an email with your preferences (that meeting should probably happen in about a month or so). (First point on the agenda: set up a mailinglist, to make this sort of communication somewhat easier ;-)

Next.

Previous.