I could probably write it myself, given time, but I'm guessing it's going to cost me a lot more time understanding the Ethereal API than it is going to cost someone else undestanding the NBD protocol. Which is really simple. So, here's a short description of the protocol; now if someone could write me an Ethereal dissector, I'd be very happy. And if you need it, just ask for more information.

"Negotiation" phase:

• Connection is opened by the client
• The server sends out the literal string NBDMAGIC to the client (which is the "INIT_PASSWD" in the code and in error messages), followed by the the 'cliserv_magic', 0x00420281861253LL. The NBDMAGIC is there to identify a server; the cliserv_magic may be updated when the protocol changes incompatibly.
• The server sends the size of the device as a 64 bit unsigned integer.
• The server sends 128 bytes of zeroes (discarded by the client currently; these are reserved for further use)

(whether or not the above are all in one packet is something I don't know and don't really care about; they're all done using different write calls, so can technically be in different packets)
After that, the client is expected to be happy with the connection (if it isn't, it'll just drop the connection without explanation).

Main data pushing phase:

(I just made that name up, give it a reasonable name please ;-)

• Clients sends a struct nbd_request to the server (which is defined in <linux/nbd.h>. The fields in that struct are:

  magic (32bit unsigned int)

  must be NBD_REQUEST_MAGIC aka 0x25609513 (again defined in <linux/nbd.h>); if it isn't, the server disconnects (this should really be handled better, but at the moment it isn't; and changing the protocol to do this might be less than trivial). It is possible that previous versions of the protocol used 0x12560953 instead (the header file mentions that as something not to be used, but I don't know where it comes from).

  type (32 bit unsigned int)

  is NBD_CMD_READ aka 0 (to read data), NBD_CMD_WRITE (to write data) aka 1, or NBD_CMD_DISC aka 2 (to request a clean disconnect).

  handle (char array of eight bytes)

  used to uniquely identify a request; this is later sent back in the reply, and can be used to associate a reply with a request.

  from (64 bit unsigned int)

  the offset in the export where the read or the write should start.

  len (32 bit unsigned int)

  the length of the requested read or write.

• If struct nbd_request.type==NBD_CMD_WRITE, the client will then send struct nbd_request.len bytes to the server, containing the data it wants to see written.
• Server sends a struct nbd_reply to the client:

  magic (32 bit unsigned int)

  functions similar to struct nbd_request.magic, only its value is supposed to be 0x67446698. Analoguous to struct nbd_request.magic, there's a value, possibly from a previous version of the protocol, which says 0x96744668 instead.

  error (32 bit unsigned integer)

  is zero, unless an error occurred

  handle (char array of eight bytes)

  the exact same value that was received in struct nbd_request.handle

  If struct nbd_request.type==NBD_CMD_READ, the server will then send struct nbd_request.len bytes to the client, containing the data it read.

As you can see, a very simple protocol really... now if only somebody could write me a dissector—having to dissect it by hand isn't very funny, really.

According to this page, my blog is worth $2,822.70. Now, I have no idea what metric they use to determine that, but that’s not really important. What is important is that you freeloaders start paying up. For far too long, I’ve put my blood and sweat into producing this fine verbiage, without any sort of compensation for my labor. As we all know, if humans aren’t compensated for their art (and trust me, this blog is pure art), they have absolutely no incentive to continue creating it. So, I demand payment in return for my hard work, or Toby gets it! I hereby announce that this blog is for sale. Once purchased, you may claim ownership over any past and future ideas, anecdotes, and pictures presented on here. I will entertain the best offer. I am such a whore. For the humor impaired: yes, this is a joke.

Weblog Software

I’ve been keeping an online journal over at LiveJournal since 23rd February 2002. To support them I actually bought a “Permanent Account” a while back, an example of putting money behind companies and/or services which are useful - even when it isn’t required. After getting into the swing of things there I started the blog which is used upon Planet Debian and have also been enjoying that. Livejournal is a nice service, and they have great software powering the site. The Livejournal code is available and it is reasonably simple to install. (I’ve setup a few installations of the codebase, and played with them. I admit I’ve never run it in anger). Obviously it is completely overkill to run your own installation of Livejournal - unless, perhaps, you’re a school or a large company. So for my personal blog I use Wordpress. I don’t particularly enjoy using wordpress. To be honest I need only four features/abilities of weblog software:

• The ability to create posts.
• The ability for users to be able to leave comments upon entries.
• The ability to search previous entries.
• The ability to export the blog entries as a feed.

Everything else is utterly overkill. The downside of using Wordpress is threefold:

• Because it is popular there are multiple bots crawling the web and leaving spammy comments.
• Because it uses some funky addressing system any replacement would leave lots of broken links.
• It has historically had a couple of security issues.

I’ve been telling mysefl for the past few months:

• This software has got to go.
• I’ll wait until there is another security hole discovered.

So .. HTML::Template & CGI::Application will at some point come to the rescue. I’m just not sure when. I guess this means I’ve not yet written a bloggin system…. I want to only cover the basics:

• Multiple users.
• Multiple catagories/tags.
• Comments with a sane anti-spam system.

PS - The Sisters of Mercy rock.

Apathy

I’m sleeping a lot more than I should be which is making me pretty apathetic at the moment. Even checking email is a real effort. Let alone replying.

Last Post

As previously mentioned I’m utterly unfranchised with WordPress. This will be my last post using it. All future updates will be made using Chronicle, a simple Perl-based application I’ve been working on for a while. (The best part is the name!) So far I’ve got code to :

• Import old wordpress entries into a new database.
• Add new posts.
• Show archives.
• Do searching.
• Add new entries.

TODO:

• Import old comments.
• Add code to handle displaying/editting/deleting comments.
• Add some mod_rewrite magic so that all my old links still work.

The application is pretty nice, comprised of several perl modules and built using the CGI::Application base-class. I realise this contradicts the apathetic stanza above …

Here's an easy recipe to filter those annoying SSH scanners at your firewall:

iptables -A FORWARD -i ethLRZ -p tcp --dport 22 -m state --state NEW \
        -m recent --set --name SSH
#$iptables -A FORWARD -i ethLRZ -p tcp --dport 22 -m state --state NEW \
#       -m recent --update --seconds 60 --hitcount 5 --rttl \
#       --name SSH -j LOG --log-prefix "SSH_brute_force "
iptables -A FORWARD -i ethLRZ -p tcp --dport 22 -m state --state NEW \
        -m recent --update --seconds 60 --hitcount 5 --rttl --name SSH -j DROP

This configuration will allow up to 5 SSH connections in a 60 second timeframe. This will usually make SSH-scanners go away after their 5th retry, and seriously slow them down otherwise. If you have users who "rightfully" need to do more SSH connections, make them use some VPN, a "safe" source-IP-range, whatever. To protect your firewall host, use INPUT instead of FORWARD. Note that you can also implement port-knocking with the recent match.

Since interviewing the creator of Ruby on Rails, David Heinemeier Hansson, I've been thinking a little more about what he calls "opinionated software." That handy little phrase seems to sum up what attracts me to a lot of technology, at all levels. What I like Free software--it's not really economics that drew me to free software, but the principles and aims. When you release or use free software, you're exercising your right to free speech and adding your voice to an argument. Desktop environments--GNOME 2.0: The move from GNOME 1.0 to version 2.0 was highly opinionated (Mac users, think OS 9 to OS X). Complexity of user interface was thrown out in favour of sensible defaults. Web standards--CSS, RDF, XHTML: not only is there a right way to do things, but it turns out to be better too! I don't automatically fall in love with everything that emerges from the W3C, but some things are so obviously the right way it's impossible not to love them--warts and all. Muine music player--this player throws out the paraphernalia of iTunes clones and gets you close to the music. Queue, play. What I want to do. Epiphany web browser--a simple text bar does for bookmark lookup, search and URLs. The bookmarks work like del.icio.us tags. Despite the lure of Firefox extensions, simplicity and efficiency keeps me with Epiphany. Make friends and enemies The thing about opinions is that people are inclined to disagree with them. Opinions won't please everybody and, being opinionated, you will never carry everybody with you. But opinions give you something with which to scythe through indifference. In both my writing and software development work taking a particular stance enables me to create interest, and gives direction to my projects. I tend to get bored easily, and if I'm bored, then my readers and users will never make it. Being opinionated in software development is easier when there's a strong leader, or it's a one-person project. It's more of a dilemma if you're running a free software project with multiple contributors. Such projects need such contributions, but run a risk of losing focus and style. I think that there we need to be brave again. If being opinionated means losing potential users, it also means losing potential contributors. Opinions help in dealing with users, too. It's obvious that the mere act of listening to users isn't good enough. They often won't say what they mean, or their requests mask a deeper issue. How often have you given your boss exactly what he asked for, to find it wasn't what he wanted? A combination of reading-between-the-lines and having opinions helps cut a way through the noise of user feedback. Stirring the pot Here are a few wishes for some free software projects that I want to see succeed further, but which really need to get opinionated to be effective. These projects aren't failing, but they're not reaching the potential I know is there. Each of them are infrastructural in some way, and that's probably where the temptation to avoid being opinionated is strongest. But here's the genius: the rewards of being opinionated here are high. Ruby on Rails is the existence proof.

• Beagle, live search technology for GNOME. Needs to decide where its primary use will be, and get on and hit the 1.0 point for that use only. Choose one or two neat applications for integration, and be so utterly cool that every other app author flocks to integrate too.
• Synchronization. First MultiSync, then OpenSync, have rumbled on for ages, without ever being truly effective for any particular device. My take: choose a few "must-have" target devices--say, iPods and Nokia phones--and make them work completely. Nearly-but-not-quite pleases nobody.
• Mono. Mono's great, but where's the story? There's loads of potential in the people and the ideas behind the project, but playing catchup with Microsoft isn't very inspiring. Early projects like Muine helped Mono's cachet here, but we need more. My personal favourite option would have been a completely free alternative to ASP.NET for web applications. (I've seen a project called MonoRail, but don't know if it's the one...)

What's better? A handful of users with the "right" idea about how fundamentally important you are to everything, or loads of users who only see a portion of the capabilities of a project? Copy the experts It may seem like cheating to copy Apple's lead, but a lot of what they've done in OS X is to take a stand on what's useful and not. On many occasions this has inconvenienced people, but it's not eroded that special lure that Apple products hold. There's not harm in asking "What have Apple done?" with some of our projects. Another poster-child for being opinionated is Ubuntu Linux. Technically, it's only marginally better than other Linux distributions. Strategically, it's miles ahead. Its founder, Mark Shuttleworth, had very solid opinions about where Ubuntu should go, and pursued them. Over time, these have pervaded the product itself, but at the start it was the message and the style that made the difference. Opinions not expressed are rarely effective.

This posting is part of my Finnish criminal court series. See the introductory essay for more information and an overview of the Finnish criminal court system. The following is a fictional account of a criminal case, one of the most frequent ones, written in the form of a play script. A PDF is also available. There is also an OpenOffice-generated XHTML renderition, which may be more pleasant to read than the version below (but lacks most of its logical markup). DRIVING WHILE INTOXICATED
A glimpse of Finnish criminal justice
Part One Dramatis personae
Presiding judge
Prosecutor
Defendant Extras
Three lay judges
Court reporter Sets
Courtroom
Courthouse lobby DISCLAIMER This is a work of fiction. Although this play is based in general terms on real life, all the specific characters and events portrayed or described in this play are fictitious, and any resemblance to real people or events is purely coincidental. The verdict presented is fictitious, and while it is based on the author's understanding of the law, it may be erroneus. No legal advice should be inferred from this work of fiction. Scene One: And so it begins The courtroom is about the size of a large school classroom, and shaped as a rectangle in terms of floor layout. The judges' bench, a long wooden table, lays near one of the short walls; the space between the bench and the wall is enough to give the judges and the court clerks comfortable room to sit and to move behind the sitting judges. At the center of the bench is the seat of the presiding judge, who has easy access to a telephone, a public address system and a panel for controlling the LED display visible in the lobby above the courtroom door. To his or her right is the court reporter's seat in front of a PC and in control of recording equipment. To the reporter's right there is room for one additional individual. To the presiding judge's left, there are three chairs for the lay judges. If one were to stand directly opposite the presiding judge, on the other side of the bench, one would find an open Bible. In front of the bench, disregarding the Bible, there is some room, approximately two meters or so. Along each long wall there is a small table, and those two tables face each other, and are therefore turned sideways from the bench's point of view. By the desk on the bench's right is the seat of the prosecutor, and the other, facing desk is reserved for the witness being examined. Facing the bench, separated from it by that empty space, are two rows of tables with chairs. Beyond them there are two rows of seats for the audience without tables; the farther row is right at the other short wall. The long wall to the judges' right contains one double door leading to the court building lobby, and the other long wall contains two sets of double doors at each end of the wall, leading to a part of the building not accessible to the public. We'll call the door closer to the bench the judges' door. The PRESIDING JUDGE is a 50-something man dressed formally in black and blue. The three LAY JUDGES are more casually but still essentially formally clothed. LAY JUDGE ONE is a sixty-something male, LAY JUDGE TWO is a female in her fourties and LAY JUDGE THREE is a female just under thirty. The COURT REPORTER is a female in her fifties, also dressed formally. It is 9:00. The four JUDGES and the COURT REPORTER are sitting on the bench. Each JUDGE has papers in front of him or her, the PRESIDING JUDGE's stack being by far the thickest. The PROSECUTOR, fourty-something man, enters through the judges' door, carrying law books and a large stack of paper. He sits down at his desk. PRESIDING JUDGE (Nodding to the prosecutor.) Okay. Let's start. (Enters codes on the LED machine. Then picks up the public address microphone.) The criminal case 'prosecutor against Matti Meik l inen' begins in room three. Parties to room three. Scene Two: Main hearing The DEFENDANT enters through the double door from the lobby. He feels a little lost. PRESIDING JUDGE (To the defendant:) Matti Vihtori Meik l inen? (DEFENDANT nods.) Please, take a seat in the front row. (He waits for the defendant to comply.) This case will be tried as follows. First, the prosecutor will read the charge. You will then respond by telling us if the charge is correct or not, that is, if you in fact did commit the acts specified in the charges, and if you did not, or the charge is only partially correct, you should state briefly why the charge is in error. The prosecutor will then present the case, he will describe us his understanding of the events in question. When he is finished, you will again have opportunity to state if the prosecutor is correct or not, and if not, in what way is he wrong. If there is disagreement between you and the prosecutor, we will hear the evidence. After that, the prosecutor will state his or her concluding opinion of the case as presented here. Again, you will have the opportunity to respond. We will conclude with discussion of what punishment, if any, would be appropriate. The court will then discuss the case alone, and will finally announce the verdict. Is this understood? (DEFENDANT nods.) Okay. Prosecutor? PROSECUTOR (Reads the charge from a paper everyone has in front of his or her.) I demand punishment to Matti Vihtori Meik l inen for driving while intoxicated. Matti Vihtori Meik l inen had on the first of May this year operated his car while his exhaled air contained 0,597 milligrams of alcohol per liter. Matti Vihtori Meik l inen's driving license must be suspended. PRESIDING JUDGE (To the defendant.) Is the charge correct? DEFENDANT Yes, it is. PRESIDING JUDGE Okay. The prosecutor will now tell us the circumstances and events in detail. PROSECUTOR Thank you. It was to my understanding late evening on May day, and the defendant had come from a bar and was apparently trying to move his car to a cost-free parking space. A police patrol happened to observe this and the defendant had apparently been a little shaky in his walking, so the officers gave him a breathalyzer test, which showed significant alcohol content. The defendant was brought in the station where a precision test was performed. The first reading was 0,597 and the second reading was 0,617; the mean of these two is 0,607, and after performing the confidence subtraction, the reading 0,597 milligrams per liter was obtained. PRESIDING JUDGE (To DEFENDANT) Well then, is what the prosecutor just said true? DEFENDANT (Speaking in a colloquial manner and very somberly. He is ashamed and wishes this would be over soon.) Yeah, it is. PRESIDING JUDGE All right. It seems to me that there are no points of disagreement, so we don't need to hear witnesses. (The PROSECUTOR nods.) Okay, it's time for the concluding statements. I think we will hear consequence opinions at the same time. Prosecutor? PROSECUTOR Thank you. I don't think there is much to say. The case has been shown beyond reasonable doubt, in my opinion. We have the precision test readings and the defendant's admission. As to the penalty... the amount of alcohol we are talking about is just below the limit for serious intoxication, so this is no small deal. On the other hand, the circumstances were not very dangerous. I think suspended imprisonment could well be considered, though a stiff fine would probably be more appropriate. The driving ban must be fairly long, obviously. PRESIDING JUDGE All right. (To the defendant.) Do you have anything to say before we adjourn to consider the case? DEFENDANT Well, I suppose it's clear that something is going to be given to me. I prefer a fine, naturally. PRESIDING JUDGE Naturally. The court will consider all options. Do you have any special requests concerning a driving ban? DEFENDANT As short as possible, of course. PRESIDING JUDGE Do you have a particular need for a car, do you need it for work, for example? DEFENDANT No. PRESIDING JUDGE Okay. Now, your income. We have here the last completed tax report for you, it says you earned 500 euros a month and had no children. Has this changed? DEFENDANT No change. PRESIDING JUDGE Okay, thank you. If you will then go outside to wait while we consider this case, thank you. The DEFENDANT exits to the lobby, the PROSECUTOR exits through the judges' door. Scene Three: Deliberation We are now in the lobby outside the courtroom. There are many doors that lead to many courtrooms. Above each door, attached to the wall, is a display showing the current case number; there is also a light saying do not enter, which is lit for most of the rooms, including Three. Also there is a numeric display which is out at the moment for most of the rooms. For some rooms, they indicate when the session will resume. Each courtroom door also has a paper pasted to the door frame listing the cases for the day. The door leading outside is separated from the lobby by a routine security check area, and one of the courthouse police guards stands there waiting for people to enter. Some other guards and janitors sit in their glass booth. There are many people sitting on the chairs, waiting for their turn to enter one of the courtrooms. DEFENDANT enters from the Courtroom Three door and sits down, waiting. It takes about fifteen minutes. During this time, announcements from other courtrooms could be occasionally heard, and people would enter or exit other courtrooms. Then... PRESIDING JUDGE (Over the PA) Verdict in the case 'prosecutor against Matti Meik l inen' in Courtroom 3. DEFENDANT exits through the Courtroom Three door. Scene Four: The Verdict The DEFENDANT enters from the lobby and sits down. After a minute, the PROSECUTOR rushes through the judges' door. PRESIDING JUDGE (Announcing to all people present.) The District Court has examined this case and has decided that Matti Vihtori Meik l inen did in fact commit the crime he was accused of in the way specified in the charge. The justification is the combination of the measured alcohol content in Meik l inen's breath and his own admission of guilt. Matti Vihtori Meik l inen is therefore sentenced to pay sixty-five day-fines, that is three hundred and ninety euros. Penal code, Chapter 23, Section 3. His driving license is suspended until the first of November this year. Road Traffic Act Section 75 Paragraph 1. One who is dissatisfied with this verdict can appeal it to the Vaasa Apellate Court within thirty days. Dissatisfaction must be declared, however, either right now to me or within a week to the Office of this District Court. (To DEFENDANT) Come and take the fine ticket and written appeal instructions (DEFENDANT takes those from the COURT REPORTER.) This hearing is concluded. The DEFENDANT leaves, as the court and the prosecutor prepare for the next case. TO BE CONTINUED
(with the next case)

WARNING: nothing in this post has been implemented or tested, except maybe at a minimal proof-of-concept level ("it compiles!"). It's just some musings on how you can make the computer do your work for you.

I was reading a defense of Hungarian notation when I started to get a familiar notion tickling the back of my brain. The author was explaining that the original idea behind Hungarian was not to replicate type information in variable names, but rather to include some sort of usage information: one example would be giving a string value holding a value that is untrusted the name utName. While it's more reasonable than lpschwzMyVariable, this suggesion made me suspicious that the author was really hacking around suckiness in the C language.

Tagging values with usWhatever and then verifying that you never mix usBlah with Blah isn't just tedious and error-prone, it smells a lot like doing work by hand that the computer ought to do for you. It seemed to me that it ought to be possible to "tag" unsafe values at the type level and let the language's type-checker sort things out. The simple approach is of course to define something like this:

newtype Unsafe t = Unsafe t

NB: I'm using Haskell because it was the first language that sprang to mind when I thought "type system hack".

This newtype declaration lets you attach an Unsafe tag to any type. Presumably the type signatures of routines that return untrusted data (such as a hypothetical getHTMLFormValue or a routine to retrieve previously entered form values from a database, if they aren't sanitized first) should be modified; for instance, if the signature of getHTMLFormValue was

getHTMLFormValue :: HTMLForm -> String -> String

it should instead read

getHTMLFormValue :: HTMLForm -> String -> Unsafe String

To create a new Unsafe value you simply apply the Unsafe constructor to the value:

getHTMLFormValue = Unsafe (oldGetHTMLFormValue)

To unwrap Unsafes you use pattern-matching:

let (Unsafe unwrapped) = wrapped in ...

So, this sort of works, but it's unsatisfying on several levels:

• It allows the creation of types such as Unsafe Unsafe Integer and even Unsafe [Unsafe Integer]. If a value is already unsafe, how can it be "more unsafe"? And what in the world is an unsafe list, or an unsafe list of unsafe values?
  
  Unfortunately, this is probably a limitation that any scheme that relies on type-tagging will have. You can get around it by either explicitly defining each new unsafe type:
  
  

  newtype UnsafeString = Unsafe String

  
  
  This is arguably a good idea anyway, since virtually all unsafe values (in the sense the previously linked article mentiones) are likely to be strings, and as I note later, it's not clear how to "sanitize" a non-string. OTOH, placing unnecessary type restrictions feels un-Haskellish to me, and it seems likely that you can trust the programmer not to pile up Unsafes.
  
  
• This scheme makes it very tempting to strip off the unsafeness of a value and lose the fact that you did so.
  
  Since the only way to strip off unsafeness is via pattern-matching, it seems to me that it would be tempting to pattern-match away the unsafeness in function declarations:
  
  

  foo (Unsafe x) = ...

  
  
  This may be just fine, but my preference would be to forbid pattern-matching altogether (by not exporting the constructor directly to other modules; just export a normal function that constructs an Unsafe value) and instead providing two routines to remove unsafety:
  
  

  escapeData :: Unsafe t -> t
  unsafeExtractRawData :: Unsafe t -> t

  
  
  The first routine encodes the data into a "safe" form, while the second simply fetches the wrapped value:
  
  

  unsafeExtractRawData (Unsafe x) = x
  
  foo x = doSomeStuffWith (unsafeExtractRawData x)

  
  
  The rationale here is twofold: first, by giving the dangerous extraction routine a scary and hard-to-type name, you remind yourself to think twice before invoking it (cf unsafePerformIO); second, you can easily audit all the locations where the unsafety is "stripped away" with a textual search.
  
  There are some issues involving the safe encoding -- you probably ought to make a typeclass of things that can be safely encoded from an Unsafe value; not too hard and this entry is already too long.
  
  
• The final issue involves manipulation of Unsafe values. So far you can pack and unpack Unsafes; however, this seems like it would get a bit annoying if you had to do any significant operations on them. For instance, searching in an unsafe string for a substring seems like it would be a pain and a bit error-prone (since you'd have to remember to wrap the string back up in an Unsafe). You might think that ideally, all normal String operations should work directly on an Unsafe String or pairs of Unsafe Strings -- you just shouldn't be able to convert between the two.
  
  Unfortunately, I can't see any way of doing everything that you'd want to here. For some types you can. For instance, the following definitions let you compare values wrapped in Unsafe in the usual way:
  
  

  instance (Eq t) => Eq (Unsafe t) where
      (==) (Unsafe v1) (Unsafe v2)   = v1 == v2
      (/=) (Unsafe v1) (Unsafe v2)   = v1 /= v2

  
  
  This admits (Unsafe t) to the Eq class, and looks reasonable enough (assuming that it's safe to compare unsafe values, of course). However, for probably the most important Unsafe values -- Unsafe Strings -- I can't see how to do this. The problem is that String is not a typeclass, it's a synonym for [Char], a list of characters. The important string operations are either defined directly on the String type, or generically on [t]; i.e., "a list of anything". Because there is no typeclass, you simply can't overload these operations to apply to unsafe strings as well as regular strings.
  
  One way around this is to provide an "unwrapping combinator":
  
  

  liftUnsafe :: (t -> t) -> Unsafe t -> Unsafe t
  liftUnsafe f (Unsafe v) = Unsafe (f v)

  
  
  This would automatically convert string operations to unsafe-string operations; not ideal, but reasonable enough. Operators are still a little annoying, but you can mangle their names:
  
  

  (++u) :: Unsafe [t] -> Unsafe [t] -> Unsafe [t]
  (++u) = liftUnsafe (++)

  
  
  So you have to use ++u instead of ++ to concatenate unsafe lists; not too bad.
  

While it may not be perfect -- it only makes sure that you don't accidentally violate safety, and doesn't stop you from deliberately doing something dumb -- this approach feels a lot safer to me than ad-hoc mangling of variable names. I used Haskell because you can do type hacks fairly easily in it, but I don't see why you couldn't play a similar game (albeit much less conveniently) in C++, or even in a dynamic language like Python. Of course, Python would give you runtime errors about safety violations instead of static ones, but that still beats getting runtime security holes.

I would add as a hypothesis that it should be possible to express just about any variable-name-mangling technique whose goal is to "make wrong code look wrong" in the type system, provided your type system doesn't suck; and that doing so is preferable to doing it manually, since it means that a lot of repetitive error-checking can be handled by the computer. And since the computer is better than you are at verifying formal invariants, forcing it to handle them whenever possible is always a good idea.

According to Jeff, I’ve become a smelly Nokia contractor (the reason he stayed with Luis instead of at Mako’s place in Boston, although I’m sure he was referring to Rob Taylor causing the smell). Part of this entails exchanging project schedules with various managers, where time is often talked about in terms of numbered weeks of the year, like: Foo task will be completed by week 42 of 2005. We ran into some problems with varying definitions of these numbers when exchanging schedules with people running on different platforms. Then we realised that even the software we’re using seems to have different ideas about what’s going on too…

• Outlook and Evolution seem to agree, and define the week as starting on Monday, and the first day of the year is always week 1, even if this results in a truncated week. Hence, week 1 of 2006 starts and ends on Sunday 1st January, and then week 2 begins on Monday 2nd, etc. It’s my belief that this is the correct behaviour as defined by some ISO standard somewhere which governs week numbers.
• GtkCalendar gets the week number correct, in that the first day of the year is always in week 1, but varies its definition of the first day of the week depending on your locale. This is presumably so that the display of the weeks is correct, but using it to number the weeks results in bogus week numbers. For me in LOCALE=en_GB it claims that week 1 of 2006 starts on Sunday 1st January, and ends on Saturday 7th January, making my week numbers one lower than the rest of the world.
• Planner uses GtkCalendar for inputting dates, so gets it wrong there as described, but seems to get it wrong in its Gantt chart view in a different way. It has the week starting on Monday, but claims that January 1st 2006 is the last day of week 52 of 2005. This results in week numbers that, match those GtkCalendar give me on weekdays, but differ over weekends, and are still totally bogus when I try and talk about project plans with my managers.

The result is a whole world of pain, and at best causes extreme confusion when we provide documents referring to both dates and week numbers which are inconsistent with each other in their minds, and at worst makes our managers think we’ll have things done a week sooner than we do. I’m not using Evolution at the moment for calendaring stuff, so I can manage if I remember to add one to all my 2006 week numbers when interacting with managers based on GtkCalendar and Planner, but this all seems to be horribly broken. I really don’t want to have to do this for the whole year.

Social network services (Wikipedia) such as Friendster, Linkedin, OpenBC are one of the top "web 2.0" hypes. I'm very sceptical about them, because I don't see the real benefits for me. (I'm referring to the pure "keep contact with your schoolmates" thing, not stuff which does much more such as serious blogging - e.g. livejournal - or planets such as planet.debian) The typical use case I've heard is much like "Imagine you have made a cool product, and want to sell it to some company. Now you can search for people at that company you know (or a friend of yours knows) and call them, this will help you get a feet in the door", as well as "you can use it for finding a job". I think that's bullshit, and to a large extend because I'd neither be willing to update my information in these services all the time, nor would I be willing to help just everybody here - most likely I'd just delete and ignore the email, and be really annoyed by a call (I'm annoyed by calls anyway)... It's not that I despise social networking per se. But I despise quantity instead of quality. I care a lot for my friends, and I'm in several groups grown around some technical issues; when someone is looking for a job I pay attention for suiteable offers around me; if someone needs something else I do try to help. But I'll only do that for people I'd really call friends. If you just want a quantitative number, last I checked my ranking was #36 in the global PGP keyring, meaning I have the 36th shortest average distance worldwide to verify a cryptographical key. So what? Of course I also tried one of the social networing services given above, joined a group for a university I was at. Even before the adminstrator of that group added me, I got an "invitation" by one "linkaholic". I rejected it, and even considered to report it as "spam" to the operator. I did indeed know that person, but all we had was a flamewar on a mailinglist, and I found him highly unsympatic. So why should I add him to my network? Where are the benefits for me (except that I'll maybe get an annoying call sometime in the future, when I could have forgotten about him)? No, in my opinion this social networking hype is built entirely on a hype, basically saying "look how well-connected I am, everybody knows me, and I know everybody", even when these relationships are only present on the computer, without anything worth being called a "relationship" behind. The flame-war I mentioned earlier was btw. on the topic of Resume/CV books. Which are even worse than this social networking thing, IMHO.

Well, there's nothing like posting a really rubbish program to your blog to find out the right way to do it. Firstly, Philipp Kern has come up with another, even shorter version than his version I posted in the update to my original Ruby post, this time blowing me away with the strangeness of explicit casts in Ruby. Apparently, casting an input stream to an array produces an array with one-line-per-element, as in this example:

 STDIN.to_a.sort_by rand .each l puts l 

As Philipp warns, however: "Ruby is full of fun and power; one must watch out that the scripts remain readable". For a fairly toy example, this solution is still fairly readable, however I can see hair growing on this thing if your program grows large. Next, Pierre-Charles David showed his qualifications for becoming a co-author of the next edition of the Pickaxe (AKA "Programming Ruby: The Pragmatic Programmers' Guide"), with his example and explanation. He's kindly given permission for me to reproduce his e-mail in it's entireity, since it's too good to chop up:

Here is the shortest and most readable version of your "randomize stdin" script I can think of:

  #!/usr/bin/env ruby
  print ARGF.readlines.sort_by   rand  
 

ARGF is a special variable representing either stdin or, in the case where file names are passed on the command line, the content of these files (think of Perl's <>). ARGF behaves like an IO object (although I think it is not one). In particular, it responds to the #readlines method which splits the receiver's content into lines and returns an array. In other words,

  lines = ARGF.readlines
 

is the idiomatic way of doing

  lines = []
  while line = gets() do
    lines << line
  end
 

in your example. The next part (sort_by) uses a new form of the sort method, which I think is not documented in the Pickaxe. It implements the Schwartzian transform technique. In essence, a random number is associated to each element of the array, which are then sorted based on this index. As a general style remark, note that although Ruby supports "while" and "for x in y" loops, they are rarely used, in favor of iterators and/or higher order methods. For example, to gather all the input lines in an array without using readlines, one might write

  lines = Array.new
  ARGF.each_line    l  lines << l  
 

or

  lines = ARGF.inject([])    lines, l  lines << l  
 

Hope it helps, and welcome to Ruby!

Pierre, your explanation definitely helped, and I hope it helps others too. The third person to help me out is Decklin Foster, through a post in his blog. What particularly caught me about his solution was the neat demonstration of another feature of Ruby which I think I will really learn to love: the ability to dynamically add methods to an existing class. "Aiee!" I hear all the purists scream -- and I can see the potential for severe misuse, too. But, since every good idea seems to get deeply perverted, we may as well have some real useful features to pervert while we're at it. Decklin has the clever idea of adding methods to the Array class that shuffle the elements of the array in a random fashion. His code is beautiful in it's simplicity:

 class Array
    def swap!(a, b)
        self[a], self[b] = self[b], self[a]
        self
    end
    def shuffle!
        each_index do  i 
            j = i + rand(length - i)
            swap!(i, j)
        end
        self
    end
 end

He then goes on to rewrite my bumbling example in probably the most comprehensible manner I've yet seen:

 STDIN.readlines.shuffle!.each    line  puts line  

Decklin writes, "Now that is cool.". I couldn't have put it better myself. My thanks to Philipp, Pierre-Charles, and Decklin, for helping me along the Ruby path. So, two new Ruby Rules for me to live by:

• Whenever you reach for a while, first take a good look around for any iterator methods which might apply.
• Never dismiss the power of defining your own methods on an existing class if it will improve the comprehension of your program.

Hmm, is it possible that Ruby is nothing more than Functional Programming by Stealth? Aiee!

Ok, time to dive into the kernel and find out what "execmem" exactly means:

 /*
  * We are making executable an anonymous mapping or a
  * private file mapping that will also be writable.
  * This has an additional check.
  */

Basically, this appears to be a mmap with PROT_EXEC and PROT_WRITE. Now lets see if I can find where this is used in pthreads, and if I can just remove either PROT_EXEC or PROT_WRITE there... Okay, most likely, this is the offending line:

nptl/allocatestack.c
static int allocate_stack (...
	mem = mmap (NULL, size, PROT_READ   PROT_WRITE   PROT_EXEC,
		MAP_PRIVATE   MAP_ANONYMOUS   ARCH_MAP_FLAGS, -1, 0);

So pthreads stacks are executeable memory by default... but disabling that sounds risky to me... :-(

I'm not usually a grand fan of the latest and greatest fads'' in programming. The only C# I know is the black key next to D#, and I loathe coffee. An IDE in the hands of a poor programmer will still result in crap code. While I do think that agile methods (such as XP) can do well in certain development environments, I certainly don't subscribe to the notion that XP can solve all the world's software production ills. And so on. I do know quite a number of programming languages, and tend to pick up new ones out of intellectual curiousity (although Ada did put a severe dent in that habit) or because a bit of research tells me that the language might be particularly suitable for a particular task (although I've since learned that the actual purpose of PHP isn't dynamic websites so much as bringing the security levels of Linux boxes in line with Windows-led industry standards). I also learn languages because I want to hack on projects written in them, which is motivational, but not a particularly good way to get a deep understanding of a language. Why all of this introspection in a blog? So you have the background to understand how everything about Ruby is different for me, including how I'm getting started using it. I have no interest in hacking on a Ruby-implemented project, and -- although Rails proponents give me an impression similar to that of a horny teenager with a shiny new unlimited broadband account -- I don't have any particular use for Ruby at the moment (I want to write less webapps, FFS, not more!). So, why the hell did I fork out $90 hard-earned South Pacific pesos for a copy of "Programming Ruby: The Pragmatic Programmers' Guide"? A couple of reasons:

• I'll admit it, Ruby caught my attention because of the hype surrounding it. What made this hype different was that it was coming from people who I knew as being (a) intelligent, and (b) usually immune to the typical hype machines. This suggested to me that the hype about Ruby might be, at the very least, a different sort of marketing bollocks, if not evidence of actual utility. Worth a look, at any rate.
• I had previously read a fantastic book on the general concepts of programming, entitled "The Pragmatic Programmer: From Journeyman to Master", by Dave Thomas and Andy Hunt. I had enjoyed the style of the book, and respected their ideas, and so when they were advocating Ruby as "the language that made programming fun again", and they had a complete tutorial and reference book out on the topic at hand, it seemed natural that I would look at their Ruby book as my "book of first refusal", as it were.

I'm glad I got "Programming Ruby". The book is superbly organised, reads very easily, and gives you all of the love as and when you need it. Surprisingly for me and a technical book, I read the first section of the book "cover to cover". By the end of it, I came away with a real (if naturally superficial) understanding of the language, and a feeling that I was ready to tackle the world in Ruby. It is a sad state of affairs when the first piece of new programming you can sink your teeth into is some four days after finishing your initial studies into a language, but this afternoon I wrote my first Ruby program in anger. It's quite simple -- just take stdin and rewrite it all on stdout with the lines in a random order. But I wrote it in a new programming language, in about 3 minutes, with nothing more than ri (Ruby's answer to firing up python and typing help(xyzzy), more or less) and "Programming Ruby". I think I also produced a suitably reasonable result:

#!/usr/bin/env ruby
lines = []
while line = gets() do
	lines << line
end
	
while ! lines.empty? do
	idx = rand(lines.length)
	puts lines[idx]
	lines.delete_at(idx)
end

(Any Ruby afficionados out there, feel free to e-mail me and tell me what egregious sins I've committed). I think it looks reasonably neat and clean, it's certainly simple, and the use of real Object Orientation makes for a certain readability (I think the method call lines.empty? is particularly cute, although I did initially consider the use of a question mark in a method name to be a crime against humanity). So, will I be using Ruby more in the future? I think so. There are some features in Ruby which I'm hoping to get a lot more cosy with -- the idea of "blocks" looks particularly interesting, and I don't think the utility of languages that have been designed, top-to-bottom, to really live the OO ideal has been explored enough (you SmallTalk bigots can stop glaring at me now -- really, I mean it -- stop it!). It's far too early to tell whether Ruby will become my "go to" language for random development tasks, but anything that stops the nightmares (where I start thinking about using PHP as a general scripting language) cannot be all bad. And, if I continue to make new webapps, Rails will probably come in handy. Update: HTML <pre> tags don't like having < inside them unescaped. Also, Philipp Kern pointed out that the Ruby documentation browser is called ri (not ir as I originally wrote). Philipp was also kind enough to provide a much shorter version of my original script, which shows the power of Ruby so much better:

 lines = []
 STDIN.each_line    line  lines << line  
 lines.sort_by   rand  .each    line  puts line  

Now tell me that isn't a language worth using!

The photo is from yesterday, but it is still new for most of you so the photo-a-day is still going strong for me :)
Today was spend in a very productive fashion - bugfixing. I had a lot of feedback for SBackup over last couple of months - it was time to fix some of the bugs people were reporting. Therefore version 0.9 of SBackup will be hitting you nearest mirror as we speak (I mean, I am uploading it now :)).
This realease is mostly a hey-this-thing-is-still-alive-and-kicking-ouch release and it also includes a TODO file with my plans for its future. I expect more feedback both on features and bugfixes for the stable 1.0 branch and for the planned 2.0 rewrite.
Also I spent couple of hours reviewing Wiki engines with an idea to put one on SourceForge to provide documentation for SBackup - I found no good candidates. I wanted a simple wiki, preferably in python, with MySQL database (SourceForge mounts web directories read-only). Not good. Now I am considering making a page in Wikipedia and using that as a documentation page. Scream at me violently if have any other solutions :)

On Thursday, I went up to my father's house to pick up my desktop, now that I finally have Internet access at home so I can have permanent access to it from a more convenient place. While I was there, I decided it'd be a good idea to do an upgrade on my dad's box. While I was doing this, the nfs share of the Debian mirror in the home server that serves this blog stopped working, and I couldn't ssh in, although the console and ICMP appeared to work. Having no monitor attached to that box, I decided to reboot it and resolve the problem in a lame way. Had I known my primary hard drive had decided enough is enough, I would have looked for a monitor instead. After reboot, the server started doing a very loud and scary noise, which was not new to me. When the BIOS tried to probe the disk, it would not start up and would instead whine like that. When this happened in the past, switching the box for a few minutes and trying again was enough. Not this time, though, it looked like it was the end. Horrified by the fact that I had no really useful backup of /etc and /home, I tried over and over to get it working. I plugged the HD into another old AT box (which once served as my GNU Hurd playground), but I still got those scary noises. At least I knew it had nothing to do with the mother board. As it was getting late, I decided to take the HD with me, and another Maxtor of the same size (but not exact model) with me, to have a look in office the next day. I thought I'd have to restort to try to transplant the logic board from the good one to the faulty disk, but before that, I realised I could try mounting it in a USB cage. For some reason or another, this worked, and I quickly saved thetwo partitions it contained with no read errors. After this, I realised I had not done the copy as root, so I had lost all my ownerships. I plugged the disk again, and re-rsynced. It still worked. Alleviated, I went back home, copied the data to another old disk, and I don't remember why, I tried mounting the faulty disk again. I got scary noises even using the USB case, and no other tries have been successful, so I think it's completely dead now, just before my final mount which saved the data. The box is now back online, using a Seagate disk I had stored in a drawer and with no notes on it about it having any kind of problem. I suspect I will need to do real backups now, because the drive isn't as safe as it looked...

hda: dma_timer_expiry: dma status == 0x20
hda: DMA timeout retry
hda: timeout waiting for DMA
hda: status error: status=0x58   DriveReady SeekComplete DataRequest  
hda: drive not ready for command
hda: status timeout: status=0xd0   Busy  
hda: drive not ready for command
ide0: reset: success

My little P150 needs a 6GB drive. I'll have to find one somewhere.

Where do I start? It's been an eventful couple of days... The flight to LA was uneventful. Qantas is certainly orders of magnitude more pleasant to fly with than United, even though we were seated in a bulkhead area. I had a 13-month old baby seated next to me (no, Mikal was on the other side), and he was about as well behaved as you could possibly ask a 13-month old baby to behave. Mikal also behaved himself. I had the intention of trying to get something approximating 8 hours sleep, as we arrived at LAX at 7am local time, but I only managed about 4 hours. I did spend about 8 hours with a mask on and my brain in idle, so I arrived at LAX being mostly coherent. Entering the US via LAX was a much less traumatic experience than I was expecting. The E-3 visa was accepted fine, and I now have an I-94 stapled into my passport. Quarantine involved getting looked up and down by a guy in a booth, who waved me through. We had 3 hours to spare (or about 2 by the time we cleared all the immigration/customs queues) and then hopped on a 50 minute American Airlines flight to San Francisco, hired a car and drove to Santa Clara. The accommodation we're staying in for a month is very spiffy indeed. It's a largish gated community of units. Mine's got a ground floor entry, but everything else (except the garage) is upstairs. When Sarah gets here, we'll put up some photos. The weather is lovely as well. Our first outing was to the Social Security office to attempt to get social security numbers. This wasn't as successful as I'd hoped, as we were too freshly in the country to show up on their systems, so we left with letters saying we'd get one within four weeks, when our details had been verified. Yesterday evening, we went for a general recce of the surrounding area, and checked out a supermarket. That was less disconcerting than I expected, although I don't think it was a terribly huge supermarket. Having a GPS certainly raises one's confidence when navigating significantly. Last night I discovered that all the keys in my apartment were not created equal, and that the two loose in the lock box on the front door were the only two that opened the unit, and the two sets on key rings on the coffee table only opened the amenities and my mailbox. So I got to spend a bit of time sitting on my front doorstep until the nice man from Synergy Relocations came out to let me in (except his key didn't work either, so he got to make conversation with a legal alien while we waited for someone else to come out with a key). Today, we drove around to see a shopping mall, and the first set of shops we found happened to be around the corner from an enormous (by my standards) apartment complex, which had some signage implying that it had vacancies, so we decided to get the low down on how renting worked before we did our rental tours. Casey drove us around in a golf buggy and gave a tour of what two different floor plans looked liked. It seems that in this part, it's standard for an apartment to include the refrigerator, dishwasher, microwave, washing machine and clothes dryer, which is good to know. Let me just say that if this is what living in an apartment complex is like, it's more like living in a resort. The place had pools everywhere, tennis courts, two gyms, a community centre, it was just unreal. It was also cheaper than I was expecting. We then went to a massive shopping mall to do some window shopping. Discovered Wetzel's Pretzels. Hope they're not too fattening... Tomorrow, I'm going to avail myself to the complex's gym, and generally get ready for the week ahead. Future posts will be in a new category, which I won't be feeding to Planet Debian as not to bore people. I will inflict Planet Linux Australia readers with them until someone asks me to stop. If you'd like a custom feed that excludes my American adventures, please let me know. I wonder if adjusting Blosxom's idea of timezone buggers up existing posts and aggregators like Planet in general?

Since giving XTech 2006 the tagline "Building Web 2.0", I've received a few questions as to how wise this was. The Web 2.0 label has been accused in some quarters of being a vague, marketing-oriented, term. Perhaps an attempt to kickstart a new bubble. I was going to write further, defending my choice: that, practically speaking, Web 2.0 encapsulates a combination of technologies and practices driving the web forward. Yes, it's a label of convenience, but the implementation trends are real. The maturation of XML, a revival in browser technology, a willingness to share data openly and new web development frameworks. Just as I embarked on distilling these thoughts into a fuller form, Paul Graham published an excellent essay on Web 2.0. In it he analyses the Web 2.0 term:

Does "Web 2.0" mean anything? Till recently I thought it didn't, but the truth turns out to be more complicated. Originally, yes, it was meaningless. Now it seems to have acquired a meaning. And yet those who dislike the term are probably right, because if it means what I think it does, we don't need it.

So what's the reasoning that means we don't need the term? Graham writes: "Web 2.0 means using the web the way it's meant to be used." There's nothing fundamentally new, but there is a groundswell of understanding of what's been there all along. For me, that's a wonderful encapsulation of both current web trends and my aspirations for XTech. "Web 2.0" is a just a handy label for this. "The Web as it was meant to be used" will strike a chord with many web practitioners today--whether it's sensible use of web services with REST, improving user interaction in web design, or open semantics and data interchange. If you're involved in using the web as it's meant to be used, consider submitting an XTech proposal.

I've just uploaded sponsored package libpam-encfs. With it it's possible to have encrypted home directory, which will be mounted during login.

In another installation of my pointing to Cringely's columns, let me mention yesterday's column speculating about Google. Choice quotes:

[...]
Google hired a pair of very bright industrial designers to figure out how to cram the greatest number of CPUs, the most storage, memory and power support into a 20- or 40-foot box. We're talking about 5000 Opteron processors and 3.5 petabytes of disk storage that can be dropped-off overnight by a tractor-trailer rig. The idea is to plant one of these puppies anywhere Google owns access to fiber, basically turning the entire Internet into a giant processing and storage grid.
While Google could put these containers anywhere, it makes the most sense to place them at Internet peering points, of which there are about 300 worldwide.
Two years ago Google had one data center. Today they are reported to have 64. Two years from now, they will have 300-plus.
[...] There will be the Internet, and then there will be the Google Internet, superimposed on top. We'll use it without even knowing.
[...] All this is based, of course, on Google's proven network and hardware expertise. Have you seen Google's Search Appliance? They ship you a 1U prebuilt server. You connect it to your network, fill out a simple configuration screen, and it scans and indexes your web site (or sites) for you. Google monitors and manages it remotely, and sucks up the data and adds it to theirs. You just plug the thing in and turn it on. It just works. You need do nothing else to keep it running. Google understands how to do this stuff. Microsoft definitely does not.
[...] Microsoft can't compete. Yahoo probably can't compete. Sun and IBM are like remora, along for the ride. And what does it all cost, maybe $1 billion? That's less than Microsoft spends on legal settlements each year.
Game over. And yet next week I'll take it one more step.

Fact? Fiction? Feasible? As they say, read the whole thing.