Our adventurous friends Eric and Katie had invited us to join them on a hike in the Grand Canyon over the 4th of July weekend. As this struck us as something we'd never have considered doing otherwise, we thought we'd accept their kind offer. We're so glad we did. It was quite a bit of traveling just to get there. We flew out of San Jose on Thursday night, got into Las Vegas, rented a car, and then drove to Peach Springs, Arizona, where we stayed the night. We didn't get in at the Grand Canyon Caverns Inn until around midnight, and we had to get up again for a 5:00am departure (I think we actually got away by more like 5:30am), to drive to the Hualapai Hilltop, where the trail head was. Zoe handled all of this very well. She didn't end up falling asleep in the car until about 10:30pm I think, and dealt with the transfer out of the car at midnight very well and went back to sleep. The drive to the Hualapai Hilltop, along Indian Road 18, took about an hour. We had to dodge a bit of livestock on the road. The road itself was sealed and of good quality. We remarked on the way back out that as soon as you left the Havasupai Reservation and entered the Hualapai Reservation that there was a distinct degradation in the quality of the road. We parked at the Hilltop, and got ourselves organised, and commenced our descent by about 7:10am. The first 1.5 miles was very steep. There were a bunch of switchbacks and there was a fair few people coming up the other way already. There were also a few mule trains coming in the opposite direction that we had to dodge. The mule trains have right of way, so we had to stop and get out of the way to let them pass. Zoe thought they were pretty cool because they invariably had a few dogs with them as well, and she thinks dogs are awesome. The remaining 6.5 miles (it was an 8 mile/13 kilometre hike in total) were fairly easy. There was a lot of variety in terrain. Rocks, sand, dirt. It wasn't all in the sun, which was made it more comfortable. Heat wise, I think it was in the high 30's Celsius, with pretty much no humidity, so it didn't feel too uncomfortable. Towards the end, as we got near the water, there was a distinct change in the canyon. There was more greenery, and we started hearing insects and seeing bird life. When we first saw the water, it was hard not to just jump in, it looked so inviting. The total hiking time was about 4 hours 45 minutes, which included a reasonable stop to give Zoe some breakfast. We arrived in the village at around midday. Unfortunately the lodge wasn't ready for check ins until 1pm, so we had to twiddle our thumbs for a little bit. Zoe handled the hike incredibly well. We used a Macpac Possum child carrier that Nigel had passed down to us, and she didn't complain the whole time. She even took a nap in it. We were very relieved that it wasn't a problem for her. Sarah carried her, and I carried everything else. (I'm so glad I didn't see the photo of how I looked until well afterwards). In hindsight we probably brought too much stuff, but we didn't really know what to expect down there, so we'd rather have too much than too little. The village completely exceeded my expectations. I had this (silly) mental picture of it being just the lodge and not much else, but somewhere between 450 and 700 people live in the village. It had two general-type stores, a cafe that was pretty much like a fast food/takeaway place, an elementary school, and some sort of basic medical clinic. There was also a small general purpose community centre of some sort. The main store had pretty much everything you'd need in the food department. The cafe frequently ran out of various different things, but you weren't going to go hungry. For the remainder of Friday, we just stayed put in the village and let Zoe stretch her legs. Eric and Katie and their kids explored the first set of falls down from the village, unofficially named Rock Falls. On Saturday, we all hiked down to Havasu Falls. We met a couple of other families (amusingly both from the Bay Area) along the way, so we had some interesting conversations. I have no idea how long the hike was. Havasu Falls was impressive. Quite tall and narrow at the top, and quite loud. Zoe was impressed. We took her for a bit of a swim in the pools in the bottom. We had lunch there, and then Zoe and Sarah stayed there (Zoe took a nap in the KinderKot (also courtesy of Nigel) in the shade by the falls, and I took the Macpac with a water bladder in it to refill it at the spring in the campground, which was further down stream, and to check out Mooney Falls with Eric and Katie (and their kids), which was further downstream again from the campground. Mooney Falls was pretty insane. A lot taller than Havasu Falls, and a lot more treacherous to get to the bottom of. Parts of it were vertical. Chains and rickety ladders were involved. Caves were involved. I'm so glad I didn't have Zoe in the Macpac, because on multiple occasions, the top of it got bashed into rocks, so without having Sarah with me also to act as a guide, there's no way I could have safely descended with Zoe in the backpack. My favourite photo illustrating how crazy this was is this one that I took of Eric and Katie. This photo highlights some of the craziness involved to get down. I pretty much got to the bottom, had a quick look around and left Eric and Katie to it, because I figured Zoe would have woken up by now, and was probably bored. Turned out I had all of the sunscreen with me as well. So I climbed back up, and trekked back to the campground, refilled at the spring, and headed back up to Havasu Falls. After we reunited, we then hiked back up to the village, which to be honest, was the hardest bit of hiking we did. I was carrying Zoe in the Macpac, and it was hot, and it was mostly loose sand all the way. We were very glad to get back to the village. Zoe started showing signs of coming down with a cold and a cough, so we decided to cut short our stay in the village and leave on Sunday instead of Monday. We'd been tossing up between getting a helicopter or horses out. We ended up opting for the helicopter after learning that the horses still took about 3.5 hours, and you basically had no control over the animal. We were dubious as to how Zoe would go being bounced around on the back of a horse for that long. The helicopter ride also took all of about 5 minutes. The tribal members get first dibs on the helicopter, and there had been a death in the village, and I think a lot of people had come to the village for the viewing (the casket itself was flown out earlier on Sunday morning for a burial out of the village), so I suspect there was a higher number of tribal passengers than usual. We didn't get out until 2pm. Again, Zoe was incredibly good. She just hung out with us while we waiting for a flight out. There were plenty of dogs for her to be entertained by, and some other kids. She also took a pretty decent nap. So we finally got out, and got back to our car, and then the heavens opened. It absolutely poured down with rain. Sarah was keen to check out Hoover Dam, so we decided to stay near Boulder City in Nevada, and got a suite in the Hacienda Hotel and Casino for cheap. The next day was the Monday public holiday for Independence Day, and we were tossing up whether or not to take Zoe to a doctor to get her checked to see if she had bronchitis (we've been a bit gun-shy about bronchitis since the time she came down with it when we were in Australia). She ended up falling asleep in the car while we were hunting around for an Urgent Care facility that was open, so we decided to let her sleep and drove to Las Vegas. After she woke up, we took her to see the flamingos at Flamingo and the lions at MGM Grand. Zoe absolutely loved all the lights, sights and sounds of Las Vegas. In the afternoon, we took her to a doctor, and got the "first child, huh?" from the doctor (she didn't have bronchitis). After that, we went and briefly checked out Hoover Dam (just walked half way across the bridge in front of it and walked across the dam itself). The following day, we packed up and spent the morning on the Las Vegas strip again. This time we went to the Venetian (Zoe thought the gondolas were pretty cool) and then we flew out in the afternoon. So it was a lovely little family vacation. We ended up spending about as much time in Las Vegas as we did in the Grand Canyon, but that was okay. I'd really like to go back to Supai again, but I'm not sure if the stars will align themselves appropriately. If we did, knowing what I know now, I'd take a lot less food and put all of the luggage on a horse, and just hike in with the minimum of stuff. I'd probably be game to hike out as well if the bulk of the luggage was on a horse. Photos from the trip are here.

Short version arrival was quite easy, the arrival wiki page is now up-to-date and my photos are uploaded every day to this Flickr set. I had a small problem departing from CDG to Zagreb after scanning my suit twice and complaining that it had too many electronic devices, they had me remove my Nexus S, Nokia N950 and Kindle 3 from the suit pockets and then scanned that all lot again. After that another security guy asked me to open my suitcase and started squishing my Ligo cheese and declaring in a grave voice that it was too soft to be legal . As I kept calm and explained the DebConf cheese party to him, he looked around, put the cheese back in the bag and said ok, ok, off you go . So I still have that softness for the party. I heard an even better anecdote from our next years hosts when they were travelling trough Germany, a border official asked a typical so, what are you here for? question. They answered that they were in transit for an IT conference and gave him the invitation letter. He read the letter, nodded and then said: DebConf, huh? So, who was the author of the Linux kernel? and when after the initial shock they gave the right answer, they were immediately waved trough. Now that s what I call good border security. I met up with the guys in the Zagreb airport caf , which was immediately after exiting the secure zone. We waited for dkg to arrive and get his baggage. The passport check too a couple of minutes, but the baggage claim looked to be taking almost a half an hour. After that was sorted out, we took out some local money in one of the 5 ATMs around and went to the bus. We were waived to just put our bags into the bus luggage space and get on, the driver went around to sell us tickets just before we drove off. It was one of the most comfortable and well air-conditioned buses I have been on in a long time. We were in the Zagreb bus station in less than half an hour. Then a funny part started an old man came up to dkg and immediately asked Are you going to Banja Luka? , and when he said yes, the man started dragging us along to the bus station saying that there is a bus to there that will leave in a couple minutes, but we were in luck, so we should follow him right to the bus. Driven by curiosity, we went on. The man dragged us to the far side of the bus terminal bus stops where there was a minibus with Bosnian number plates, but no official designation as a passenger transit bus and no other passengers. They asked a reasonable 120 kuna for the trip, but they looked very shady to me, so we refused. I think that if we agreed and we were lucky enough that they actually did get us to Banja Luka, the price would have risen along the way, for example using the highway tolls as an excuse. After getting cursed at by the old man and being demanded to buy him a beer for his efforts, we went upstairs to the real bus station where we were told that there were two buses to Banja Luka: one at 15:00 and another at 16:30, but that those two buses arrive in Banja Luka almost at the same time, so we got us tickets for the latter bus and went to a local bar to get some food and drinks. I somehow managed to get sausages in deepfried bread that for some reason cost almost triple of a hamburger and fries that others got, but it sure helped to pass the time and after all those 9 was not a bad price for a good meal. We were a bit surprised by the bus drivers asking us to pay 1 (or 8 kuna) for bags to be put into bus bag storage, especially as we for several minutes could not quite understand what they want of us as neither of them spoke either English, Russian or German. I don t remember when was the last time I crossed a real (i.e. non-Shengen) border in a bus, so getting out of the bus and walking one-by-one to the border official was a new experience. On the other border of two bus drivers collected the passports an then brought them back later, so that was less painful. And by then we were in Bosnia! And of the things that greeted us was a billboard for the casino in Banja Luka Hotel Bosna. By the time we got to Banja Luka, dkg had already made friends with an official from Croatian ministry of regional development that sat behind us on the bus and knew English very well (after living in Canada for some time). The bus station in Banja Luka looked almost abandoned like the bus stopped in the field with only a few taxi drivers and a bunch of buildings some distance away showing that this was indeed part of civilization. The taxi drivers would take you to the venue for 8-10 KM 5 or 50-100 kuna (if their meter is broken and you look to be really ignorant of the exchange rates). If you walk on a bit, there is a bus station with an ATM taking Visa cards and a bit further on a bus stop that will take you to the centre of the city for 1.5 KM. The train station looked fully abandoned when we went there, only a news kiosk was working there. The bus driver reacted well to the name Hotel Bosna and wave toward it when we passed by it. It is hard to miss, especially the blinking Casino sign or the white pillars with black letters Hotel Bosna on top. You just get off at the bus station Central and walk back a couple hundred meters. Then see all you Debian friends drinking in the cool bar and walk around the corner to the main entrance of the hotel to check in. The rooms are looking really great this is the best DebConf location that I have been to (I was not in Argentina). Soft beds with linens changed daily, nice air conditioning, towels (changed daily, if used), good shower, It is lacking in the speed and power of its free WiFi and in some place there is a clear impression that the hotel was built as a top of the line hotel a couple decades ago, but somehow has fallen into a disrepair since. Or it is simply used much more than expected. So far I would say that the DebCamp is running full steam two hacklabs have desks, chairs, power and network connection, one of them even has sufficient air conditioning. The network works great for me so far, at least in the venue. The hotel wireless is barely usable to read a blog post, preferably without photos. The food is being served well and on time. So far for lunch and dinner you walk up to the reception and get a food ticket based on your room number and go to the M level for the food (the M level is between the ground floor, labelled as PR , and first floor) where for breakfast and dinner there is a buffet, but for lunch you sit at a table and they bring you food. Vegetarians should clearly say vegetarian while pointing at themselves to get the special vegetarian option. So far food has been very good, except that that they often run out of popular items, such as fries, in the buffet. You can also ask for beer and other extra drinks with your food, but know that they cost extra up to 2.5KM for a beer, for example.

On the bus heading to the airport to go to DebConf 11, I chanced to sit down beside an older Croatian gentleman. If I've ever met another Croatian living in Portland before now, they did not have occasion to make themselves known to me. He spoke highly of Tito at some length, and of Tito's success at holding the Russians at bay in the 1960s when the rest of the Communist block was being held there by the thumb of the Russian military. If I've ever met another person living anywhere who approved of Tito, they've not had occasion to make that opinion known to me either. Wacky experiences like that are why I love Portland's mass transit network.

This weekend, the boys got what they ve been waiting for: another trip on Amtrak. For the last week or so, Jacob has had a morning ritual. He ll look at the calendar, figure out what today is, figure out when we ll get on the train, and then figure out how many days it will be. As the number gets lower, the excitement gets higher, of course. Friday morning, we woke the boys at 2:30AM to get to the train station. The only Amtrak trains through our area are middle of the night departures. Jacob is normally hard to wake up, but when I tell him that I m waking him up to go to the train station, he wakes up faster than I ve ever seen him before. It takes about 3 seconds for him to process that in his groggy state, and then he sits up straight, throws off the covers, and is instantly ready to go. Both boys were excited even in the station waiting room. Oliver has been on an Amtrak train before, but it s been awhile and he probably doesn t remember it. He constantly talked about it, jabbering as much as his vocabulary lets him. Once we were on the train, Terah and I would have liked to get some more sleep. The boys, on the other hand, were now wide awake, and didn t fall back asleep until about 5:30. That means Terah and I didn t, either. Here s what it usually looks like: When it came time for breakfast, we went to the dining car as usual. Jacob had already been telling us for days what he would eat for breakfast on the train: I always have French toast on the train, dad. And so he did. A few minutes later, I heard the waitress telling other people they were out of French toast, so I was glad I didn t have to disappoint Jacob over that! But it was Oliver that really came alive on this trip. I don t think I ve ever seen him so excited. It was almost constant. He happily talked about anything he could see or touch. But he also listened to other conversations, and would frequently pick out one word from a sentence and try to say it. He does that at home too, but not nearly so often as on the train. Both boys wanted to go exploring a lot my word for taking them for a walk in the train and seeing what they might find. We walked up and down the train several times, Jacob excited over opening the doors between the cars, and Oliver excited just to be there. Oliver s excitement kept him from sleeping well. He did eventually get a short nap, but that wasn t quite enough to avert a couple of tantrums later in the day. The reason for the trip was the 80th birthday party for Terah s grandpa. And it so happened that Oliver s 2nd birthday would be over the same time. So, Saturday morning, we all went for breakfast at Das Dutchman Essenhaus, one of the favorite local restaurants in northern Indiana. After that, it was over to a relative s place for some birthday festivities. The children got mini cakes to decorate as rail cars. There was frosting and all sorts of toppings. Great fun was had by all, and it was wise that this activity took place in a garage rather than indoors. Yes, that is a marshmallow stuck to Jacob s nose. Then in the evening, it was off to another relative s place for some more family time. Jacob had a great time all day, and was in high spirits. He asked me to sit by him at dinner, and started one of the longest conversations I ve had with him in some time. We just talked about the things that happened in the day, but it was nice when I told him, I like sitting by you, Jacob, and he said, Dad, me too! The big highlight for the day happened in the evening. We were gathered around a fire with a guitar to do some singing. Jacob was happily perched in his lawn chair, but got very excited when he saw some lightweight airplanes flying overhead. These kept flying at some distance, and he kept pointing them out to us. But that wasn t even the most exciting part. That came when the fireflies came out. Jacob ran around, catching them in his hands, and excitedly showing them to whatever person happened to be closest. He was laughing with joy for such a long time. At one point, someone asked him if the bugs were tickling his hands. He said, evidentally just realizing it, Oh yes, they ARE tickling my hands! He was one very happy boy. Sunday I had to leave to get back home, while Terah and the boys will return a couple of days later. Although I do sort of look forward to a train trip that I can relax without having to manage two young boys, I do miss them already and will be happy to have everyone back home in a few days. (This post written during the trip and posted a week later after arriving home)

When I recently switched my blog over to Jekyll as my blog/site generation platform, one of the driving forces was implementing comments. I ve been a comment-free blogger for years, but I ve just decided to change my opinions on things and see what happens. (Turns out: you get some cool comments. And a lot of spam.) However, the standard advice when it comes to handling comments with Jekyll is use Disqus (or, very occasionally, some other Insert-Javascript-Here cloud commenting service). Not being one to stand for the conventional wisdom, and given that I m allergic to the cloud , I decided to go a different way an all-static comment handling system for Jekyll. First off, you need a way to render comments inside Jekyll. This was disturbingly easy with Jekyll s plugin system. With around 30 lines of Ruby, it was done. It simply reads a pile of YAML files from a _comments directory, and adds them to the individual blog post objects in the templating system, which can then be iterated over when rendering a post. Obviously, you need to also adjust your blog post templates to suit. This is fairly straightforward; just render each of page.comments in a list. Adding a comment submission form is also necessary, and similarly trivial. The only moving part in this whole system is the bit of code that handles comment form submissions. The simplest way to handle it, which is what I ve gone with, is a few lines of PHP that writes the contents of the form out as YAML and e-mails it to me. Although I m not a fan of PHP for web applications, this is sort of job is where PHP excels it s ubiquitous, trivial to deploy, and easy to understand (in small doses). Using a CGI or anything larger (like, say, a sinatra app) would be massive overkill. Since I m still a control freak at heart, I m hand-moderating all comments by the simple expedient of having all blog comments e-mailed to me¹, and I have to import them into the blog and re-upload. Mutt makes this trivial; I ve got a script that I trigger by pressing C (for comment) that pipes the message to formail to strip the RFC2822 header, then drops the rest into my blog s _comments directory. A quick rake upload and they re published. If you re not as obsessive as me about hand-moderating your comments, it wouldn t be hard to write a slightly different PHP script that commits the comment directly to a git repo² and re-renders the site. All you d have to do when you wanted to make a new post is merge those new comment commits from the live git repo and go to work. This would then be a completely static site (with all the performance benefits that implies) with immediate-gratification comments. So, if you re suspicious of the cloud, or just want to be the cool kid whose blog comments load even when people have turned cookies off, check out my Jekyll static comments plugin.

Keeping an eye on something with a webcam is really easy: just a matter of setting up for example (c)vlc to multicast from /dev/videoX to the network, and done. Example:

# Streaming:
cvlc v4l2:///dev/video0 --no-audio --sout "#transcode vcodec=mp2v,vb=3072 :std access=udp,mux=ts,dst=224.0.0.1 "
# Playing, possibly on a remote machine:
vlc udp://@224.0.0.1:1234

Notes:

• v4l2 vs. v4l depending on the distribution (sid vs. squeeze).
• Triple slash needed, two for the v4l[2]:// part, one for the device (/dev/video0).
• Why this address? See Wikipedia s Multicast address article.

Possible problems:

• If switches don t have IGMP snooping enabled, throwing multicast at them turns it into plain broadcast. In other words: instead of having packets sent to the hosts subscribed to that multicast address, everyone gets those packets, polluting the network.
• Trying to stream from multiple devices may fail, due to USB bandwidth issues. At least that can be seen with two random devices using the uvcvideo module, getting No space left on device when the second (c)vlc is started.

Where to go from here:

• fswebcam to the rescue! It can capture a frame from a given device. The following options are particularly useful: --resolution to request a specific resolution (the device will fall back to another resolution if the requested one is unavailable); --frames to specify the number of frames to capture (lower noise, but possibly higher blurriness).
• A tiny python wrapper around it makes it possible to set various webcams, with different options (number of frames, delay between two captures, resolution), operating at the same time thanks to the help of the threading module. To avoid encountering the same bandwidth issue, a tiny lock ensures a single fswebcam instance runs at a time.
• That also makes it possible to save all images as videoX-timestamp.jpg and to log rotate them after a while, for further viewing/reviewing during a given time window.
• Thanks to ImageMagick, it s also trivial to process the captured images:

  # Only keep a rectangle of width W, height H, starting at the point (X,Y):
  convert -crop WxH+X+Y capture.jpg capture.jpg
  # Include a timestamp on the top-left corner:
  convert -box white -font Fixed -draw "text 2,12 \"$timestamp\"" capture.jpg capture.jpg
  

• Keeping a videoX-current.jpg symlink for each /dev/videoX device is trivial.
• Add to that a lightweight webserver and a single page showing the most recent capture for each device thanks to the symlinks, with a meta refresh of say 5 seconds, and tada! Keeping an eye on multiple things at once. Of course that s not a real video, but that can fulfill some needs.

Yesterday's blog entry garnered a lot of attention and some responses (in the form of e-mails, which is totally fine). Two of them deserve, as far as I'm concerend, their own little entry in the mini-series I kicked off. So, here we go: First up was reader Sylvain "ythier" Hitier, who suggested adding

set list lcs=eol: ,tab: -,trail: 

to your .vimrc and making use of the :list option. The above line will highlight tabulators, trailing whitespaces and the end of a line. And it'll allow you to differentiate the three.
A warning though: you would want to deactivate this, in case you want to do copy and pasting with your mouse. [UPDATE] Sylvain just sent me a little key binding for C&P he's using to toggle the highlighting:

function CutPaste()
  setlocal paste! nu! list!
endfunction
map <F10> :call CutPaste()

[/UPDATE] The second really nice idea came by courtesy of Thilo Six to my attention:

" automatically delete trailing whitespace & Dos-returns     2
fun! <SID>MyDeleteTrailingWhitespace()
  if ! &bin
    let l:l = line(".")
    let l:c = col(".")
    silent! :%s/[\r \t]\+$//
    call histdel("search", -1)
    call cursor(l:l, l:c)
  endif
endfun
autocmd BufWritePre,FileWritePre * call <SID>MyDeleteTrailingWhitespace()

The above snippet will automatically delete trailing whitespace and carriage returns as added by a lot of Windows programs.
Again a little warning: the function is executed automatically, so be sure you have only replacements included in the regular expression you really want to get deleted! Thanks a lot for all the input I've received!

Hats off to the GnuCash developers and the Parit Worker Collective for the python bindings to GnuCash. If there is any doubt that free software makes life easier it's being able to write your own accounting import scripts while reading the original source code. I had to re-compile Debian's version of GnuCash because it doesn't have the python bindings enabled by default. As root:

apt-get build-dep gnucash
apt-get install python-dev

As a non privileged user:

apt-get source gnucash
cd gnucash-N.N.N # varies depending on your version)

Then, edit debian/rules and add --enable-python-bindings to the dhautoconfigure command. I replaced the line:

--enable-locale-specific-tax

With the lines:

--enable-locale-specific-tax \
--enable-python-bindings

Then, update the change log with:

dch

Built the new package:

dpkg-buildpackage -rfakeroot -uc -b

I then wrote a python module. It logs into the server running our members database, runs a script to export tab delimited lists, and then imports them into our gnucash xml file. It's far from perfect (I'm fairly new to python). Also - I have sheepishly inserted code that searches for certain values by accessing the gnucash file directly (via python's xml handling modules). I couldn't figure out how to do the search I needed with they available python methods, although it may very well be there and I just couldn't figure it out. If anyone is interested in using it for a template:

#!/usr/bin/python
import sys 
import os
import time
import csv
import warnings
from xml.dom.minidom import parse
from gnucash import Session,  GncNumeric
from gnucash.gnucash_business import Customer, Invoice, Entry, BillTerm
from decimal import Decimal
import datetime
import tempfile
import subprocess
def main(argv = None):
  if argv is None:
    argv = sys.argv
  path = os.environ['MFPL_GNUCASH_PATH']
  if not os.path.exists(path):
    print """Path (%s) does not exist. Please set 
    MFPL_GNUCASH_PATH environment variable\n""" % path
    return 1 
  if os.path.exists(path + '.LCK'):
    print """Lock file exists. Is GNUCash running?\n"""
    return 1 
  command = argv[1]
  import_file = None
  if 2 in argv:
    if argv[2] == '--from-file':
      if argv[3]:
        if os.path.exists(argv[3]):
          file_path = argv[3]
          import_file = open(file_path,'r')
        else:
          print "Can't find file to import: %s" % argv[3]
          return 1
      else:
        print "Please pass path to file if using --from-file"
        return 1
    else:
      print "I didn't understand the argument: %s" %s
  # download import file if not provided
  if import_file == None:
    import_file = get_data(command)
  # The file might be gzipped, so
  # open and save it 
  s = get_gnucash_session(path) 
  s.save()
  s.end()
  # Wait a second, so we can create a backup
  # without a name collision
  time.sleep(1)
  # Open for real now...
  s = get_gnucash_session(path) 
  xml_dom = parse(path) 
  if command == "members":
    ret = import_members_from_file(s, xml_dom, import_file)
  if command == "invoices":
    ret = import_invoices_from_file(s, xml_dom, import_file)
  s.save()
  s.end()
  return ret
def get_data(command):
  import_file = tempfile.NamedTemporaryFile()   
  if command == 'members':
    remote_cmd = 'acc-member-list'
  else:
    remote_cmd = 'acc-invoice-list'
  args = [ '/usr/bin/ssh', 'mayfirst@hay.mayfirst.org', '/home/mayfirst/bin/' + remote_cmd ]
  subprocess.call(args,stdout = import_file)
  return import_file
def import_invoices_from_file(s, xml_dom, csvfile):
  dialect = 'excel-tab'
  csvfile.seek(0)
  reader = csv.reader(csvfile, dialect)
  nodes = xml_dom.getElementsByTagName('gnc:GncCustomer')
  already_entered = 0
  entered = 0
  for row in reader:
    member_id = row[0]
    invoice_id = row[1]
    invoice_date = row[2]
    invoice_amount = row[3]
    invoice_desc = row[4]
    if not member_id.isdigit():
      # header line or not coded to a member?
      continue
    member_id = int(member_id)
    invoice_id = int(invoice_id)
    if invoice_exists(xml_dom, invoice_id):
      # print "Already entered %s (%s)" % (invoice_date,invoice_id)
      already_entered += 1
    else:
      if not member_exists(xml_dom, "No Name McGoo - fix me" , member_id):
        print "Skipping member id %s, invoice id %s - member doesn't exist" % (member_id, invoice_id)
      else:
        invoice_enter(s, member_id, invoice_id, invoice_date, invoice_amount, invoice_desc )
        entered += 1
  print "Invoices entered from last two months: %s (%s already existed)" % (entered, already_entered)
  return True
def get_guid_from_member_id(nodes,member_id):
  member_id = pad_number(member_id)
  for node in nodes:
    for child in node.childNodes:
      if child.nodeName == 'cust:guid':
        guid = child.childNodes[0].data
      if child.nodeName == 'cust:id':
        if member_id == child.childNodes[0].data:
          return guid   
def import_members_from_file(s, dom, csvfile):
  dialect = 'excel-tab'
  csvfile.seek(0)
  reader = csv.reader(csvfile, dialect)
  already_entered = 0
  entered = 0
  for row in reader:
    member = row[0]
    member_id = row[1]
    if not member_id.isdigit():
      # header line?
      continue
    member_id = int(member_id)
    if member_exists(dom, member, member_id):
      #print "Already entered %s (%s)" % (member, member_id)
      already_entered += 1
    else:
      member_enter(s, member, member_id )
      entered += 1
  print "Members entered from last two months: %s (%s already existed)" % (entered, already_entered)
  return True
def get_gnucash_session(path):
  with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    return Session(path, is_new = False)
def member_enter(s, member, member_id):
  print "Entering %s with id %s" % (member, member_id)
  book = s.book
  root = book.get_root_account()
  commod_table = book.get_table()
  USD = commod_table.lookup('CURRENCY', 'USD')
  member_id = pad_number(member_id)
  new_customer = Customer(book, member_id, USD, member)
def invoice_enter(s, member_id, invoice_id, invoice_date, invoice_amount, invoice_desc):
  book = s.book
  root = book.get_root_account()
  commod_table = book.get_table()
  USD = commod_table.lookup('CURRENCY', 'USD')
  invoice_id = pad_number(invoice_id)
  member_id = pad_number(member_id)
  invoice_date_object = datetime.datetime.strptime(invoice_date, "%Y-%m-%d %H:%M:%S") 
  member = book.CustomerLookupByID(member_id)
  assert( member != None )
  assert( isinstance(member, Customer) )
  assets = root.lookup_by_name("Assets")
  recievables = assets.lookup_by_name("Recievables")
  income = root.lookup_by_name("Membership Dues")
  invoice = Invoice(book, invoice_id, USD, member, invoice_date_object )
  invoice_value = gnc_numeric_from_decimal(Decimal(invoice_amount))
  invoice_entry = Entry(book, invoice)
  invoice_entry.SetDescription(invoice_desc)
  invoice_entry.SetQuantity( GncNumeric(1) )
  invoice_entry.SetInvAccount(income)
  invoice_entry.SetInvPrice(invoice_value)
  invoice_entry.SetDate(invoice_date_object)
  invoice_entry.SetDateEntered(invoice_date_object)
  print "Entering invoice %s %s %s" % (invoice_id, invoice_date, invoice_amount)
  invoice.PostToAccount(recievables, invoice_date_object, invoice_date_object, "", True)
def pad_number(num):
  return "%(number)06d" %   'number': num  
def member_exists(dom, member, member_id):
  member_id = pad_number(member_id)
  for node in dom.getElementsByTagName('gnc:GncCustomer'):
    for child in node.childNodes:
      if child.nodeName == 'cust:name':
        for grandchild in child.childNodes:
          if member == grandchild.data:
            return True
      if child.nodeName == 'cust:id':
        for grandchild in child.childNodes:
          if member_id == grandchild.data:
            return True
  return False
def invoice_exists(dom, invoice_id):
  invoice_id = pad_number(invoice_id)
  for node in dom.getElementsByTagName('gnc:GncInvoice'):
    for child in node.childNodes:
      if child.nodeName == 'invoice:id':
        for grandchild in child.childNodes:
          if invoice_id == grandchild.data:
            return True
  return False
def gnc_numeric_from_decimal(decimal_value):
    sign, digits, exponent = decimal_value.as_tuple()
    # convert decimal digits to a fractional numerator
    # equivlent to
    # numerator = int(''.join(digits))
    # but without the wated conversion to string and back,
    # this is probably the same algorithm int() uses
    numerator = 0
    TEN = int(Decimal(0).radix()) # this is always 10
    numerator_place_value = 1
    # add each digit to the final value multiplied by the place value
    # from least significant to most sigificant
    for i in xrange(len(digits)-1,-1,-1):
        numerator += digits[i] * numerator_place_value
        numerator_place_value *= TEN
    if decimal_value.is_signed():
        numerator = -numerator
    # if the exponent is negative, we use it to set the denominator
    if exponent < 0 :
        denominator = TEN ** (-exponent)
    # if the exponent isn't negative, we bump up the numerator
    # and set the denominator to 1
    else:
        numerator *= TEN ** exponent
        denominator = 1
    return GncNumeric(numerator, denominator)
if __name__ == "__main__":
  sys.exit(main())

One of the ways in which EFI is *actually* better than BIOS is its native support for multiple boot choices. All EFI systems should have an EFI system partition which holds the OS bootloaders. Collisions are avoided by operating system vendors registering a unique name here, so there's no risk that Microsoft will overwrite the Fedora bootloader or whatever. After installing the bootloader the OS installer simply sets an NVRAM variable pointing at it, along with a descriptive name and (if they want) sets the default boot variable to point at that. The firmware will then typically provide some mechanism to override that default by providing a menu of all the configured variables.

This obviously doesn't work so well for removable media, where otherwise you'd have an awkward chicken and egg problem or have to force people to drop to a shell and run the bootloader themselves. This is handled by looking for EFI/boot/boot(architecture).efi, where architecture depends on the system type - examples include bootia32.efi, bootia64.efi and bootx64.efi. Since vendors have complete control over their media, there's still no risk of collisions.

Why do we care about collisions? The main reason this is helpful is that it means that there's no single part of the disk that every OS wants to control. If you install Windows it'll write stuff in the MBR and set the Windows partition as active. If you install Linux you'll either have to install grub in the MBR or set the Linux partition as active. Multiple Linux installations, more problems. It's very, very annoying to handle the multiple OS case with traditional BIOS.

This was all fine until UEFI 2.3 added section 3.4.1.2 of the spec, which specifies that in the absence of any configured boot variables it is permitted for the firmware to treat the EFI system partition in the same way as removable media - that is, it'll boot EFI/boot/bootx64.efi or whatever. And, if you install Windows via EFI, it'll install an EFI/boot/bootx64.efi fallback bootloader as well as putting one in EFI/microsoft.

Or, in other words, if your system fails to implement the boot variable section of the specification, Windows will still boot correctly.

As we've seen many times in the past, the only thing many hardware vendors do is check that Windows boots correctly. Which means that it's utterly unsurprising to discover that there are some systems that appear to ignore EFI boot variables and just look for the fallback bootloader instead. The fallback bootloader that has no namespacing, guaranteeing collisions if multiple operating systems are installed on the same system.

It could be worse. If there's already a bootloader there, Windows won't overwrite it. So things are marginally better than in the MBR days. But the Windows bootloader won't boot Linux, so if Windows gets there first we still have problems. The only solution I've come up with so far is to have a stub bootloader that is intelligent enough to scan the EFI system partition for any other bootloaders and present them as a menu, and for every Linux install to just blindly overwrite bootx64.efi if it already exists. Spec-compliant firmware should always ignore this and run whatever's in the boot variables instead.

This is all clearly less than optimal. Welcome to EFI.

On Friday, June 24, the last day of Open Source Bridge, I won a scarf! It says, "Open Source Citizen." I wasn't the only one. The attendees nominated people who "made an extra effort to help others and share their knowledge," and the conference committee chose the three people they felt exemplified this. The winners were Sumana Harihareswara, Igal Koshevoy, and me. In case you don't know Sumana, she's the new Wikimedia Volunteer Development Coordinator, a friend, and a commenter here at Asheeshworld. In the photo of her and her new scarf, you can also see Ward Cunningham. I suggest reading her wrap-up about the conference, and checking out the notes from her talk. I haven't met Igal, but I've learned he's a fixture of the Portland tech scene. He's user number one on opensourcebridge.org and one of the original contributors to calagator, the most important event calendar in the Portland tech world. That's just a brief summary; follow him on Twitter to keep in touch. I started the conference a little shy, not wanting to introduce myself to people, so I hung out with Sumana. She talked to everyone she could find, and there I was standing next to her. On the first day, it was only because of Sumana's outgoingness that people knew who I was. On the second and third days, I was involved in two sessions: a panel on open source communities and a talk about the OpenHatch training missions. I was extremely honored to be chosen. There were so many other spectacular people I met for the first time, like Valerie Aurora, Jacinta Richardson, Brian Aker, and Alex Linsker. These are all people with three crucial traits: they're extremely knowledgeable, friendly, and opinionated. I'm glad I had a chance to attend and meet them! Photo credits: Thanks to Noah Swartz for the photo of me. Sumana's photo comes from her post to wikitech-l. Igal's comes from his Open Source Bridge user profile.

As the local geek I get all sorts of (Linux) questions asked, like "How can you delete the nth line with Sed?" or "Is there a way to search for the following in a file?" (the latter being a request to construct a regular expression for grep). And while I'm pretty sure, you can find answers for such questions quickly with $SEARCH_ENGINE I find myself generally typing the answer into the IM session. This, and just seeing Bernd's post about Vim on Planet Debian, prompted me to start a little, irregular series of posts, to which I can point people, whenever I get asked such questions. I start this off with a tip for Vim, a very powerful text editor. The problem is simple: you get (source code) files with trailing whitespaces (sometimes accumulated in "empty" lines). This makes diffing (and merging) difficult. Thus the question is: how do I prevent that from happening? How do I notice, that I have whitespaces at the end of a line? The solution consists of two lines in your .vimrc:

:highlight TrailWhitespace ctermbg=red guibg=red
:match TrailWhitespace /\s\+$\  \+\ze\t/

If you just want this functionality if syntax highlighting is also active, then you should use

:autocmd Syntax * syn match TrailWhitespace /\s\+$\  \+\ze\t/

instead of the :match line. The regular expression used in both cases matches trailing whitespaces and whitespaces in front of tabs. Of course there are several other options on how to do this or what you might want to highlight, but that would be beyond the scope of this little post.

Somehow, even though I'd know about it for the longest time, I had never get around to reading George R.R. Martins A Song of Ice and Fire series until I heard about the Game of Thrones HBO series. Then I thought I had better hurry up and read the books as TV adaptations tend to be inferior. (I must say HBO did a pretty good job though.) I was instantly hooked and finished all four books in a week. Today the long-awaited fifth installment called A Dance with Dragons and I immediately got myself a copy. (Nowadays I mostly do my recreational reading with the Kindle app on my Android smartphone.) As before, I'm spellbound. I'm already about 25% through it and loving it. What I love about Martin is that he really gets how traditional, "pagan", "feudal" societies work. Take religion for instance. A lot of Sci-Fi/Fantasy authors if they deal with the subject at all usually give a cartoon version of polytheism that owes more to Christian canards and Dungeons & Dragons than any existing human religion. But in Westeros, one can be simultaneously devout, amoral, superstitious, and cynical in a way that makes a lot of sense to me as a Hindu. Or class. Being a peasant in Martins' world is horrible. Your life is completely at the mercy of Lords and Ladies and can end in an instant. But the upper class despite its wealth and power is trapped in its own way too. They must constantly protect their status from jealous competitors and a reversal of political or military fortune can destroy an ancient noble family just as suddenly as a bad harvest can destroy a family of farmers. Like the other books in the series, this one has a huge cast of characters whose stories weave in and out with each chapter told mainly from one characters viewpoint. This is a tactic which I for one found helpful to keep my attention from flagging but it means you can't really enjoy it unless you've read the previous volumes in order first. I definitely think it's worth the effort though.

Seems like everybody s writing Google+ reviews these days, but most of the ones I ve found seem to be fluff pieces. I haven t used it long, but have some initial impressions to share. First, an analogy. Facebook reminds me of the sleazy guy selling stuff from his car down the street. They do things like change privacy defaults when they think it s good for Facebook, rather than good for you. Google+ reminds me of an Apple product. It s beautiful, easy to learn, but locks everything down and is lacking some critical features. So here goes with the details. The good I think that this can be best summed up by: it s not Facebook. Facebook has, perhaps, set an incredibly low bar but still, this holds. Facebook is the only website I can ever remember using that changes things so much, so often, and so completely that I keep having this feeling of not knowing how to do things. Try maintaining a Facebook page or two and you ll especially feel my pain then. But even the basics: how many times have I accidentally posted a partial comment because I pressed Enter to start a new paragraph (that has only been the button used for that for, hmm, let s see now, decades), and instead it posted the comment. Principle of least surprise, anyone? Google+ has a simple and, on the surface anyway, intuitive interface. However, it does get muddy; more on that below. I m going to spend a lot more space on the bad below, but don t let that diminish my excitement about having a viable alternative to Facebook. I am keen to get rid of that monster. The Bad I ll start of this section with the fact that Google+ is a tightly-controlled walled garden. There is no way to take a copy of your status updates, comments, etc. and back them up with your own devices. If your Google account goes away, so do all your updates about your kids first words. Facebook does have this feature these days. It has broken half the time, but it exists, works, and I use it. Twitter doesn t have the feature built in, but its API makes it pretty easy; you could easily use my twidge program to do this, for instance. But whatever you put in G+ stays in G+. There is no autoposting it to twitter or Facebook, no backups, nothing. I m disappointed in that, given Google s prior attitude about openness to individual data ownership. (Update 7/12/2011: There is a way to get a backup, which I hadn t noticed; see comments below.) Almost as bad, there s also no way to get data in. So I now have identi.ca, twitter, Facebook, and now also G+ accounts. When I post an item in identi.ca, it autoposts to twitter, and then it autoposts from twitter to Facebook. Handy one place to notify people of my public activities. My uploads to Flickr and blog posts also auto-post to Facebook, so people I m friends with there get a picture of what I m doing outside of Facebook without me manually having to link it in. Not so with G+. There is no way to even add an RSS feed for my blog to auto-post to G+ as there is with Facebook. If you use G+, the only way to get stuff into it is with a keyboard. Poor integration with other Google apps is also an issue. G+ is nowhere near Google s first social app. They also have Blogger, Google Reader, Buzz, Gmail, etc. So here s the rub. These aren t integrated well, and when they have tried to integrate them, they ve done an exceedingly poor job of it. As an example: Buzz is a twitter-like service for posting updates. Very similar to what G+ does, right? Well, the best you can do is link your Buzz account to a separate tab on your profile. You can t even have your Buzz updates flow straight into G+, from what I can see. You have a separate list of followers in Buzz from G+. Reader is even worse; it has a sharing feature, but it s not G+ aware, so it goes to a reader inbox. But what takes the cake is the integration with Gmail Contacts. Sounds easy? Not entirely. When you add someone to G+, apparently it sometimes links the record with their entry in Contacts. And the help gives you the handy warning that when you remove them from G+, you have the option of removing them from Contacts, which could impact what you see on a mobile device. It is unclear what happens when it doesn t link the record, or how it decides which record to link (some people have more than one person in a family sharing phone numbers and email addresses, for instance.) One very nice feature of G+ is you can add email addresses, even if they aren t G+ members. Then when you post updates to the circle you ve added them to, G+ offers to send them an email with an update. Very handy. Except the undocumented part is that if you select Your circles (meaning all your circles), rather than individually ticking the box by each circle, they don t get the email. And they also don t get it if you set it Public. You have to select their specific circle, which is a significant difference from how people with G+ accounts are treated. And that s just an example. There are a ton of things that work one way 95% of the time, but have non-obvious exceptions. Some of these exceptions are documented in the help, and some aren t. I m still confused about the integration with Picasa. G+ help says that you can upload unlimited numbers of photos for free (though they will be downscaled at a certain point). Picasa has definite storage limits. But G+ uploads are showing up in Picasa. Do I really get free storage by uploading to the same place via a different tool? Very odd. And finally, a gripe about web standards. I ve been using Firefox/Iceweasel 3.5, which came out in Debian squeeze and is still supported there. It is also pretty darn new by corporate IT standards. And Google Plus refuses to let me log in with it, saying it s incompatible. Perhaps it can t do some fancy animation, but then again I don t really care. Seems Google has forgotten the old RFC adage: be liberal in what you accept and conservative in what you generate. If you can detect that my browser won t animate something right, then you could give me a stripped-down version of the page rather than an error message.

English version of this posting lese stattdessen die deutsche Version The head system administrator of a henceforth unnamed company I happen to be employed at wants to force us to use the shell s persistent history feature disabled by default in mksh as is sensible, for privacy reasons on all systems, so he can retrace what we do (of course, his wording was more like, so he can reconstruct how I fixed a problem the last time it occurred yeah, sure). Now I ve escalated this to the boss, since this just cannot happen, what ll be next, video monitoring our offices or what and the boss said he sees my concerns and would like a meeting to get an agreement between all parties involved. Most sysadmin coworkers actually don t even think about this, they are sheeple, really; but my project manager (for when I m not a sysadmin but a programmer) and the leaderess of our QA department agree with me, luckily. Now I m in dire need of arguments, especially German-Legalese, to fight this total surveillance of employees, mostly targetted at these who use the command line almost exclusively, which I m a prime example of. Please mail me at my address, which is not junk@mirbsd.org but with tg instead of the waste part here to catch address grabbers (sending mail to that address yields a 24-hour total blacklist of your IPv4 address I ve yet to get enough spam via IPv6 to say filtering it is worth the cost). Thanks in advance!

---

Update 2011-07-12: We talked, and it was agreed to not force a shell history; instead I will now put problem solutions into smallish scripts and commit them into the sysadmin SCM. It was not solved using legal arguments, as many people commented would have been difficult; instead, they acknowledged that I was concerned with this, and we searched for a different solution to the head sysadmin s original concerns (apparently, to figure out how to fix things without reinventing the fix every time, and to find out what was changed on a server in case it breaks).

During my stay in India, I sent weekly post cards to my girlfriend. Not all of them arrived, but even now, seven weeks after my return, occasionally a card arrives. Yesterday, a card arrived that I have sent on April 30th from Leh, by giving it to the hotel reception. According to the stamp, it has reached the post office in Leh already by June 17th! But even that does not explain the delay completely but another stamp on the card does: Missent to MPC Jakarta 10900 . I have no idea why anyone would think this card was destined for Indonesia, but it s nice to be able to send cards from places where I haven t even been close to.
Flattr this post

Filing bug reports is the most common way for users to contribute. Even if it s not too difficult, I ll give you some advice to improve the quality of your reports. After all, when you go out of your way to report a bug, it is in the hope to see it fixed so let s see how we can make this more likely. 1. Try to reproduce the bug If you can t reproduce the bug, it s next to impossible to find the root cause and thus to fix it. In that case, I would suggest you to wait until you experienced the bug multiple times. Maybe you ll be able to find something that triggers it (or that makes it more likely to encounter it). If the application has a debug/verbose mode, it might be a good idea to enable it until you experience the bug a second time. The generated log might be helpful for the developer to understand what happens exactly. So don t file bug reports straight away unless you can reproduce it. The exception to the rule is when the application gives some useful information like a core-dump, a back-trace or an error message. Obviously if the bug happens during an upgrade, it s difficult to reproduce it (unless you have multiple computers) but you should still report it. Be sure to include all the relevant information (versions of packages before and after the upgrade, logs of the upgrade, etc.). 2. Do your best to identify the faulty package When you report a bug to Debian, you must assign it to a package. While there are pseudo-packages useful for problems which are not directly attributable to a real package, in most of the cases you should report a bug against the specific package that seems to be the cause of the problem you encountered. In turn this often requires you to attribute the problem to a file (for example the executable of the application that triggers the bug). Once you have a filename you can use dpkg -S to identify the corresponding package.

$ dpkg -S /usr/bin/hamster-time-tracker
hamster-applet: /usr/bin/hamster-time-tracker

Note that reportbug accepts a filename as parameter and will do the above conversion for you. If you only know the name of the application (but not the filename of the associated executable), you can use dpkg -S with a pattern to let it return all possible matches:

$ dpkg -S hamster
hamster-applet: /usr/share/applications/hamster-applet.desktop
hamster-applet: /usr/share/gnome/help/hamster-applet/es/statistics.page
[ ]
hamster-applet: /usr/bin/hamster-time-tracker
[ ]

Or you can also verify in the list of installed packages:

$ dpkg -l "*hamster*"
Desired=Unknown/Install/Remove/Purge/Hold
  Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
 / Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
 / Name            Version         Description
+++-===============-===============-==============================================
ii  hamster-applet  2.32.1-1        time tracking applet for GNOME

3. Verify that the bug is not already reported and/or fixed If there s a newer version of the software available, it s a good idea to try to reproduce the problem with this version too. Because the developers tend to care only about the latest version, they will want to reproduce it with this version, and they will be annoyed if the problem that you reported is already fixed. That s why bug reports of users of testing/unstable tend to be more useful than bug reports of stable users. In any case, you want to verify that the bug has not yet been reported: filing a duplicate bug is useless and only generates more work for the developers to merge both bugs together. On the opposite, it s highly appreciated to add supplementary information to an existing bug report, even a simple confirmation that the bug still exists on a newer version is useful. Note that reportbug will automatically show you the list of open bugs before allowing you to submit a new one. 4. Use reportbug While the Debian bug tracking system allows anyone to submit a new bug with a simple mail, you should really use a dedicated program like reportbug (or reportbug-ng) because it will automatically include lots of useful information in the generated report (version of dependencies, current architecture, etc.) and will assist you in all the steps. 5. Describe the problem so that the developer can reproduce it Ideally your report should include everything required so that the developer can reproduce the problem on his system. If a given document triggers the bug, attach it. Describe the steps required to reproduce the bug in great details just like you would explain it to your grand-ma. Explain how you expected the program to react and what happened instead. You can learn much more on how to draft a good bug report in this article: How to report bugs effectively. It s a bit long but well worth it if you intend to report bugs and thus interact with developers. 6. Be kind and willing to help When you draft a bug report, keep in mind that you re writing to a volunteer free software developer and not to a customer service. You should be respectful and follow the usual rules of courtesy. Developers attention is scarce and should not be wasted. Be willing to help, if the developer starts investigating your problem, he might need your help to get supplementary information (in particular if he can t reproduce it) and you should be ready to provide it. Thus it s important to keep whatever you need to reproduce the problem. In some cases, the Debian maintainer might be overworked and you can offer your help to forward the bug to the upstream bug tracker, it s always appreciated. If you re reasonably confident that the problem is not Debian-specific, you can do it straight away and set the forwarded field to the URL of the upstream bug report (for example with bts forwarded <bug> <url>). 7. Use the correct severity The Debian bug tracking system lets you set the initial severity of the bug report (in decreasing severity: critical, grave, serious, important, normal, minor, wishlist). Pick the correct severity according to the official definitions but don t misread them. In particular, don t over-inflate the severity: for instance if you lost some data due to a misuse of the software, it s not critical (i.e. rm -rf * doesn t warrant a critical bug against rm). If you use only a tiny part of a software, and that part doesn t work, the package might be unusable for you but it s not unusable for everybody, so it doesn t warrant the grave severity. The important severity is often a good choice in those cases. Do not under-estimate the severity either, if a problem is important enough that it must be fixed before the next stable release (for example a regression compared to the previous release), pick a release-critical severity (i.e. at least serious ). The maintainer and the release manager can always lower the severity if they do not agree with your initial judgment. And now, happy bug-reporting!

Follow me on Identi.ca, Twitter and Facebook. Or subscribe to this blog by RSS or by email.

6 comments Liked this article? Click here. My blog is Flattr-enabled.

Okay, so the story continues: Joe Brockmeier has posted a rebuttal to Nina Paley's post that I commented on earlier today. Basically, Joe says that Culture is not the same things as Software, and gives a lot of arguments as to why this would be true. But I can't agree with him. Let's see why. His first argument is "One of these things is not like the other". As in,

But it's far more reasonable and sensible to discuss Perl scripts and software written in assembler embedded on a chip together, and the rights that they come with, than to try to talk about rights to modify sculpture and essays.

I don't see how one could combine a perl script with an embedded assembler program, except perhaps over a network, or by using exec() two cases in which the FSF would consider that they are not even derivatives anymore, they are "at arm's length", and it would be legal to do something like this if one bit was released under the GPL while the other was not. At the same time, it's possible to engrave (part of) an essay into a statue, for example. Or make a technical apparatus which looks like a statue but which will play music if you come near it. Both would be valid cases of some artist combining two separate and (at first sight) distinct things into one work.

Sure, you could crowdsource sculpture, but I doubt it will have much practical application. (And certainly difficult over the Internet, but I digress...)

It's called RepRap, and it's very much possible over the Internet. Joe moves on, with 'Unintended consequences of forkable prose'.

Take the GNU Manifesto, for example. If you could "fork" the manifesto, I could easily see variant versions modified to justify open source instead of free software.

This argument had been brought forward during the discussions which Debian had on the GFDL vote, and has been seriously debunked there. In short: the ability and right to fork someone's prose does not, in any way, give you the right to misrepresent them. While I believe it should be okay for someone to base a theoretical Open Source Manifesto on the GNU Manifesto, I do not believe it should be okay for them to still keep calling it the GNU Manifesto. It's not the same thing anymore, so that should be clear. This is true within Free Software as well: if you fork, you give it a new name, so it becomes clear that you're not the original anymore. Moreover, I doubt the kind of prose found in the GNU Manifesto is the same kind of prose that Nina would like to see in a free culture movement. Not because the GNU Manifesto is a bad piece of writing it isn't but because Free Culture likely focuses on other things than manifestos. Next, we get "Solving a problem that doesn't exist"

The spark that fired off the GNU revolution was a legitimate problem because of non-free code, a user was unable to make use of hardware they (or their university) legitimately owned.

(...)

While Paley and others may want the same freedoms to go with cultural works, it is not a real problem in the same sense.

Pardon ye me? I am a musician. Not a very brilliant one, mind you, but I play the flute with much enthusiasm, and have done so in a number of places. If you've been to any debconf since 2007, surely you must've heard me play. Also, I've been singing in choirs since I was sixteen which, I just realized, means I've now been singing for more than half my lifetime. Time flies, it must be said. One of the orchestra that I've been playing in was an extended flute choir with a rather unusual instrumentarium: apart from 30-something flautists, the orchestra also had a piano and drummer, a number of guitar players, and a bass guitar. Such an unusual set of music means you have three options: you play original music, written specifically for your group; you play something written for a different set of instruments, without changing anything, and hope nobody notices that it doesn't actually sound very well; or you make modifications. You make a derivative work of the music, so you can actually play it. There, I've said the word. This isn't just true for music; I've yet to see the first theatre performance that is true, word-for-word, to the original script. Also, movies use music all the time. When they do, there are usually two options: either they have original pieces of music written specifically for their movie (as is the case for most Hollywood blockbusters), or they take a piece of existing music, and adapt it so it fits the events in the movie better. By, say, shortening it, so it doesn't run into the next scene. Or by repeating the refrain an extra time, so the words line up nicely with the action. Or by having a group of musicians record a version of that piece specifically for your movie, which is the same thing in everything except the speed so that the feet of the actor walking on the big screen hit the ground in sync with the music. Or... well. You get the idea. By claiming that artists have no need to modify and derive from others, the only thing Joe accomplishes is to show that he really, really, really doesn't know what the fuck he's talking about. Artists have a need to modify other artists' works all the time. The only difference with free software is that they either need to invoke the fair use clause, or need to pay big bucks to the original rightsholder. Sound familiar? Joe finishes up with stating that despite all of that, Free Culture is still possible. And:

That doesn't mean that free culture is impossible. It simply means that different standards apply.

Exactly: different standards apply. And why? There is no good reason for that. Because people make money from cultural works? Well, money is made from software, too, that has never been a good enough reason. Because Culture is not functional? How many in the Free Software movement would even be willing to only be doing software all the time, and not ever do anything remotely related to Culture? None, I would say. Culture is just as functional as Software, it just serves different functions. In the comments, Joe clarifies that he isn't against people preferring not to use the ND clause in the CC licenses they select, he's just against demanding that others do the same. That argument would make sense if not for the fact that the FSF demands, mostly, that Software Should Be Free!!1!. By not complaining about the GNU Manifesto, but complaining about Paley's "Rantifesto", which basically complains about the lack of such an environment in the free culture movement, Joe just makes a fool of himself. Anyway, 'nuff said.

Nina Paley, author/actor/director/animatrist of the excellent sita sings the blues, blogs about the FSF's double standards when it comes to freedom. To the FSF, Software Should Be Free!!1!, and it is willing to go to great lengths to accomplish that. For example, the FSF will create dracionian requirements on their Hardware endorsement criteria, that will most likely render it unusable, just for the remote possibility that they might convert a single soul to their religion. They will add propaganda to their documentation, and slap on a license that makes it illegal for anyone to modify or remove that propaganda. They will claim to make an update to that license, just to make their friends happy, but then do not make any effort to finish the job. Most of all, the FSF is led by a person who deserves our respect for jumpstarting a movement that has made all this free software even possible, but whose atypical behaviour might be interfering with his message these days. I have stated before that I'm not a member of the FSF, and that 'I have my reasons'. In case you were wondering: the above are my reasons. I can't be a member of, or be supporting of, any organization that uses such double standards in anything. In the case of Debian, it took us a long time to come to the conclusion that the GNU FDL is a non-free license. And though we came to that conclusion through other arguments than "Culture should be free, too" (specifically, the arguments boiled down to "if it's digital, it's software, and software should be free"), I have since come to the conclusion that there is nothing special about 'digital stuff' that should mean it should be more free than 'non-digital stuff'. If I hand you a piece of music on a sheet of paper, why should you not have the right to redistribute and/or modify that, while you should have the exact same right if I were to hand you the exact same paper on a digital medium? That makes no sense. So, in short, I agree with you, Nina: there is no Free Culture movement, and that's a shame; and while it would be nice if the FSF were to take a lead role in guiding such a movement, I find it highly unlikely that they will. The FSF may claim they care about Freedom, that doesn't mean they do; they care about Software Freedom. And while that certainly was an issue thirty years ago, when the FSF was formed, it isn't so much anymore today. There can only be one conclusion: the FSF is obsolete.

There is always been an ongoing debate about how to assign disk space into multiple partitions. I think that nowadays the best thing to do is to assign about 10G for the root filesystem for every desktop and server system because 10G is a small fraction of the disk space available (even the smallest laptops seem to all have disks larger than 100G nowadays). Even if 10G turns out not to be enough using separate filesystems for /var or /usr provides little benefit now that it s easy to resize the root filesystem with LVM and a separate /usr is known to be broken [1]. In a discussion on a private mailing list there was a suggestion that multiple filesystems should be used for security. DoS Attacks There are some minor security benefits in having multiple filesystems. If a critical program will fail when there is no free disk space then allowing an unprivileged process to use up all the space on that filesystem is a minor security issue, so having unprivileged processes not being permitted to write to important filesystems is a benefit. But most failures of this type are merely DoS attacks which usually aren t a big deal if you can control a local process there are usually lots of other ways of DoSing a system. Links Links have been the cause of many security issues in Unix over the years. Using different filesystems for different tasks can prevent the use of hard links in attacks aimed at exploiting race conditions. But even if you prevent hard links there are similar issues with symbolic links. SE Linux is one of many security improvements for Linux which allow restrictions on the creation of hard links. SE Linux also allows restricting the ability of processes to follow symbolic links, so a privileged process can be denied access to follow a sym-link that was created by an unprivileged process. NFS The subtree_check option in /etc/exports causes the NFS server to verify that file access is in the correct subtree. So if you export only one subdirectory of a filesystem to a given server then hostile code on that server (or on a network device which impersonates that server) can t access other subdirectories. This option is documented as having performance implications and working best for filesystems that are mostly read-only, for this reason it s turned off by default in recent versions of the NFS utilities. So if you want to NFS export /home then it s probably a good idea to have /home be on a separate filesystem to prevent attacks on the root filesystem. But of the systems with significant use of /home (IE anything other than accounts used solely for su - ) most of them have a separate filesystem for /home anyway so this shouldn t be an issue. SE Linux When mounting filesystems with SE Linux there is a context= mount option that allows specifying the context for all files on the filesystem. This can save a small amount of storage space for XATTRs and theoretically improve performance (although the difference is unlikely to show up on benchmarks for anything other than fsck). Generally the context mount option is only used for a filesystem that has a huge number of files with the same context, such as a mail spool that uses Maildir, Cyrus, or any of the other formats that involve one file per message. But again such data is generally stored on a separate filesystem for other reasons anyway. I found one interesting corner case in regard to SE Linux systems mounting files from an NFS server. When an NFS server exports multiple subdirectories of a filesystem mounted on /foo then if one NFS client running SE Linux is to mount two subdirectories of /foo with different contexts then the second mount attempt will give the error an incorrect mount option was specified . This is because as of kernel 2.6.18 by default it s not permitted to mount parts of the same filesystem with different mount options. The option nosharecache allows you to use different mount options, but does apparently permit some undesirable behavior in the case of hard links that cross between the subtrees. Thanks to Eric Paris for the tip about nosharecache. The best example I can think of for which you might want context mount options that differ among files that are used for the same purpose on an NFS mount is a web server which has data files and CGI-BIN scripts. So it seems that a SE Linux web server that mounts it s data over NFS and is at risk of having hard links between the CGI-BIN directory and the data directory is a corner case in which multiple filesystems is required for security. This seems to be a very unlikely case. Conclusion Servers that are deployed in the real world are complex enough that there are always systems with some unusual corner cases demanding configuration choices that aren t expected. There are some real corner cases for SE Linux where multiple filesystems are compelled for security or for a combination of security and best performance. But I wouldn t make a generic recommendation of using lots of filesystems for security. I think that the people who encounter the strange corner cases can usually work out that they need to do something different. So a small number of filesystems seems like a good general aim that doesn t conflict with security.

• [1] http://www.freedesktop.org/wiki/Software/systemd/separate-usr-is-broken

I recently heard about Instapaper on the Stack Exchange podcast. It's a service that makes it easy (they say) to save web pages for reading later, possibly on another machine or device. I've been doing that for some years now, using the "File/Save" feature in my browser. Last year, this became lots better, when I found the MAFF add-on to Firefox. MAFF saves the page, and any images it includes, in a zipped file, which can later be viewed directly in Firefox. Or Iceweasel, in my case. It's much handier to have everything in one file, rather than a foo.html file plus a foo_files/ directory, which is what Firefox usually saves as. This does not make it possible to, for example, read the page on my phone, since I've saved the file on my laptop, but I can live with that. However, the complete autonomy of this solution is much to my liking. I have a "read and review" folder for things I need to read. After I'm done with something, I can easily delete it. If I want to keep it permanently, I can move it to an archive folder. I haven't got a solution for full text searching through my growing archive of MAFF files, but I'll figure that out some day, too, I'm sure.