Search Results: "jam"

29 August 2011

Benjamin Mako Hill: Donner Pass

/copyrighteous/images/bierstadt-donner_lake.png In the Peabody Essex Museum a couple weeks ago, I saw a beautiful landscape by Albert Bierstadt of Donner Pass whose label referenced the famous Donner Party of 1846 and their, "sensational story of privation, cannibalism, and death." I would reorder that sentence.

28 August 2011

Benjamin Mako Hill: An App For That

SeeClickFix makes a mobile application you can use to report Boston drivers using their smartphones while driving, while driving.

18 August 2011

Raphaël Hertzog: People behind Debian: Peter Palfrader, Debian System Administrator

You might not know who Peter is because he s not very visible on Debian mailing lists. He s very active however and in particular on IRC. He was an admin of the OFTC IRC network at the time Debian switched from Freenode to OFTC. Nowadays he s a member of the Debian System Administration team who runs all the debian.org servers. If you went to a Debconf you probably met him since he s always looking for new signatures of his GPG key. He owns the best connected key in the PGP web of trust. He also wrote caff a popular GPG key signing tool. Raphael: Who are you? Peter: I m Peter Palfrader, also known as weasel. I m in my early 30s, born and raised in Innsbruck, Austria and am now living and working in Salzburg, Austria. In my copious free time, other than help running Debian s servers I also help maintaining the Tor project s infrastructure. Away from the computer I enjoy reading fiction (mostly English language Science Fiction and Fantasy), playing board games and going to the movies. Weather permitting, I also occasionally do some cycling. Raphael: How did you start contributing to Debian? Peter: I installed my first Debian the week slink came out. That was Debian 2.1 for the youngsters, in early 1999. The one thing I immediately liked about slink was that Debian s pppd supported RAS authentication which my university s dial-up system required. No way I d go back to SuSE 5.3 when I had working Internet with my Debian box. :) During that year I started getting involved in the German language Debian channel on IRCnet which got me in contact with some DDs. Christian Kurz (<shorty>) was working on Debian QA at the time and he asked my help in writing a couple of scripts. Some of that work, debcheck, still produces parts of the qa.d.o website, tho the relevance of that nowadays is probably negligible. While trying to learn more Perl earlier, I had written a program to produce syntax highlighted HTML for code snippets in various languages. I didn t really know what I was doing but it kinda worked, and probably still does since I still get mail from users every now and then. I figured that it would be really nice if people could just get my software together with Debian. According to code2html s Debian changelog the initial release of the package was done on a weekday at 2:30 in the morning early in 2000, and if my memory serves me correctly, shorty uploaded it shortly afterwards. I started packaging a couple of other piece of software and in the same year I sent my mail to the debian account managers to register my intent to become a DD. No new developers where being accepted at that time since the DAMs wanted to overhaul the entire process so I wasn t surprised to not get any immediate reply. Of course what the silence also meant was that the mail had been lost, but I only learned of that later when I took all my courage to ask DAM about the status of application a couple months later. Once that was sorted out I was assigned an AM, did the usual dance, and got my account late in November 2000. Raphael: Four years ago, the Debian System Administration team was a real bottleneck for the project and personal conflicts made it almost impossible to find solutions. You were eager to help and at some point you got dropped as a new member in that team. Can you share your story and how you managed the transition in the difficult climate at that time? Peter: Ah, that was quite the surprise for an awful lot of people, me included. Branden Robinson, who was our DPL for the 2005-2006 term, tried to get some new blood added to DSA who were at the time quite divided. He briefly talked to me on IRC some time in summer 2005, telling me I had come recommended for a role on the sysadmin team . In the course of these 15 minutes he outlined some of the issues he thought a new member of DSA would face and asked me if I thought I could help. My reply was cautiously positive, saying that I didn t want to step on anybody s toes but maybe I could be of some assistance. And that was the first and last of it, until some fine November day two years later I got an email from Phil Hands saying I ve just added you to the adm group, and added you to the debian-admin@d.o alias. and welcome on board . *blink* What!? My teammates at the time were James Troup (elmo), Phil Hands (fil), Martin Joey Schulze and Ryan Murray (neuro). The old team, while apparently not on good terms with one another, was however still around to do heavy lifting when required. I still remember when on my first or second day on the team two disks failed in the raid5 of ftp-master.debian.org aka ries. Neuro did the reinstall once new disks had arrived at Brown University. I m sure I d have been way out of my league had this job fallen to me. Fortunately my teammates were all willing and able to help me find whatever pieces of information existed that might help me learn how debian.org does its stuff. Unfortunately a lot of it only existed in various heads, or when lucky, in one of the huge mbox archives of the debian-admin alias or list. Anyway, soon I was able to get my hands dirty with upgrading from sarge to etch, which had been released about half a year earlier. Raphael: I know the DSA team has accomplished a lot over the last few years. Can you share some interesting figures? Peter: Indeed we have accomplished a lot. In my opinion the most important of these accomplishment is that we re actually once again a team nowadays. A team where people talk to one another and where nobody should be a SPoF. Since this year s debconf we are six people in the admin team: Tollef Fog Heen (Mithrandir) and Faidon Liambotis (paravoid) joined the existing members: Luca Filipozzi, Stephen Gran, Martin Zobel-Helas, and myself. Growing a core team, especially one where membership comes with uid0 on all machines, is not easy and that s why I m very glad we managed to actually do this step. I also think the infrastructure and our workflows have matured well over the last four years. We now have essential monitoring as a matter of course: Nagios not only checks whether all daemons that should be running are in fact running, but it also monitors hardware health of disks, fans, etc. where possible. We are alerted of outstanding security updates that need to be installed and of changes made to our systems that weren t then explicitly acked by one of us. We have set up a centralized configuration system, puppet, for some of our configuration that is the same, or at least similar, on all our machines. Most, if not all, pieces of software, scripts and helpers that we use on debian.org infrastructure is in publicly accessible git repositories. We have good communication with other teams in Debian that need our support, like the ftp folks or the buildd people. As for figures, I don t think there s anything spectacular. As of the time of our BoF at this year s DebConf, we take care of approximately 135 systems, about 100 of them being real iron, the other virtual machines (KVM). They are hosted at over 30 different locations, tho we are trying to cut down on that number, but that s a long and difficult process. We don t really collect a lot of other figures like web hits on www.debian.org or downloads from the ftp archive. The web team might do the former and the latter is pretty much impossible due to the distributed nature of our mirrors, as you well know. Raphael: The DSA team has a policy of eating its own dog food, i.e. you re trying to rely only on what s available in Debian. How does that work out and what are the remaining gaps? Peter: Mostly Debian, the OS, just meets our needs. Sure, the update frequency is a bit high, we probably wouldn t mind a longer release cycle. But on the other hand most software is recent enough. And when it s not, that s easy to fix with backports. If they aren t on backports.debian.org already, we ll just put them there (or ask somebody else to prepare a backport for us) and so everybody else benefits from that work too. Some things we need just don t, and probably won t, exist in Debian. These are mainly proprietary hardware health checks like HP s tools for their servers, or various vendors programs to query their raid controller. HP actually makes packages for their stuff which is very nice, but other things we just put into /usr/local, or if we really need it on a number of machines, package ourselves. The push to cripple our installers and kernels by removing firmware was quite annoying, since it made installing from the official media next to impossible in some cases. Support for working around these limitations has improved with squeeze so that s probably ok now. One of the other problems is that especially on embedded platforms most of the buildd work happens on some variation of development boards, usually due to increased memory and hard disk requirements than the intended market audience. This often implies that the kernel shipped with Debian won t be usable on our own debian.org machines. This makes keeping up with security and other kernel fixes way more error prone and time intensive. We keep annoying the right people in Debian to add kernel flavors that actually boot on our machines, and things are getting better, so maybe in the future this will no longer be a problem. Raphael: If you could spend all your time on Debian, what would you work on? Peter: One of the things that I think is a bit annoying for admins that maintain machines all over the globe is mirror selection. I shouldn t have to care where my packages come from, apt-get should just fetch them from a mirror, any mirror, that is close by, fast and recent. I don t need to know which one it was. We have deployed geodns for security.debian.org a while ago, and it seems to work quite well for the coarse granularity we desired for that setup, but geodns is an ugly hack (I think it is a layer violation), it might not scale to hundreds or thousands of mirrors, and it doesn t play well with DNSSEC. What I d really like to see is Debian support apt s mirror method that I think (and I apologize if I m wronging somebody) Michael Vogt implemented recently. The basic idea is that you simply add deb mirror://mirror.debian.org/ or something like that to your sources.list, and apt goes and asks that server for a list of mirrors it should use right now. The client code exists, but I don t know how well tested it is. What is missing is the server part. One that gives clients a mirror, or list of mirrors, that are close to them, current, and carry their architecture. It s probably not a huge amount of work, but at the same time it s also not entirely trivial. If I had more time on my hands this is something that I d try to do. Hopefully somebody will pick it up. Raphael: What motivates you to continue to contribute year after year? Peter: It s fun, mostly. Sure, there are things that need to be done regularly that are boring or become so after a while, but as a sysadmin you tend to do things once or twice and then seek to automate it. DSA s users, i.e. DDs, constantly want to play with new services or approaches to make Debian better and often they need our support or help in their endeavors. So that s a constant flow of interesting challenges. Another reason is that Debian is simply where some of my friends are. Working on Debian with them is interacting with friends. I not only use Debian at debian.org. I use it at work, I use it on my own machines, on the servers of the Tor project. When I was with OFTC Debian is what we put on our machines. Being a part of Debian is one way to ensure what Debian releases is actually usable to me, professionally and with other projects. Raphael: Is there someone in Debian that you admire for their contributions? Peter: That s a hard one. There are certainly people who I respect greatly for their technical or other contributions to Debian, but I don t want to single anybody out in particular. I think we all, everyone who ever contributed to Debian with code, support or a bug report, can be very proud of what we are producing one of the best operating systems out there.
Thank you to Peter for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

No comment Liked this article? Click here. My blog is Flattr-enabled.

5 August 2011

Benjamin Mako Hill: Care and Trust

When you care for somebody, it is difficult to tell them "no." When you trust somebody, you will tell them.

3 August 2011

Andrew Pollock: [life] Zoe at 15 months

Boy, the last three months have been very eventful, developmentally. First off, about 2 weeks after she turned 1, Zoe started walking in earnest. Within about 2 weeks of that there was no stopping her. It's funny to think she's only been walking for under 3 months and yet she's very stable on her feet. Secondly, she developed a mild egg allergy. Sarah was giving her some scrambled eggs (one egg's worth) for lunch one day, and she mixed in some cream cheese, and Zoe absolutely scoffed it all down, so she scrambled another egg and she scoffed that down too. She then broke out in some hives around her mouth and Sarah noticed that she was coughing a bit and a little wheezy when she put her down for her afternoon nap. Upon subsequent consultation with her paediatrician and a blood test, the allergy was confirmed and we now have to avoid eggs and Zoe was prescribed an Epi Pen Jr. We went and saw an allergist as a followup, and he did some skin tests that showed eggs were the only thing she's allergic to. He said her allergy was very mild in the grand scheme of things, and we're going to do another skin test when she's 2 to see if she's grown out of it. Eggs in baked goods are generally okay, as the offending protein is denatured at high temperatures. She's more allergic to the egg whites (1.56, which is in the "moderate" range) than the egg yolks (0.55, which is in the "low" range). Egg allergies are apparently something that one does eventually out grow, so that's a good thing. We'd also been keeping her off gluten and dairy because of a strong correlation between them and rather disgusting bowel motions. We've successfully reintroduced gluten with no adverse effects, and we've just started her back on cow's milk. We of course had our epic trip to the Grand Canyon, which Zoe handled absolutely marvelously. That was closely followed by a weekend down in LA, which was a bit more of a disaster as Sarah and I both came down with a 24 hour stomach bug that Zoe had had during the week. There's nothing quite like languishing in a non-childproofed hotel room, feeling like death, with a full of beans toddler busy running around jamming her fingers in drawers. Fortunately we were able to use a babysitter for a few hours respite, but it did rather completely mess up our plans for the weekend. Zoe did handle the drive to and from LA very well though. Zoe's been doing Saturday morning swim classes at the YMCA, which I've taken her to. I don't think she's enjoyed them as much as the indoor ones she did at the Sutton Swim School, where she went with Sarah previously, and I definitely don't feel like the classes are as structured. Unfortunately we just finished an 8 week session, and the next one filled up before we got a chance to book, so we're hoping to get back into something at the Sutton Swim School once an opening comes up. I might do something ad hoc with her on Saturdays at the Rinconada Pool in Palo Alto in the meantime. Zoe has also started going to regular daycare on Thursdays, so Sarah gets a day to focus on her University study. It's a home daycare that isn't too far away, and our friend's daughter goes there too, so there's someone she knows. The first few times Zoe got pretty upset when she was dropped off, but she barely makes a whimper now when Sarah drops her off. She's also usually down to one nap a day now. She has four teeth, top and bottom, with one molar definitely through and another three starting to make an appearance. I just bought an iBert safe-T-seat for my bike, because I've been itching to get out on the bike with Zoe on the weekends when I'm looking after her so Sarah can get some more study done. The weather is currently nice, and I've just learned about a bike trail that cuts across the bay in front of Moffett Field, so I'm keen to check that out. Apparently there's geese out there. I did my first decent ride with her last Sunday to the Mountain View Farmer's Markets and back. Zoe seemed to be okay with it. The helmet we got her drew a lot of comments from people at the markets. She's very much in the "point and grunt" stage at the moment. What's really cool is she'll nod or shake her head in response to yes/no questions. It's so cool when you can ask her a question and she solemnly nods her head in response. Often in the morning Sarah takes her to the park around the back of our home, and so now Zoe's gotten into that habit, so she'll scratch around at the front door and they head over there. The other day, Sarah opened the front door and she took off in the direction of the park, so Sarah just followed her to see where she went. Sure enough, she made her way over to the park. Tonight, Zoe fetched me my shoes and then dragged me over by the hand to the front door to demand that I take her out for a walk. She's also really getting into books now. She actively seeks out books, and has a bit of a flick through them, or brings one over and climbs into our laps, or backs up and sits in our laps if we're sitting on the floor, to have us read them to her. She's now strong enough to hold her bottle herself at bedtime, so I sit with her in my lap in the glider and read a story to her while she has her bedtime bottle. It's indescribably wonderful. She seems to understand quite a large array of words. We can go "Where's your <insert object here> and she'll look around the living room and generally find the item we're talking about. She also seems to understand questions about breakfast, lunch, going to the park, going to the bathroom, having a bath, etc as she seems to respond appropriately. In the speech department, I haven't really counted the words, but she's not saying a huge number of words. This morning, apparently she yelled "MAMA! MAMA! PARK!". "Park" being a new word for her. We bought an annual membership to the Children's Discovery Museum in San Jose, and we've probably taken her there half a dozen times or so since getting the membership. She particularly loves the water area. We finally got around to getting our back yard landscaped and made generally fit for a small child to run around in. Sarah's planted a butterfly garden in one corner, so hopefully once that's established we'll have some fun things for Zoe to watch from the window. I'm looking forward to planting some bulbs in the autumn. I expect the next 3 months will be more of the same. We've got some more overseas travel coming up, but a lot of relatives will be around, so hopefully we'll get a bit of downtime to ourselves. Zoe and I on my bike

1 August 2011

Christian Perrier: DD, 10 years

On July 30th, 2001, I received a mail from James Troup stating that I was a Debian developer, this 10 days after my Application Manager, namely Martin Michlmayr, sent his AM report. Summarizing these 10 years will be....interesting. Now that I'm on holidays, I'll try too cook something up. Sorry, folks, I completely forgot celebrating this last Saturday.

31 July 2011

Benjamin Mako Hill: Cost of Computing in Coal

 by jonasclemens, on Flickr
Creative Commons Attribution 2.0 Generic License by jonasclemens
Much of my academic research involves statistics and crunching through big datasets. To do this, I use computer clusters like Amazon's EC2 and a cluster at the Harvard MIT Data Center. I will frequently kick of a job to run overnight on the full HMDC cluster of ~100 computers. Some of my friends do so nearly every night on similar clusters. Like many researchers and engineers, it costs me nothing to kick off a big job. That said, computers consume a lot of energy so I did a little back-of-the-envelope calculation to figure out what the cost in terms of resources might add up to. An overnight job that uses a 100 computer cluster might use 800 computer-hours. Although power efficiency varies hugely between computers, most statistical analysis is CPU intensive and should come close to maximizing power consumption. According to a few sources [e.g., 1 2 3], 200 watts might be a conservative estimate of much a modern multi-CPU server will draw under high load and won't include other costs like cooling. Using this estimate, the overnight job on 100 machines would easily use 160 kilowatt hours (kWh) of energy. In Massachusetts, most of our power comes from coal. This page suggests that an efficient coal plant will generate 2,460 kWh for each ton of coal. That means that one overnight job would use 59 kg (130 lbs) of coal. In the process, you would also create 153 kg (338 lb) of CO2 and a bit under half a kilogram (about 1 lb) of nitrogen oxides and sulfur dioxide each. It's a very rough estimate but it certainly generates some pressure to make sure the research counts! Of course, I've written some free software that runs on many thousands of computers and servers. How many tons of coal are burnt to support laziness or a lack of optimization in my software? What is the coal cost of choosing to write a program in a less efficient, but easier to write, higher-level programming languages like Python or Ruby instead of writing a more efficient version in C?

26 July 2011

Benjamin Mako Hill: Dates and Memory

Recently, I was working with Daf and Rob on a little offline wiki project -- more on that soon -- and we realized that we needed to parse some dates in ISO 8601 format. One of us wondered out loud if there was a Python module that could help us. I offered to take a look. Turns out, less than two months before, someone had uploaded just such a module into Debian. The maintainer? Me.

Benjamin Mako Hill: Lawn Scrabble

The Acetarium, where I live, runs what we like to think of as the world's smallest artistic residency program by hosting artists, social scientists, hackers, and free software and free culture folks for periods of 1-3 months. Our most recently graduated resident, Noah, built a lawn scrabble set on the Media Lab ShopBot and held a Scrabble picnic this weekend with some former Acetarium residents and others. I don't really like playing Scrabble, so you can see me working on an essay (and verifying words) in the background. /copyrighteous/images/lawn_scrabble_01.jpg

/copyrighteous/images/lawn_scrabble_02.jpg Thanks to Ben Schwartz as I yoinked these pictures from his blog.

24 July 2011

Tore S. Bekkedal: Ut ya: English version

If a single man can display so much hatred
think only of how much love we all can display together.

Stine Renate H heim

I wrote a Norwegian post explaining my experience at Ut ya. I had taken this blog for dead, and had entirely forgotten that it was syndicated on Planet Debian. I don t want to let Google Translate make this disaster any worse than it is the translation of bullets into balls being particularly bad so the international attention the massacre has garnered in consideration, I am writing an English translation of my experiences. I feel somehow duty-bound to make people aware of what happened, but I don t want to get into anything else but a sober description of the events and some very brief reflections. There are many details I have chosen to omit. Others have written their experiences of the events at Ut ya. I wanted to write mine down as well, and get it out there . Partly, I want to write this down because I m unsure if I will remember all the details at a later point in time, although I think I d prefer it if I couldn t. I m also writing this because people are asking about my experiences and it s much better to have an URL to give them, lest I have to keep going through the same spiel over and over again. Our former Prime Minister and current labour movement demigod Gro Harlem Brundtland had recently left the island. I had been the cameraman for a video interview of her talking about Ut ya, and I was in the media group room encoding the video into a file suitable for YouTube, when someone else in the room startled and said that Twitter was full of messages about a loud explosion in Oslo. As the newspapers brought us information about the extent of the damages, a consensus arose that an informational meeting was in order. As soon as the current round of talks finished, we were gathered into the main hall. The meeting was duly held, and after the statement was made that a TV feed would be made available, I took it upon myself as the local alpha geek to make it happen. Of course, the situation caused both the wireless network and the GPRS networks to become totally unusable. As I was waiting for someone to set up a password, I took the opportunity to face the consequences of having eaten two bits of a microwavable dish called Hold-It the local equivalent of a Hot Pocket and went to the toilet. As I was in there, I first heard agitated shouting, then screams, then gunshots coming from just outside the toilets. More than anything else, it sounded like a toy gun. I was convinced that someone was making a joke in incredibly bad taste and I stormed out of the booth with the intent of halting it. As I tore the door open, I saw two of my comrades hiding in a recessed corner. Their facial expressions left absolutely no doubt that this was no toy. They signalled for me to get back in the booth. I closed the door, did a mental double-take in utter, complete confusion, and opened it again. They were still signalling. Had they not stood there, I would have run straight into the gunman; they saved my life. I looked out into the hallway, and I made eye contact with a young boy lying in a pool of blood. He was motioning for me to help him. I heard more gunshots from inside the building and retreated back inside. As I was trying to think through my next move, I realized that the decidedly insubstantial wood-fiber door would not resist any kind of bullets. I made my way out into the hallway, with the intent of escaping outside. At that point, I was of course not aware that there was an intention to kill as many as possible, so I thought that the open spaces outside would be a place of relative safety. Of course, this proved to be wrong and my life was probably saved a second time by one of the caf volunteers taking me into a hard-to-spot employee s bathroom. We sat there for ninety minutes. Always ready to make a run for it, ready for just about anything. A peculiar group dynamic arose with these two people with whom I had barely previously spoken. We came to share a strange sense of common destiny and gallows humour. One of them had seen the shooter and described the police uniform. I perceived it to be realistic that we were the only ones aware of the wounded outside the toilet. I tried to reach the emergency services, but all their lines were busy; the terror attack in Oslo had probably clogged their lines. I finally got through to the fire services, who could inform me that the police did know about the situation and were on their way. This was to take 90 minutes and by the time we evacuated, the young boy outside my door had perished. The despair I first saw in his eyes as I passed him, fleeing from one room to the other and the empty, blank stare as we left, are burned into me and they are images I will never in my life forget. Finally, the real police arrived. We walked out. I chose the path through the minor conference hall something I now regret. The sight was simply beyond my capacity to describe fully, and so terrifying that I barely remember the sight only the terror it struck in me. There were several people bunched up in a corner, a big amorphous heap of bodies. Some were conscious and yelled at me not to do anything that could startle the police, others lay still. Their bodies were all covered in blood, and a thick pool of blood extended at least a half-metre in all directions around them. The policeman across the hall was screaming orders at me, but he was screaming so loudly that I couldn t make out his words at first. We were first moved into the camp newspaper s offices. There were about eight of us there, I think, in addition to one girl who lay wounded. Towards the end she was drifting in and out of consciousness. We covered her with sweaters to keep her warm and one of us tried to at least temper her bleeding. The bullet had missed her heart, but by the entry wound it was clear that it was not by far. I do not know who this girl was or how she is now. I sat behind and never saw her face. The wounded were evacuated first. I don t remember how long we remained; I had lost all concept of time. In spite of protests from the group who knew him, one kid was put in handcuffs. At the time I didn t understand why, and the policeman seemed to say something almost to the effect that there was no reason for it at all. I didn t see when they undid his cuffs, but I remember thinking that this treatment made a terrible experience even worse for him. I tried my best to comfort him but knew it would be little help. Later, when things stabilised a little, we were told that he was handcuffed because he had come from an unsecured area. The police was extremely good at carefully explaining what was happening and why; this was a big help and I am grateful for it. Eventually we were moved out into the main corridor of the building, where we joined up into a group of about fifty. Only when I saw the two people who saved my life did any emotion other than mild confusion arise. I broke down shivering in tears in one of their arms. After a few seconds, I came back to my senses and realised that this was not the time. I quickly gathered myself, got the shaking under control, and sat down. We were given some chocolate and soda from the kiosk. I remember making an offhand remark that an inability to find joy in free candy was a sure sign of a bad situation. We all laughed out loud. Gallows humour is a coping mechanism, but in retrospect one almost feels guilty for it. We were shown out in a single file with hands above our heads. I remember an intense concern that someone would slip in the steep, muddy slope and create a misunderstanding. Outside, there were more bodies. Some under improvised covers a tarpaulin from the waffle stand, the deflated bouncy castle but some simply lay there. Everyone I met displayed a courage, a mental discipline and unity of purpose far beyond anything one would ever wish to expect from people this young. Everyone conducted themselves with an attitude that could almost be described as stiff upper lip . Safely across the fjord we were offered blankets. I was asked if I was aware of any injuries, and asked to lift my shirt and show my abdominal region. We were shown into the bus which took us to the hotel used by the survivors and their family. I simply cannot describe in any words the relief I felt when I was able to embrace my living comrades. It was completely unlike anything I had ever felt before in my life. The euphoric feeling was tempered only by the realisation that there would be many I could never see again, comrades whom I had taken great pride in calling my friends, with futures in the service of all mankind, futures I had previously found such great joy in pondering and guessing about. The feeling which continues to upset me the most, is the feeling that so many of my comrades left behind grieving families and friends. Torn away senselessly. I do not know how much more than this relatively sober account of the events on Ut ya I can muster. I would, however, like to offer some reflections. First of all, from the bottom of my heart, I want to thank the police who saved the lives of so many still on the island, the holidaymakers who took aboard swimmers into their boats and the rescue services staffed primarily by volunteers who have spared no effort in trying to soften the blow as much as they can. The opportunity to spend time with those comrades who underwent the same experience as myself has also been an immeasurable aid. I was also so relieved to find my very closest friend among the survivors, which has also been an indescribable help. If I can name a single positive in this tragedy: Had he arrived with his automatic weapon fifteen or twenty minutes prior, he would have arrived during the informational meeting, at a time when the major hall was absolutely jam-packed the death toll would be many times what it ended up being. I am agonisingly aware of the meager comfort this provides to those who have been bereft of their closest, but I do find some solace in this. We cannot sweep under a rug that this was without question! a political attack on the labour movement. But it is thankfully also an attack which has been perceived by everyone as an attack on the Norwegian society, and on a symbol of the wide recruitment to the participatory democracy which lies at our very national soul. I cannot thank the Norwegian people, and indeed the people in other nations who have offered their condolences, enough for their shows of support and shared grief. It really has been a tremendous help to me knowing that so many people feel with us. I also want to thank from the bottom of my heart the rock-steadiness of everyone in both the national and local wings of the Labour Party and Labour Youth in supporting us, and the political milieu in general for their resolute steadfastness saving me from losing yet more that I cherish; our freedoms in a participatory democracy. Our Party has lost many of its very brightest youngsters. Personally I feel an angry spite, a deep restless urge to get the wheels of society going again. I want to show his kind that we will not be broken. We re stronger than that. I will not be frightened into silence and passivity. I want to remember the dead, and honor them by carrying on our common work. I want to end this with a request to everyone who reads this, echoing a statement I read by one of my good friends and comrades: Please, don t let me see any messages of hatred, wishes for the death penalty, anything like that. If anyone should be of the belief that anything will improve by murdering this sad little person, they would be profoundly wrong. All attention now should be plowed into caring for those victims and their relatives who did not share my luck, and not giving an audience to a perpetrator who wants one. Tore Sinding Bekkedal

Joey Hess: arrival at DebConf

The trip down from Gratz to Banja Luka was much easier than the day before. After a while you just get used to being sat in a car for ages. Plenty of nice scenery to enjoy through Slovenia. After a while our car's GPS's began to fail, showing us driving through fields, and we were stuck for 1.5 hours in a traffic jam when the 4 lane highway seemed to end. Got around that with some guesswork, and on into Croatia by back roads. The Bosnian border was an interesting experience, all the guards could say in English was "green card! green card!" -- which from an American POV is an unsettling thing to be asked for at a border, especially if they've already taken your passport away -- but at least we were not detained overnight. While drivers were away getting the car insurance settled it descended toward farce as we had to hand roll the cars forward to let trucks get into the country. (Or we thought we did.. one was rolled with the keys in it as it turned out.) Arrived at the Hotel in Banja Luka in the middle of a wedding, which was amazingly loud (I could still hear it from the 5th floor at 2 am). There's also a casino at the hotel, so first impression was garish and loud! ... But now that it's a rainy Sunday, seems much nicer here.

18 July 2011

Benjamin Mako Hill: Quiet Room

At the Copenhagen airport, Mika and I found the quiet room. It was a soft, well lit, room designed for prayer and reflection. During the hour I was in it, the only other visitor was a child cracking open the doorway to peer in. The room had a guest book with hundreds of messages left by other travelers over the last couple years. People praised the airport administrators for providing the room, made suggestions, and complained about the room, the airport, and the country's shortcomings. They talked about themselves, their travels, their happiness and unhappiness with departing or returning home, and their thoughts about the world. I spend a lot of time in airports but only rarely speak to my fellow travelers. It's amazing how little I know about the thousands of people waiting in line with me, sitting near me on the plane, and sharing in the long, lonely, and often stressful experience of moving between countries and continents. The guest book provided a rare window into these people in what is normally the anonymous and depersonalized non-place of airports. In the quiet room, I could -- for the first time -- hear some of these fellow travelers speak.

Benjamin Mako Hill: Die Technikmafia

Marcus Rohwetter has recently published a very detailed article about Antifeatures in the German monthly magazine Zeit Wissen. Although I've only read the article through automatic translation -- unfortunately, I don't read German -- I'm hugely honored that Rohwetter has taken the time to engage with the idea so deeply and to help translate the argument for a much broader community than the free software community I come from and am best able to speak to. A lot of what I've been trying to do in the last year or so is to figure out how to speak more effectively about the politics of technology control to audiences of non-technologists. Indeed, that's the whole point of the antifeatures concept. I deeply appreciate the help of Rohwetter, and others, in that project.

14 July 2011

Jamie McClelland: GnuCash with python bindings

Hats off to the GnuCash developers and the Parit Worker Collective for the python bindings to GnuCash. If there is any doubt that free software makes life easier it's being able to write your own accounting import scripts while reading the original source code. I had to re-compile Debian's version of GnuCash because it doesn't have the python bindings enabled by default. As root:
apt-get build-dep gnucash
apt-get install python-dev
As a non privileged user:
apt-get source gnucash
cd gnucash-N.N.N # varies depending on your version)
Then, edit debian/rules and add --enable-python-bindings to the dhautoconfigure command. I replaced the line:
--enable-locale-specific-tax
With the lines:
--enable-locale-specific-tax \
--enable-python-bindings
Then, update the change log with:
dch
Built the new package:
dpkg-buildpackage -rfakeroot -uc -b
I then wrote a python module. It logs into the server running our members database, runs a script to export tab delimited lists, and then imports them into our gnucash xml file. It's far from perfect (I'm fairly new to python). Also - I have sheepishly inserted code that searches for certain values by accessing the gnucash file directly (via python's xml handling modules). I couldn't figure out how to do the search I needed with they available python methods, although it may very well be there and I just couldn't figure it out. If anyone is interested in using it for a template:
#!/usr/bin/python
import sys 
import os
import time
import csv
import warnings
from xml.dom.minidom import parse
from gnucash import Session,  GncNumeric
from gnucash.gnucash_business import Customer, Invoice, Entry, BillTerm
from decimal import Decimal
import datetime
import tempfile
import subprocess
def main(argv = None):
  if argv is None:
    argv = sys.argv
  path = os.environ['MFPL_GNUCASH_PATH']
  if not os.path.exists(path):
    print """Path (%s) does not exist. Please set 
    MFPL_GNUCASH_PATH environment variable\n""" % path
    return 1 
  if os.path.exists(path + '.LCK'):
    print """Lock file exists. Is GNUCash running?\n"""
    return 1 
  command = argv[1]
  import_file = None
  if 2 in argv:
    if argv[2] == '--from-file':
      if argv[3]:
        if os.path.exists(argv[3]):
          file_path = argv[3]
          import_file = open(file_path,'r')
        else:
          print "Can't find file to import: %s" % argv[3]
          return 1
      else:
        print "Please pass path to file if using --from-file"
        return 1
    else:
      print "I didn't understand the argument: %s" %s
  # download import file if not provided
  if import_file == None:
    import_file = get_data(command)
  # The file might be gzipped, so
  # open and save it 
  s = get_gnucash_session(path) 
  s.save()
  s.end()
  # Wait a second, so we can create a backup
  # without a name collision
  time.sleep(1)
  # Open for real now...
  s = get_gnucash_session(path) 
  xml_dom = parse(path) 
  if command == "members":
    ret = import_members_from_file(s, xml_dom, import_file)
  if command == "invoices":
    ret = import_invoices_from_file(s, xml_dom, import_file)
  s.save()
  s.end()
  return ret
def get_data(command):
  import_file = tempfile.NamedTemporaryFile()   
  if command == 'members':
    remote_cmd = 'acc-member-list'
  else:
    remote_cmd = 'acc-invoice-list'
  args = [ '/usr/bin/ssh', 'mayfirst@hay.mayfirst.org', '/home/mayfirst/bin/' + remote_cmd ]
  subprocess.call(args,stdout = import_file)
  return import_file
def import_invoices_from_file(s, xml_dom, csvfile):
  dialect = 'excel-tab'
  csvfile.seek(0)
  reader = csv.reader(csvfile, dialect)
  nodes = xml_dom.getElementsByTagName('gnc:GncCustomer')
  already_entered = 0
  entered = 0
  for row in reader:
    member_id = row[0]
    invoice_id = row[1]
    invoice_date = row[2]
    invoice_amount = row[3]
    invoice_desc = row[4]
    if not member_id.isdigit():
      # header line or not coded to a member?
      continue
    member_id = int(member_id)
    invoice_id = int(invoice_id)
    if invoice_exists(xml_dom, invoice_id):
      # print "Already entered %s (%s)" % (invoice_date,invoice_id)
      already_entered += 1
    else:
      if not member_exists(xml_dom, "No Name McGoo - fix me" , member_id):
        print "Skipping member id %s, invoice id %s - member doesn't exist" % (member_id, invoice_id)
      else:
        invoice_enter(s, member_id, invoice_id, invoice_date, invoice_amount, invoice_desc )
        entered += 1
  print "Invoices entered from last two months: %s (%s already existed)" % (entered, already_entered)
  return True
def get_guid_from_member_id(nodes,member_id):
  member_id = pad_number(member_id)
  for node in nodes:
    for child in node.childNodes:
      if child.nodeName == 'cust:guid':
        guid = child.childNodes[0].data
      if child.nodeName == 'cust:id':
        if member_id == child.childNodes[0].data:
          return guid   
def import_members_from_file(s, dom, csvfile):
  dialect = 'excel-tab'
  csvfile.seek(0)
  reader = csv.reader(csvfile, dialect)
  already_entered = 0
  entered = 0
  for row in reader:
    member = row[0]
    member_id = row[1]
    if not member_id.isdigit():
      # header line?
      continue
    member_id = int(member_id)
    if member_exists(dom, member, member_id):
      #print "Already entered %s (%s)" % (member, member_id)
      already_entered += 1
    else:
      member_enter(s, member, member_id )
      entered += 1
  print "Members entered from last two months: %s (%s already existed)" % (entered, already_entered)
  return True
def get_gnucash_session(path):
  with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    return Session(path, is_new = False)
def member_enter(s, member, member_id):
  print "Entering %s with id %s" % (member, member_id)
  book = s.book
  root = book.get_root_account()
  commod_table = book.get_table()
  USD = commod_table.lookup('CURRENCY', 'USD')
  member_id = pad_number(member_id)
  new_customer = Customer(book, member_id, USD, member)
def invoice_enter(s, member_id, invoice_id, invoice_date, invoice_amount, invoice_desc):
  book = s.book
  root = book.get_root_account()
  commod_table = book.get_table()
  USD = commod_table.lookup('CURRENCY', 'USD')
  invoice_id = pad_number(invoice_id)
  member_id = pad_number(member_id)
  invoice_date_object = datetime.datetime.strptime(invoice_date, "%Y-%m-%d %H:%M:%S") 
  member = book.CustomerLookupByID(member_id)
  assert( member != None )
  assert( isinstance(member, Customer) )
  assets = root.lookup_by_name("Assets")
  recievables = assets.lookup_by_name("Recievables")
  income = root.lookup_by_name("Membership Dues")
  invoice = Invoice(book, invoice_id, USD, member, invoice_date_object )
  invoice_value = gnc_numeric_from_decimal(Decimal(invoice_amount))
  invoice_entry = Entry(book, invoice)
  invoice_entry.SetDescription(invoice_desc)
  invoice_entry.SetQuantity( GncNumeric(1) )
  invoice_entry.SetInvAccount(income)
  invoice_entry.SetInvPrice(invoice_value)
  invoice_entry.SetDate(invoice_date_object)
  invoice_entry.SetDateEntered(invoice_date_object)
  print "Entering invoice %s %s %s" % (invoice_id, invoice_date, invoice_amount)
  invoice.PostToAccount(recievables, invoice_date_object, invoice_date_object, "", True)
def pad_number(num):
  return "%(number)06d" %   'number': num  
def member_exists(dom, member, member_id):
  member_id = pad_number(member_id)
  for node in dom.getElementsByTagName('gnc:GncCustomer'):
    for child in node.childNodes:
      if child.nodeName == 'cust:name':
        for grandchild in child.childNodes:
          if member == grandchild.data:
            return True
      if child.nodeName == 'cust:id':
        for grandchild in child.childNodes:
          if member_id == grandchild.data:
            return True
  return False
def invoice_exists(dom, invoice_id):
  invoice_id = pad_number(invoice_id)
  for node in dom.getElementsByTagName('gnc:GncInvoice'):
    for child in node.childNodes:
      if child.nodeName == 'invoice:id':
        for grandchild in child.childNodes:
          if invoice_id == grandchild.data:
            return True
  return False
def gnc_numeric_from_decimal(decimal_value):
    sign, digits, exponent = decimal_value.as_tuple()
    # convert decimal digits to a fractional numerator
    # equivlent to
    # numerator = int(''.join(digits))
    # but without the wated conversion to string and back,
    # this is probably the same algorithm int() uses
    numerator = 0
    TEN = int(Decimal(0).radix()) # this is always 10
    numerator_place_value = 1
    # add each digit to the final value multiplied by the place value
    # from least significant to most sigificant
    for i in xrange(len(digits)-1,-1,-1):
        numerator += digits[i] * numerator_place_value
        numerator_place_value *= TEN
    if decimal_value.is_signed():
        numerator = -numerator
    # if the exponent is negative, we use it to set the denominator
    if exponent < 0 :
        denominator = TEN ** (-exponent)
    # if the exponent isn't negative, we bump up the numerator
    # and set the denominator to 1
    else:
        numerator *= TEN ** exponent
        denominator = 1
    return GncNumeric(numerator, denominator)
if __name__ == "__main__":
  sys.exit(main())

10 July 2011

NeuroDebian: NeuroDebian@HBM2011.ca

NeuroDebian@HBM2011.ca On June 26-30 the annual meeting of the Organization for Human Brain Mapping (HBM2011) took place in Quebec City, Canada. Encouraged by our positive experience at last year s SfN in San Diego and enthusiasm of our scientific adviser, James V. Haxby, we hosted another NeuroDebian booth. The setup was pretty much the same as last year: Some chairs and tables, lots of people, our tri-fold flyers, a Debian mirror and some virtual machine images to show Debian in action. This time we also had an LCD display attracting visitors with the package swarm, some demos, and our recent paper. We had many curious people have their first exposure to Debian, long-time users expressing their gratitude to Debian, and our upstream developers getting together to discuss various topics. Having registered the booth as NeuroDebian , we had the additional pleasure of explaining visitors the concept of a project inside Debian, in contrast to a derived distribution. But that is nothing new really, so let s talk about the differences from last year s booth. First of all, we had more people at the booth. Dominique Belhachemi volunteered to help us out and that was very much appreciated. Although HBM has only about a tenth of the attendees that SfN has, we had significantly more traffic. While last year people were primarily interested in knowing about the project, this time many of them wanted to give it a try immediately. People came with their laptops, got the VM images and started playing with Debian. After a day or so, some came back and asked for recommendations on particular software after having been exposed to the wealth of the Debian archive. What also had increased was the number of developers, or rather research labs developing neuroimaging software that came to the booth to discuss how to get their software into Debian and how to arrange ongoing maintenance of these future Debian packages. As we have our plates already quite full, we have been spending some time on mentoring interested developers to learn the art of Debian packaging and making them familiar with Debian s procedures and standards (e.g. working on #609820 with Yannick Schwartz, upstream, at the booth). ../../_images/BusyBooth216.jpg Two promising new developments need to be mentioned. First, we were approached by companies that develop hardware for brain-imaging and psychophysics research. They were curious to learn about Debian as an integrated platform that offers free software solutions that an increasing amount of their customers demands (e.g. PsychoPy). Apparently, the movement towards open research software has finally made it into the business plans of companies, as they seem to start perceiving compatibility with free software systems as a competitive advantage. We explained how software gets into Debian, and how its release cycle is managed. To foster their motivation we also pointed them to the existing open-source software that is already available or even present in Debian. Let s see whether we see more Debian-certified research products in the future. Lastly, we started talking with folks from the INCF to explore possibilities of collaborating on INCF projects using Debian as the integration and development platform. The INCF is an OECD-funded organization that develops collaborative neuroinformatics infrastructure and promotes the sharing of data and computing resources to the international research community. At least one INCF project is already relying on the efforts of the NeuroDebian project. We are going to continue this discussion during a workshop in September. A report will follow...
../../_images/DDs13.jpg

Debian people at the booth (f.l.t.r): Michael Hanke, Yaroslav Halchenko, Stephan Gerhard, Dominique Belhachemi. Not shown: Swaroop Guntupalli.

Acknowledgments This booth has been made possible by the generous support of Prof. James V. Haxby (Dartmouth College, New Hampshire, USA).

James Morrison: Code reviews

I'm reading through http://scientopia.org/blogs/goodmath/2011/07/06/things-everyone-should-do-code-review/. So, I'll make my comments here since I expect they will be long. As a caveat, I still work at Google and have done a lot of code reviews. I've made a lot of mistakes while code reviewing, I've had difficult code reviews, but my biggest problems were non-code reviews. The caveat is for most of the code reviews, I know the code being changed better than the person changing the code.

I'll reply to sections as I come across them: "Given a problem, there are usually a dozen different ways to solve it. Andgiven a solution, there's a million ways to render it as code. As a reviewer, your job isn't to make sure that the code is what you would have written - because it won't be.". I disagree with this. There are many times where code is first written in a (1) Ignorant way (2) Sloppy way (to try to save trivial amounts of work). I agree that the code probably won't be the way the reviewer wrote it, but it is good to comment on how it could have been written. Many times the reviewee isn't familiar (or ignores) with the idiomatic style of a larger bit of code.

"The second major pitfall of review is that people feel obligated to say something.". It's fine to say something, but if that's what you say most of the time then you are wasting the reviewee's time. If you do a first pass and don't find anything, you should go back and read the code and be sure that you understand it. Most times I've seen people say nothing is that they don't want to know what the new (or changed) code does.

The last section is about speed. For speed there are two types of code reviews, one which is quick, small and fixes something important. Those should only take one round and typically only need the reviewer to ask for tests. Then there is everything else, which includes small and unimportant. For those, it's ok to take more time to start the review, but each iteration should get smaller and faster.

For anyone that's gotten this far, I guess I can say my comments about Mark's blog post are purely bike shedding, but that's something with code reviews, it's ok to say you think the bike shed should be neon pink as long as you don't force the reviewee to paint the bike shed neon pink.

3 July 2011

Benjamin Drung: libkibi ready for testing

Yes, the library is finally called libkibi. The second poll ended with a tie between libbyteprefix (48 votes) and libkibi (45 votes). Then I decided to call the library libkibi. libkibi is hosted on Launchpad. A working version can be pulled from lp:libkibi with bzr. Please test the library, reports bugs, and ask questions. Why you should use libkibi instead of g_format_size_for_display (from GLib):

Benjamin Drung: Nautilus with libkibi

In this Ubuntu cycle, I work on getting the units policy implemented. For this I wrote a library called libkibi. Here are some screen shots how nautilus looks like with libkibi. Some changes are highlighted in red. The file properties will show the file size in base10 and base2: PS: I failed to launch nautilus in English. Therefore the screen shots are in German. PS : You can grab the modified nautilus package for Ubuntu 10.10 (maverick) from my experimental PPA (at your own risk!).

Benjamin Drung: libkibi 0.1 released

The first version of libkibi is released. This library is designed for formatting sizes in bytes for display. The user can configure a preferred prefix style. Packages for Debian unstable and Ubuntu 11.04 (maverick) are uploaded. It contains a README for developers and a byteprefix man page for users, which can be read with man 5 byteprefix once installed. Thanks to Stefano Rivera for writing the man page! For a demonstration how this library used by an application can look like, read my previous post.

Benjamin Drung: How to purge Wine completely

I had Wine installed and wanted to get rid of it completely. I removed the Wine Debian package and what belong to it, but the Windows applications still appeared in the GNOME main menu. I removed the .wine directory and all local wine-related desktop files: rm -rf ~/.wine ~/.local/share/applications/wine
rm -f ~/.local/share/applications/wine-* ~/.local/share/applications/mimeinfo.cache
Now there is no sign left that Wine was installed.

Next.

Previous.