Search Results: "evo"

27 April 2012

Pietro Abate: Learning from the Future of Component Repositories - CBSE 2012

Learning from the Future of Component Repositories ( Pietro Abate, Roberto Di Cosmo, Ralf Treinen and Stefano Zacchiroli ) has been accepted to be presented at CBSE 2012 (26-28 June, Bertinoro, Italy)

Abstract
  An important aspect of the quality assurance of large component repositories
  is the logical coherence of component metadata. We argue that it is possible
  to identify certain classes of such problems by checking relevant properties
  of the possible future repositories into which the current
  repository may evolve. In order to make a complete analysis of all possible
  futures effective however, one needs a way to construct a finite set of
  representatives of this infinite set of potential futures. We define a class
  of properties for which this can be done.
  We illustrate the practical usefulness of the approach with two quality
  assurance applications: (i) establishing the amount of  forced upgrades''
  induced by introducing new versions of existing components in a repository,
  and (ii) identifying outdated components that need to be upgraded in order to
  ever be installable in the future. For both applications we provide
  experience reports obtained on the Debian distribution.
The tools presented in this paper (outdated and challenges) are already in Debian as part of the 'dose-extra' package.

26 April 2012

Wouter Verhelst: Switching to duckduckgo

In the late 90s, google became popular for one reason: because they had a no-nonsense frontpage that loaded quickly and didn't try to play with your mind. Well, at least that was my motivation for switching. The fact that they were using a revolutionary new search algorithm which changed the way you search the web had nothing to do with it, but was a nice extra. Over the years, that small hand-written frontpage has morphed into something else. A behind-the-scenes look at the page shows that it's no longer the hand-written simple form of old, but something horrible that went through a minifier (read: obfuscator). Even so, a quick check against the Internet Wayback machine shows that the size of that page has increased twenty-fold, which is a lot. But I could live with that, since at least it looked superficially similar. Recently, however, they've changed their frontpage so that search-as-you-type is enabled by default. Switching that off requires you to log in. So, you have a choice between giving up your privacy by logging in before you enter a search term, or by having everything you type, including any typos and stuff you may not have confirmed yet, be sent over to a data center god knows where. Additionally, at the first character you type, the front page switches away to the results page, causing me to go "uh?!?" as I try to find where they moved my cursor to. This is annoying. Duckduckgo doesn't do these things; and since they also don't do things like combining my typing skills, phone contact list, calendar, and chat history to figure out that I might be interested in a date, I'm a lot more comfortable using them. So a few days ago, I decided to switch my default search engine in chromium to duckduckgo. It still feels a bit weird, to be using a browser written by one search engine to search something on another; but all in all, it's been a positive experience. And the fact that wikipedia results are shown first, followed by (maybe) one ad, followed by other search results, is refreshing. We'll see how far this gets us.

22 April 2012

Joey Hess: moving my email archives and packages to git-annex

I've recently been moving some important data into git-annex, and finding it simplifies things while also increasing my flexability. email archives I've kept my email archives in git for years. This works ok, just choose the right file format (compressed mbox) and number of files (one archive per mailbox per month or so) and git can handle this well enough, as email is not really large. But, email is not really small either. Keeping my email repository checked out on my netbook consumes 2 gigabytes of its 30 gigabyte SSD, half of which is duplication in .git. Also, I have only kept it at 2 gigabytes through careful selection of what classes of mail I archive. That made sense when archival disk was more expensive, but what makes sense these days is to archive everything. For a while I've wanted to have a "raw" archive, that includes all email I receive. (Even spam.) This protects against various disasters in mail filtering or reading. Setting that up was my impetus for switching my mail archives to git-annex today. The new system I've settled on is to first copy all incoming mail into a "raw" maildir folder. Then mailfilter sorts it into the folders I sync (with offlineimap) and read. Each day, the "raw" folder is moved into a mbox archive, and that's added to the git annex. Each month, the mail I've read is moved into a monthly archives, and added to the git annex. A simple script does the work. I counted the number of copies that existed of my mail when it was stored in git, and found 7 copies spread amoung just 3 drives. I decided to slim that back, and configured git-annex to require only 5 copies. But those 5 copies will spread amoung more drives, including several offline archival drives, so it will be more robust overall. My netbook will have an incomplete checkout of my mail, omitting the "raw" archive. If I need to peek inside a spam folder for a lost mail, I can quickly pull it down; if I need to free up space I can quickly drop older archives. This is the flexability that git-annex fans love. :) By the way, this also makes it easier to permanantly delete mail, when you really need to (ie, for contractual reasons). Before, I'd have to do a painful git-filter-branch if I needed to get rid of eg, mail for old jobs. Now I can git annex drop --force. Pro Tip: If you're doing this kind of migration to git-annex, you can save bandwidth by not re-transferring files to machines that already have a copy. I ran this command on my netbook to inject the archives it had in the old repository into the new repository, verifying checksums as it goes:
cd ~/mail/archive; find -type l -exec git annex reinject ~/mail.old/archive/    \;
Note on mairix compatability: I use mairix to index and search my mail. But it refuses to follow git-annex's symlinks to the content. So I have to point it at .git/annex/objects/. I also configured annex.backend to SHA256E, which keeps the extensions on my compressed mailbox files, which is necessary for mairix to realize they're compressed. debian packages I'd evolved a complex and fragile chain of personal apt repositories to hold Debian packages I've released. I recently got rid of the mess, which looked like this: dput local mini-dinstall repo dput→mini-dinstallrepo on my server →dput Debian The point of all that was that I could "upload" a package locally while offline and batch transfer it later. And I had a local and a public apt repository of just the packages I've uploaded. But these days, packages uploaded to Debian are available nearly immediately, so there's not much reason to do that. My old system also had a problem: It only kept the most recent single copy of each package. Again, disk is cheap, so I'd rather have archives of everything I have uploaded. Again I switched to git-annex. My new system is simplicity itself. I release a package by checking it into a "toupload" directory in my git annex repository on my netbook. Items in that directory are dput to Debian and moved to "released". I have various other clones of that repository, which I git annex move packages to periodically to free up SSD space. In the rare cases when I build a package on a server, I check it into the clone on the server, and again rely on git-annex to copy it around. Now, does anyone know a good way to download a copy of every package you're ever released from archive.debian.org? (Ideally as a list of urls I can feed to git annex addurl.) conclusion My email and Debian packages were the last large files I was not storing in git-annex. Even backups of my backups end up checked into git-annex and archived away. Now that I'm using git-annex in every place I can, my goal with it is to make it as easy as possible for as many of you to use it as possible, too. I have some inotify tricks up my sleeve that seem promising. Kickstarter may be involved. Watch this space!

18 April 2012

Richard Hartmann: Potpourri I

Google rant Seems venting can be a worthwhile once in a while; the content of this post made its way into Google. No promises of any kind were made, but it's good to know that this is getting some exposure within Google :) That was quick. moreutils et al Steve Kemp is right; moreutils is very useful in some situations. And yes, it's a pity that we all have our own scripts without a central repository to host them all. The problem is one of diminishing returns and information overload; if any given collection becomes too large, it becomes tedious to look through it. You may have the Most Awesome Ever solution to any given problem, but unless whoever needs it can find it quickly, it's useless to them. Thorsten Glaser suggested this collection, but it suffers from the same basic problem: Not enough content? You won't find what you need. Too much content? You won't find what you need. Finding the right balance is highly non-trivial. Linux storage Russell Coker shared yet another chapter of his continuing quest for modern, reliable, software-based storage on Linux. The short version is "ZFS is not that good on Linux while btrfs is not ready for prime time, yet." That's, unfortunately, not much of a surprise. fsck.btrfs is still too new and the lack of RAID 5/6 is an absolute show-stopper as far as I am concerned. COW solves the writing speed issues so, hopefully, that old argument can finally be laid to rest. And on top of the "wastng" of more disks for RAID 10, the minimum number of failed disks that can lead to data loss on RAID 10 is two whereas it's three for RAID 6. No-brainer, imo. Anyone who does not agree with Dell's pricing structure should seriously consider looking at SuperMicro, by the way. Cheap(ish), reliable, high disk densities. Password hashes Saku Ytti talks about password hashing and the relative merits of different algorithms. While I agree that bcrypt incorporates a good approach inasmuch it's designed to be slow to compute and can be made slower as needed, I think he focuses on one aspect while disregarding all others. Still, the premise that deterring brute force attacks on known hashes is the only concern is wrong. Avoiding collisions, reasonable certainty that there are no computational shortcuts, irreversibility, pseudo-random output, full use of the output's possible values, and thorough analysis by experts in the field are important concerns, as well. Brute forcing known hashes is hardly the only attack. When brute forcing a service, i.e. against an unknown hash, no matter if the system is remote or local, the computational cost of hashing is mostly irrelevant. Rainbow tables help mitigate computational cost when trying to extract clear text in case the hash is known. And if there are a lot of collisions or an easy way to find them, knowing the clear text password is not needed. Simply use whatever generates the correct output and you're done. Again, I am not saying that bcrypt is a bad idea; it's just that there are more concerns than computational cost alone. Travel If you know your way around Novosibirsk, please contact me. We have almost no idea what to do while there other than visiting the railroad museum; part of the reason why we are stopping there is to have a break after ~50 hours on the train and because Novosibirsk is just an awesome name. Our focus while there will be on seeing interesting industrial or research sites. Given that there are large industrial complexes and formerly secret cold war secret research cities situated around Novosibirsk it would be a pity not to see any of them. We may end up hiring a taxi via the hotel and having that drive us around a bit. Somewhat boring and potentially costly, but a feasible backup. Blogging frequency I have to admit being am a bit worried about spamming the various aggregators which are fed from this blog. Writing this catchall post is an attempt to mitigate this. While I am getting a surprising amount of positive feedback, I would be interested to hear anything negative, if applicable, as well. You know how to reach me.

11 April 2012

Guido G nther: Bits from the 5th Debian Groupware Meeting

This went out to d-d-a already but I figured that this might be of interest here too: The fifth Debian Groupware Meeting was held in the LinuxHotel, Essen, Germany. Eight persons attended which is an all time high! This is a short summary of what happened during the weekend:

5 April 2012

MJ Ray: Debian Project Leader Election 2012

Voting is open in the Debian Project Leader Elections 2012 So now I need to figure out who to vote for. This year I didn t take part in the discussions (all my spare time was bought, basically). The platforms are linked from the Debian Project Leader Elections 2012 page above and the key discussions were: Thanks to everyone who asked these great questions. So, what do you think?

Jon Dowland: Greg Bear Hull Zero Three

In (roughly) 2008 I read "Blood Music" my first Greg Bear novel and devoured it in a weekend, in only two or three reading sessions. That happens very rarely, and when it does I often wonder whether it's the quality of the writing or some combination of that and your personal moods and tastes at the time that is responsible. My next Bear was "Darwin's Radio" in mid-2010 which, whilst good, did not provoke the same feverish read. I might write short reviews for both at some point in the future, but right now I can't remember either well enough to do so. In the last few weeks I read "Hull Zero Three" which was quite a different beast. The novel first captured my attention due to the unusual way it was promoted: via a video, which was created using a modified Quake 2 engine. It wasn't a great video but the novelty made it interesting. The story begins with a first-person narrator waking up from a sort-of hyper sleep on a generation starship that has clearly had some serious problems. He has to run to escape danger before he has an opportunity to figure out exactly what has happened and what he is running from, or to. Something has gone horribly wrong on the ship: it's nowhere near where it should be going and nobody should be awake. The narrator's memory is fractured, fabricated and unreliable; the ship is hostile, and whilst it's mechanical, it cyclically changes and re-arranges it's hulls in an almost organic way. He is joined by a rotating cast of similarly confused refugees of varying shapes and sizes. Language and cultural barriers serve to further magnify the sense of alienation that you experience on behalf of the story teller. The troupe are persued through the corridors by a collection of lethally efficient killer "things", contrasting with the background by being organic in nature but mechanical in behaviour and appearance. The scale of the ship is necessarily massive, and Bear does an impressive job of capturing the awe you might feel when considering it. Three huge, cigar-shaped hulls, joined by "spars" to an enormous chunk of ice captured from the Oort cloud on exodus from the solar system. The ice serves as a mixture of "fuel" for the biological ship systems and as part of a shield system for the hulls from cosmic rays. However, in part due to the narrator's confusion, it was hard to follow the very abstract descriptions of the environment in the earlier stages of the story. I found myself re-reading passages to try and understand the comparisons being offered and scales presented. In common with the two earlier Bears I've read, 'body horror' features prominently in this story. It feels like a "Dead Space" or "Resident Evil" in space in a hard SF novel which seriously attempt to capture real speculative science. Like "Blood Music", I found myself compelled to read "Hull Zero Three" at every available opportunity. I finished it off in a very enjoyable reading session outdoors in the spring sunshine. It was an enjoyable read, well written but not beyond criticism, with some interesting takes on well-worn themes. Very reminiscent of the movie "Pandorum" (which was a surprisingly good movie).

4 April 2012

James Bromberger: Goodbye Linux.2.6.x

It s taken some time, but now none of my personal Linux hosts (4 in total) are running the 2.6 kernel any more. From the start (January) my company web host on Amazon EC2 has been running a 3.x kernel. My little Acer Aspire Revo low power home server, with attached disk pack that sits in my shed in a network cabinet has run 3.x for the last 6 months or so. My Linux laptop (Dell Studio 1558) which only recently got installed (and, since removing Windows, hasn t overheated once!) went to 3.x immediately. And the last piece of the puzzel is a virtual machine I ve had for many years with Bytemark.co.uk they re now offering a 3.2 kernel in their menu of selectable kernels. Not that 3.x is that much different than 2.6.3x; but its a line in the sand of feature and security thats easy to identify. But with nearly 15 years of looking at a 2.x kernel, its about time we moved to 3.x!

1 April 2012

Jamie McClelland: Property is Theft

I've recently been making my way through the Proudhon Reader, a collection of writings by Pierre-Joseph Proudhon recently published by AK Press. Proudhon, known as the father of Anarchism is most famous for his declaration: "Property is theft." The simple declaration is based on pages of explanation which, simplistically, can be boiled down to the idea that workers should all equally share in the fruits of their labor. While Proudhon's influence on Marx is hotly debated, this idea is fundamental to Marx's idea around the alienation of labor. If we are working to produce a product that is then owned by the proprietor, rather than the worker, we are alienated from our labor. Many May First/People Link members, such as Palante Tech, Openflows, Union Web Services, and Agaric Design are all committed to fighting this alienation by organizing as worker-run cooperatives. Nobody is the boss, all workers share in all profits. However, Proudhon got me thinking about free software shops that aren't worker-run cooperatives. If you are employed by a for-profit corporation as a worker and your only job is to write code that is released under a free software license, are you alienated from your labor in the Marxist sense? The fruits of your labor might be owned by your company, but they are freely licensed to the world (which, of course, includes you). There is still surplus value accrued to your company (if you do good work for your company, you will be contributing to their reputation from which they will profit). However, if you are primarily writing code, the vast majority of your labor is producing a product that you are fundamentally not alienated from. Proudhon, while sprinkling the word revolution throughout his writings, was notably not particularly revolutionary in his calls for action. He was reformist. Rather than calling for a direct confrontation with capitalism, he called for worker-run collectives to be formed throughout the world to make capitalism irrelevant. While many people point to corporate use of free software as an argument for why free software and capitalism are perfectly compatible, the truth may be more complicated. Fundamentally, and in a non-confrontational way, free software seems to undermine one of the basic tenants of capitalist: worker alienation. Now we just need to work on all the other ways corporate work places are alienating...

Gregor Herrmann: RC bugs 2012/13

due to some new incoming RC bugs, this week was more devoted to fixing bugs in "our" (= the Debian Perl Group's) packages. here's the list:

25 March 2012

Y Giridhar Appaji Nag: Moving to 4096 bit RSA GPG key AA544AA1

This is an annoucement that I am moving to my 4096 bit RSA GPG key AA544AA1, henceforth please try and use this.

I created and have (sparingly) been using this key ever since SHA1 was found to be weak, and larger RSA keys and stronger signatures began to be the norm. There weren't many signatures on the key and the WOT was not strong but it looks good now. The older key in the Debian keyring (1D389887) has also been replaced with the new key and I will revoke it when the number of signatures on the new key crosses those on the old key (that might be in an year or two, I suppose).

Fingerprint information:

Current key:

pub 1024D/1D389887 2006-04-08
Key fingerprint = 5B6B F223 B769 63E9 E2CB 0A3F E1EB BEA5 1D38 9887
uid Y Giridhar Appaji Nag <giridhar@appaji.net>
sub 2048g/73EAF0B5 2006-04-08
sub 2048R/DBBECADC 2009-04-04


New 4096R key:

pub 4096R/AA544AA1 2009-05-22
Key fingerprint = BF81 3A99 E07C D0F8 653D 9666 A8D2 CD13 AA54 4AA1
uid Y Giridhar Appaji Nag <giridhar@appaji.net>
sub 4096R/843A5776 2009-08-08

22 March 2012

Richard Hartmann: Svalbard

Svalbard The things you find when cleaning out a disk; preparing for re-installation of your laptop on a larger disk once the laptop comes back from repair... I thought I had posted this in early January, but apparently not. As it would be a shame to just throw this away, here goes: I am sitting in the Oslo airport, waiting for boarding back home to start. Seeing the sun after a week of darkness still feels strange. Inital trouble It's been a very interesting week, starting with our trip to Oslo in the other direction. We spent New Year's Eve in Oslo, timing our forced overnight stay before reaching Svalbard to coincide with something interesting. The close timing of our travel meant that there was exactly one flight to Oslo we could take. Never having heard of Air Baltic before and finding out that they are a discount airline, my gut feeling told me to be be wary. I refuse to fly with those airlines on principle, not wanting to support their business model while hurting airlines with decent treatment of customers. Unfortunately, I was forced to book with them in this case. As it turns out, my gut was right. More on that below. Our plane was late in landing and my luggage was lost. As it was New Year's Eve, the staff at Oslo airport understandably wanted to be home, not at work. Still, getting them to file a report was tedious and finding out days later that the report was incomplete was, well, not good. The express train from the airport to Oslo central station had closed early without any advance notice or local signs, presumably because of NYE. The gates were simply closed and that's that. We figured out how the bus system worked, got our tickets from a vending machine, saw the one bus to central station drive away, and proceeded to wait in the outside waiting area; at least we had a front spot in the queue. While we don't get cold easily, it was funny to see Norwegians in thin clothes stand around in the biting wind, apparently being comfortable. A young mother with a baby, who didn't anticipate being forced to wait outside at -15C, as opposed to just sitting down in a train, had no warm clothes for her son; something that was fixed by wrapping him in spare clothes from Ilona's luggage. After waiting for about thirty minutes for the next bus to appear, it parked twenty or thirty meters away from the designated parking spot. The rough queue disintegrated and if not for Ilona's leaving me with luggage and backpacks and storming off to fight the masses, said mother with baby and ourselves would have waited for another thirty minutes. More than hundred people were left stranded at the airport to celebrate there or on the bus. Meh. We hurried, by taxi, from central station to hotel to harbour and arrived about five minutes to twelve. NYE itself was nice. We stood on top of the opera house and watched a rather impressive show of fireworks through the thickening mists. Norwegian fireworks pack lot more punch than German ones; you actually feel your clothes shake when they go off. Getting to Longyearbyen The flight from Troms to Longyearbyen had free in-flight Wi-Fi and flying over the edge of satellite coverage demonstrated how far we were from everywhere else rather impressively. Arriving at Longyearbyen, I fixed the lost luggage claim with the help of an extremely nice woman working at the airport. She confirmed that Air Baltic is legally required to reimburse me after an arcane system based on a virtual IMF currency to the tune of about 1.500. That may sound like a lot, but seeing as I had most of my scuba gear with me, that's not even half of what my luggage was worth. I was forced to get by New Years with what I had on my body and went shopping the next day when I could buy at least a few things. I got by with spending about 180 by going for non-fitting bargain bin clothes, wanting to reduce impact on Air Baltic. Shortly after that, my luggage arrived, unannounced. Air Baltic refused to reimburse me even though they are legally required to, again the airport staff confirmed this. But unless I sue in the country of destination, Norway, I won't see any money. Long story short: Avoid Air Baltic if possible. They will break the law to cheat you out of money when they have a reasonable expectation of getting away with it. Update: Yep, seems they got away with it unless I take legal measures. You have been warned. Longyearbyen itself was very nice. We started off with a short, guided taxi tour around the city, seeing literally everything of it as there's not a lot of Longyearbyen to start with. Dogsledding Next day, as noted earlier, we went shopping and spent the "evening" with dogsledding which turned out to be tons of fun. We helped with harnessing the dogs which is somewhat cumbersome as the dogs are so eager to burn off their energy that they want to run all over the place, not being put into a harness and snapped onto the pull-line. Never having been dogsledding before I was a bit wary, but riding over not too rough terrain is almost trivial once you get the hang of it; listen to what the musher in front of you yells and emulate the same shouts with your own dogs. If the dogs become too fast, step onto the brake pad which simply drives spikes into the snow. If the dogs slow down while going uphill, skate with one foot to help them. That's it for speed. As the dogs are following the musher's sled anyway, steering the dogs was not a concern. Fun fact: The musher used a laser pointer to steer his dogs; simple, efficient, and presumably fun. We learned, by demonstration, that sledding dogs can eat snow, take a leak, and take a dump while running at full speed and pretty much all at the same time. The dogs are left either in cages or on long chains far away from the city as they tend to bark and howl a lot. We were surprised to learn that, even when there are seals left to hang dry as dog food nearby, there are no problems with bear attacks. Apparently even the extremely aggressive and hungry one year old males will not go near dogs. Still, while out in the ice and snow, our guide carefully flooded all crevices and cliff bases to root out female bears with young ones early. They hide their children from the wind and they will attack, dogs and all, if we get too close. That's why our musher carried rifle in his sled and flare gun on his body. Snowmobile The next "morning", we drove around by snowmobile. This turned out to be extremely boring as it was a curated trip with over half a dozen snowmobiles in our group; a stark contrast to our two-sled tour the day before. The last time I went snowmobiling, we raced each other up and down a two-star black (i.e. steep, bumpy, and curvy) ski slope, jumping several meters when racing over larger bumps and crossing streets, so riding single file at 30 km/h was... anticlimatic. Again, the guide had rifle and flare gun with him. We spent the afternoon and evening walking around the city. Ice bears, part I As the ice caves and the glacier were still closed, we decided to have another walk around the city. Having planned make it a quick tour, we lost track of time due to lack of sun and ended up walking around for seven hours. A note about that trip.. If you are alone with your wife, unarmed, climbing up a very steep and slippery mountainside over a sheet of ice with deep snow underneath and loose rocks in between, and then start shining around with your flashlight under the stilts of an abandoned mine that looks like in a horror movie, the correct answer to "What are you doing?" is never ever "Looking for ice bears". Even if it's the truth, this is not an acceptable answer. I crawled up the last part on all fours, camera and tripod in hand while Ilona stayed about two dozen meters below the mines' entrance, refusing to go another step towards the mine. As soon I made it up the creaky and shaky, for a lack of a better word, let's call it ladder, she forced me to come down again. Bleh, but I guess I deserved that. Armed photographer Next day, we got final confirmation that we would not be allowed to rent snowmobiles to explore the hinterland on our own and that other for some, and I quote, "crazy Russians", no one would even attempt to cross over to Barentsburg. Thus, we ended up renting a car for the ~20 kilometers of total road length. That turned out to be a great idea as it allowed us to get away from the light and take some very nice long exposures. It was then that I got a rental rifle, as well. There is a law against leaving the town unarmed and I was not about to test my luck too much. Turns out that, as part of Germany's WWII reparations, Norway received our all hand guns and as they still function perfectly when dirty and in cold climate which makes them still popular in Svalbard, today. The Karabiner 98 Kurz which I received is built incredibly well. It's somewhat scary inasmuch every detail is designed to make this weapon ready to fire. If you hurry or are inexperienced, you will end up with an unlocked and loaded rifle after putting in the bolt. Putting the safety in and not chambering a bullet takes conscious effort and knowledge of the weapon. This is in stark contrast to other weapons I had the chance to dissect, which all defaulted to being safe. Even other military weapons such as the AK74 and the M4A1 are inherently more secure, designed to be locked and safe. The Mauser K98... not so much. As an aside, they didn't bother to remove the Reichsadler and Swastika from the rifle, contending themselves with striking out the German registration number and stamping the rifle with a Norwegian one. I guess Norwegians don't really "care" about these signs as much as we Germans do. In a way, that's a good thing I guess, at least as long as it's an indication of indifference towards the sign, not one of forgetting or ignoring the underlying issues. Still, I was very glad to have rented the rifle. While Ilona tended to stay in the running car with heat and lights on, I went out and away from the car. Even when standing near a street, a medium snow storm will make you appreciate the four powerful arguments against being eaten by a random bear which are at the ready over your shoulder. We even went to the shooting range so I could get some practice. The procedure is very trusting, as is anything in Longyearbyen. After accessing the interior of one of the houses in a particular way which I won't specify here, you simply switch on the floodlights, put up the red flag, position a target and write your name into the guest book. Once you're done shooting, toss a few coins into the bowl next to the guest book, remove the targets and flag, turn off the lights, close the door and that's that. Unfortunately, the way in was under a few meters of snow so I couldn't get in any practice shots. Ice bears, part II Later, as I was standing on a wind-polished slate of ice taking pictures of the Seed Vault (located here), I heard an ice bear roar behind (i.e. to the north of) me. I consciously remember hearing the bear, I consciously remember facing the other way around, half-crouched, rifle raised in the direction of the roar. I also consciously remember smacking the safety off and chambering a bullet after having regained control of my body. I do not remember spinning around on a wind-polished slate of ice, so treacherous I hand to balance with my arms and didn't even lift my feet when walking over it, without losing balance or footing, nor do I remember crouching and raising the gun. Evolution really is amazing; no matter what primal chord that roar struck, it certainly saved a ton of people over the years. In my case, thankfully, there was no bear to be seen down the slope. There may be no sun or anything, but the snow reflects the starlight so you tend to see surprisingly far and as I was on on the mountainside and the roar came from down from the coast, and as I had my rifle, I decided to finish the photo session while keeping the slope in close view. In hindsight, I am still glad I decided to do that as the shots came out rather nice. Next day, we drove out to Mine 12, the farthest you can away from Longyearbyen. The dump truck transporting coal alternated between driving a full load of coal back to the city and being its own snowplow. One quick trip to get coal, one slow and empty trip to plow away snow, rinse, repeat. If not for that, our 4WD would never have made it all the way up to the mine. Neither snow storms in North America nor around the Alps prepared me for what people on Longyearbyen consider normal wind in their backyard. This is where the word wind-swept was invented. The main reason that Svalbard is inhabited at all, other than the Gulf Stream, is coal. We have been told that the coal up there is of extremely high quality and while I can't say much about that, I can say that it's hard as stone. This is nice as it does not smear when you get coal all over yourself. Just shake out your jacket and pants and you are good as new. On our way back, we met two locals who had just prepared the glacier and ice caves for tourists. Had I known that in advance, I would have tried to go with them to take pictures completely away from all artificial light. Oh well, can't have everything. Speaking of not being able to have everything, the outdoor hot tub in our hotel which integrated ice bar and BBQ grill was still under several meters of snow so we couldn't use that, either. Finally, the few divers who are in Longyearbyen didn't have time to take me onto a trip while I was there. As I already missed my opportunity to dive the Arctic circle when the one diver on Gr msey happened to be on the mainland and barely missing it by diving Str tan instead, this was kind of a bummer. On the plus side, this has given me a goal to pursue and achieve. Random notes For the rest of my notes, I will resort to a largely unsorted list of bullet points as there's just too much to talk about in prose. All in all, it was well worth it. PS: If you know anyone working with Google Maps, ask them to consider improving their coverage of the Arctic. This is a real pity. As is the rest of the Arctic and the fact that Google Earth cheats you out of the North and South Pole by stretching adjacent tiles into and over them.

20 March 2012

Andreas Barth: Traveling to DebConf 11 - in Sarajevo

Sarajevo gave a rainy look the next morning. That wasn't too bad, as I was able to walk around a bit without it being too crowded. Incidently I also saw the corner where Franz Ferdinand was shot in 1914, which started the first world war - Europe has many links beyond todays state borders, in good and bad times. It's not like transeuropean politics (or communications) are something too new.
Bazar in Sarajevo
Franz Ferdinand was shot here in 1914
Trolley buses
River, tram and city
nice building hidden by cars - should our cities be dominated that way?
more of the old city After having spent the morning exploring the old part of Sarajevo and along the river, I used the time after hotel checkout to use the tram link and take a few more pictures. I arrived at the train station as planned, but the train I wanted to take didn't go that day (in spite of checking it within the hotel in the morning). So, I ended with another bus trip (this time unplanned) to Banja Luca. Again, bus trips are worse than train trips - one cannot move in the vehicle, and there are delays for every stop. The events in Banja Luca are discussed elsewhere, so nothing about that here (except that DebConf was great - I enjoyed it very much).

Neil McGovern: LibreOffice and UK Government

One of the things that Collabora does is work with other open source projects to improve things for open source software in general. I was chatting with Michael Meeks of SUSE fame and we had a chat about my recent decision to buy Microsoft Office 2010 for the city council. One of the overriding problems with the comparison was the lack of support for macros within OpenOffice and I was glad to hear about improvements being made in that area. However, there was still an issue. Doing improvements like this cost lots of time, and therefore for a commercial open source company, money. A question arises as to who can pay for this. A district council has limited funds, and can't devote it to (essentially what equates to) a large search and development software engineering task. Companies themselves won't want to take the risk that comes with the large development without a buyer, as once the work is done, it is open source so is harder to monetize. In the mean time, millions of pounds of public money is being siphoned into a private company, and vendor lock in continues. An easy out for a councillor can be to say "Central Government should pay" but given the current state of the economy, I can't see is happening in the immediate future. However, there is a consultation on Open Standards that's currently accepting submission until May, and if that insisted on ensuring that open standards are adopted throughout the UK, it may help unlock this chicken and egg problem.

Dirk Eddelbuettel: R / Finance 2012 Open for Registration

The annoucement below just went to the R-SIG-Finance list. More information is as usual the the R / Finance page:
Now open for registrations: R / Finance 2012: Applied Finance with R
May 11 and 12, 2012
Chicago, IL, USA
The registration for R/Finance 2012 -- which will take place May 11 and 12 in Chicago -- is NOW OPEN! Building on the success of the three previous conferences in 2009, 2010, and 2011, we expect more than 250 attendees from around the world. R users from industry, academia, and government will join 40+ presenters covering all areas of finance with R. This year's conference will start earlier in the day on Friday, to accommodate the tremendous line up of speakers for 2012, as well as to provide more time between talks for networking. We are very excited about the four keynotes by Paul Gilbert, Blair Hull, Rob McCulloch, and Simon Urbanek. The main agenda includes nineteen full presentations and eighteen shorter "lightning talks". We are also excited to offer six optional pre-conference seminars on Friday morning. Once again, we are hosting the R/Finance conference dinner on Friday evening, where you can continue conversations while dining and drinking atop a West Loop restaurant overlooking the Chicago skyline. More details of the agenda are available at:
http://www.RinFinance.com/agenda/
Registration information is available at
http://www.RinFinance.com/register/
and can also be directly accessed by going to
http://www.regonline.com/RFinance2012
On behalf of the committee and sponsors, we look forward to seeing you in Chicago!
Gib Bassett, Peter Carl, Dirk Eddelbuettel, Brian Peterson,
Dale Rosenthal, Jeffrey Ryan, Joshua Ulrich
Our 2012 Sponsors:
International Center for Futures and Derivatives at UIC Revolution Analytics
Sybase
MS-Computational Finance at University of Washington Google
lemnica
OpenGamma
OneTick
RStudio
Tick Data
See you in Chicago in May!!

17 March 2012

Matthew Garrett: More ways for firmware to screw you

Some of my recent time has been devoted to making our boot media more Mac friendly, which has entailed rather a lot of rebooting. This would have been fine, if tedious, except that some number of boots would fall over with either a clearly impossible kernel panic or userspace segfaulting in places that made no sense. Something was clearly wrong. Crashes that shouldn't happen are generally an indication of memory corruption. The question is how that corruption is being triggered. Hunting that down wasn't terribly easy.

My first thought was that we were possibly managing to load the kernel over a region used by UEFI code. UEFI defines two types of code - boot services and runtime services. While runtime services code and data must be preserved by the OS, in theory boot services code and data is available to the OS once the firmware has exited. In practice, that's not true. It seemed entirely possible that the kernel might be ending up on top of some of that boot services code or data and getting trodden on. Grub now has code to avoid putting the kernel on boot services, so testing the latest code seemed like a good plan. But no, crashes still happened.

That pretty much ruled out the bootloader. My next thought was that executing some of the firmware code was triggering a write to some other memory that contained the kernel. Josh Boyer suggested the next trick, which was to try marking the kernel read-only to see whether anything was hitting it. x86 lets you mark pages as read-only - any attempt to write to them should take a fault. UEFI functions are executed in the context of the kernel, so share the same page tables. That let me rule this out, since everything still went just as wrong and I wasn't taking an extra fault first.

However, at this point I was reasonably happy that it wasn't the kernel itself being overwritten - faults were occurring in userspace code as well. That was a pretty strong indication that what was happening was continuing to happen once userspace had started, so it wasn't a direct response to a firmware call. I made sure of that by stubbing out all the calls that could be triggered after kernel initialisation, and saw the same failures. Once all attempts to be clever have failed, it's time to just start using brute force. The kernel lets you reserve areas of RAM by passing arguments like memmap=0xlength$0xstart to block length bytes starting at start from being used. It took a while, but I finally found a 256MB range that made a difference - reserving it resulted in the machine booting reliably, letting the OS use it resulted in occasional crashes.

Definite progress. Comparing that memory range to the EFI memory map was helpful. There were several blocks of UEFI boot services data present there, which really seemed like too much of a coincidence. By reserving each of them in turn, I'd traced it down to a single 31MB region of boot service data - that is, memory reserved by the firmware for use by the UEFI boot services. Per spec, this is available to the OS once the boot environment has been exited. Nothing other than the OS should be touching this after boot, but something clearly was. Tracking down what was far easier than I expected, although the first attempt was a failure. Setting it read-only should have triggered a fault, but didn't. That was rather confusing. But, rather than give up, I patched the kernel to fill the region with 0xff at kernel init. Then I booted the system, read it back and looked for values that weren't 0xff. I got this:
00000000  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................ 
*
001568a0  ff ff ff ff ff ff ff ff  ff ff ff ff 84 00 00 00   ................ 
001568b0  00 20 a7 ac 46 00 00 00  00 00 00 00 00 00 06 01   . ..F........... 
001568c0  c2 0b 0c 00 ff ff ff ff  ff ff ff ff ff ff ff ff   ................ 
001568d0  ff ff 0a 04 f0 03 82 0d  40 00 00 00 ff ff ff ff   ........@....... 
001568e0  ff ff 00 21 00 36 9a 80  ff ff ff ff ff ff 00 7e   ...!.6.........~ 
001568f0  00 09 43 48 41 2d 47 75  65 73 74 01 04 02 04 0b   ..CHA-Guest..... 
00156900  16 32 08 0c 12 18 24 30  48 60 6c 2d 1a 0e 18 1a   .2....$0H l-.... 
00156910  ff ff 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00156920  00 00 00 00 00 00 00 dd  09 00 10 18 02 00 10 01   ................ 
00156930  00 00 dd 1e 00 90 4c 33  0e 18 1a ff ff 00 00 00   ......L3........ 
00156940  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00156950  00 00 bd ea f8 b3 ff ff  ff ff ff ff ff ff ff ff   ................ 
00156960  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................ 
*
022fb000
That's a lot of 0xffs (around 31MB of them) with one small section that contains an 802.11 probe packet with the SSID of the hospital across the road from my house. Apple support network booting off wireless networks. It seems that the firmware brought up the wireless card, associated with this network (it's the only public one nearby) and then left the card DMAing packets into RAM. The read-only page attribute only applies to CPU-initiated accesses, so it could do this without triggering a page fault. It also explained why it was so random - whether memory corruption occurred would depend on whether a packet appeared between that memory being used by the OS and the kernel reinitialising the wireless card. It certainly explains why I couldn't reproduce it when I left the machine repeatedly rebooting on the bus home.

How do we fix this? Unsure. With luck disconnecting the UEFI driver in the bootloader should quiesce the hardware, but without testing I'm not sure of that yet. For now it's just another example of firmware managing to break expectations in deeply strange ways.

comment count unavailable comments

15 March 2012

Andreas Barth: Traveling to DebConf 11 - Mostar to Sarajevo

During daytime, I finally bought a train ticket from Mostar to Sarajevo. The ticket for the 100 kilometers trip was 10 KM, which are 5 Euro.
Mostar train station Right on time the train entered the station. However, we had a 30 minutes delay there without any obvious cause or communications. This train was (as well as the previous in Hrvatska) between full and overcrowded, in spite of the bad connections. I however learned soon why these carriages shouldn't go to other parts of europe: The window in our compartment was replaced, so it couldn't be opened any more (and we had no means of fresh air). That wasn't as bad as it sounded, as the train doors weren't locked, so someone opened the door while the train was moving and "locked" it with paper so we could get fresh air (and this configuration stayed all the way to Sarajevo at least, so it was that way for about 4 hours). After leaving Mostar, there were soon signs between the train and the river that the area was mined, so one shouldn't go there. The train drives through beautiful landscape. After it got dark, the electric light within the train was not working, so we had (at best) our mobil phones to provide us with light (strange modern times).
Mine warning between the train tracks and the River Neretva In spite (or: because?) of these technical issues, I meet a few local people from Sarajevo and from all parts of Europe. So the train trip proved to be nice and entertaining, and I learned a bit about government issues there. Along the tracks, the signals were non-functional. The trains were directed only by flags (and hand-lights) from personal. On entering any station the train had to slow down till I got shown the relevant flag or hand-light that allowed it to leave the station again. Another heritage from the last war. With due delay, the train arrived in Sarajevo. Due to missing information at the tram stop, I didn't know that the trams to the train station didn't run that late (but I had the information that there would be trams if the train was on time). As it was late, I shortcut that by taking a taxi to the hotel (if I were to arrive in Sarajevo again, I would walk one tram station to the main line - but well, travelling is also about experiences). Unplanned that evening was also the Sarajevo film festival, with one stage opposite of the hotel window. As this was almost closing down when I arrived in Sarajevo, nothing to worry but to enjoy.

9 March 2012

Petter Reinholdtsen: Debian Edu interview: Nigel Barker

Inspired by the interview series conducted by Raphael, I started a Norwegian interview series with people involved in the Debian Edu / Skolelinux community. This was so popular that I believe it is time to move to a more international audience. While Debian Edu and Skolelinux originated in France and Norway, and have most users in Europe, there are users all around the globe. One of those far away from me is Nigel Barker, a long time Debian Edu system administrator and contributor. It is thanks to him that Debian Edu is adjusted to work out of the box in Japan. I got him to answer a few questions, and am happy to share the response with you. :) Who are you, and how do you spend your days? My name is Nigel Barker, and I am British. I am married to Yumiko, and we have three lovely children, aged 15, 14 and 4(!) I am the IT Coordinator at Hiroshima International School, Japan. I am also a teacher, and in fact I spend most of my day teaching Mathematics, Science, IT, and Chemistry. I was originally a Chemistry teacher, but I have always had an interest in computers. Another teacher teaches primary school IT, but apart from that I am the only computer person, so that means I am the network manager, technician and webmaster, also, and I help people with their computer problems. I teach python to beginners in an after-school club. I am way too busy, so I really appreciate the simplicity of Skolelinux. How did you get in contact with the Skolelinux/Debian Edu project? In around 2004 or 5 I discovered the ltsp project, and set up a server in the IT lab. I wanted some way to connect it to our central samba server, which I was also quite poor at configuring. I discovered Edubuntu when it came out, but it didn't really improve my setup. I did various desperate searches for things like "school Linux server" and ended up in a document called "Drift" something or other. Reading there it became clear that Skolelinux was going to solve all my problems in one go. I was very excited, but apprehensive, because my previous attempts to install Debian had ended in failure (I used Mandrake for everything - ltsp, samba, apache, mail, ns...). I downloaded a beta version, had some problems, so subscribed to the Debian Edu list for help. I have remained subscribed ever since, and my school has run a Skolelinux network since Sarge. What do you see as the advantages of Skolelinux/Debian Edu? For me the integrated setup. This is not just the server, or the workstation, or the ltsp. Its all of them, and its all configured ready to go. I read somewhere in the early documentation that it is designed to be setup and managed by the Maths or Science teacher, who doesn't necessarily know much about computers, in a small Norwegian school. That describes me perfectly if you replace Norway with Japan. What do you see as the disadvantages of Skolelinux/Debian Edu? The desktop is fairly plain. If you compare it with Edubuntu, who have fun themes for children, or with distributions such as Mint, who make the desktop beautiful. They create a good impression on people who don't need to understand how to use any of it, but who might be important to the school. School administrators or directors, for instance, or parents. Even kids. Debian itself usually has ugly default theme settings. It was my dream a few years back that some kind of integration would allow Edubuntu to do the desktop stuff and Debian Edu the servers, but now I realise how impossible that is. A second disadvantage is that if something goes wrong, or you need to customise something, then suddenly the level of expertise required multiplies. For example, backup wasn't working properly in Lenny. It took me ages to learn how to set up my own server to do rsync backups. I am afraid of anything to do with ldap, but perhaps Gosa will help. Which free software do you use daily? Nowadays I only use Debian on my personal computers. I have one for studio work (I play guitar and write songs), running AV Linux (customised Debian) a netbook running Squeeze, and a bigger laptop still running Skolelinux Lenny workstation. I have a Tjener in my house, that's very useful for the family photos and music. At school the students only use Skolelinux. (Some teachers and the office still have windows). So that means we only use free software all day every day. Open office, The GIMP, Firefox/Iceweasel, VLC and Audacity are installed on every computer in school, irrespective of OS. We also have Koha on Debian for the library, and Apache, Moodle, b2evolution and Etomite on Debian for the www. The firewall is Untangle. Which strategy do you believe is the right one to use to get schools to use free software? Current trends are in our favour. Open source is big in industry, and ordinary people have heard of it. The spread of Android and the popularity of Apple have helped to weaken the impression that you have to have Microsoft on everything. People complain to me much less about file formats and Word than they did 5 years ago. The Edu aspect is also a selling point. This is all customised for schools. Where is the Windows-edu, or the Mac-edu? But of course the main attraction is budget.The trick is to convince people that the quality is not compromised when you stop paying and use free software instead. That is one reason why I say the desktop experience is a weakness. People are not impressed when their USB drive doesn't work, or their browser doesn't play flash, for example.

8 March 2012

Rapha&#235;l Hertzog: People Behind Debian: Gregor Herrmann, member of the Perl team

Photo by Aigars Mahinovs

I followed Gregor s evolution within Debian because I used to be somewhat active in the Perl team. His case is exemplar because it shows that you don t need to be an IT professional to join Debian and to make a difference. His QA page is impressive with hundreds of packages maintained and hundreds of non-maintainer uploads too. While he started out slowly, I remember meeting him at Debconf 7 in Edinburgh and after that he really got more implicated. Again a case of someone joining for technical reasons but getting more involved and staying there for social reasons! :-) Let s jump into the interview and learn more about him. Raphael: Who are you? Gregor: I m 41 years old, and I live in Innsbruck, Austria, in a shared apartment with a friend of mine. In my day job, I m working at the regional addiction prevention agency, so I m one of the few Debian guys who s not an IT student or professional. I started maintaining packages in 2006, and I am a DD since April 2008. Raphael: How did you start contributing to Debian? Gregor: After having used Debian on servers for some years, I finally switched to it on the desktop after some procrastinating. Soon afterwards I wanted to know more about the making-of , started to join mailing lists, filed bugs, and tried to learn packaging. Luckily I quickly found a permanent sponsor Tony Mancill, and we re still co-maintaining each others packages. And when I packaged my first Perl modules, Gunnar Wolf invited me to join the Debian Perl Group, an offer I accepted a few days later. And I m still there :) Later, the NM process, although it involved some waiting times, was also a good learning experience due to my AM Wouter Verhelst. (And in the meantime the organization of the NM process has vastly improved, from what I hear.) So my starting point for joining Debian was my curiosity but what really helped me to find my way into the project was the support of the people who invited and helped me. Raphael: What s your biggest achievement within Debian or Ubuntu? Gregor: I m not sure I can name a single big achievement but I guess I can say that my contributions to the Debian Perl Group have helped to make and keep the team a success story. Raphael: The pkg-perl team seems to work very well. As an active member, can you explain us how it is organized? How do you explain its success? In particular it seems to be a great entry point for new contributors. Gregor: The team is huge, both in numbers of members and packages (over 2200). Since last DebConf we manage our source packages in git, we have 2 mailing lists and an IRC channel, and we manage to keep an overview by using PET, the Package Entropy Tracker. It s true that we get new members on a regular basis; we try to invite people (like it happened to me 6 years ago :) ) but there are also quite a few new contributors who find our docs and introduce themselves on the mailing list. Maybe someone should conduct a study and ask them what motivated them to join. :) We hand out group membership/commit access quickly, and we try to mentor new contributors actively during their early times in the group. Some of them leave for other projects after some time, but many also stay and become DDs later. I m not sure what the reasons for the group s success are, maybe a combination of: For everyone interested in joining the Debian Perl Group, our Welcome page on the wiki is a good starting point. Raphael: What are your plans for Debian Wheezy? Gregor: Nothing overly exciting. What I should do is getting a newer JabRef into Debian (which involves packaging some new Java libraries any takers?). A solution for libdatetime-timezone-perl (which ships timezone data converted to Perl modules and tends to get outdated when the timezone data change) would be nice; let s see if #660404 leads to some results And some Perl packages will also need a bit of work for the hardening build flags release goal (cf. #657853). Raphael: What s the biggest problem of Debian? Gregor: Inertia. While I really like the fact that Debian is a volunteer project, and that every contributor works when and on what they decide to work on, I get the feeling that Debian could do better in moving forward, innovating, taking decisions. I also think that more uniformity in managing source packages would make things easier; it s quite amazing to see how many source formats, packaging helpers, patch systems, RCSs etc. are used all over the archive. I m not advocating for mono-cultures, and I consider this diversity a strength in general, but having to find out first how this particular package works when preparing a bug fix can be annoying. On the bright side, I think that the myth Debian and its mailing lists are mostly about flames can be seen as dispelled in the meantime. Sure, sometimes the tone could be a bit more civil, but in general most of the interactions I ve seen in the last years were friendly and helpful. IMO, the Debian Project consists of mostly nice and cooperative people, and that s what makes it fun for me. Raphael: You re one of the most dedicated participants to RCBW (Release Critical Bugs of the Week), an initiative to fix RC bugs every week. How much time do you spend on it? What would you advise to people who are considering to join the movement? Gregor: I got into the habit of fixing RC bugs after having been invited to my first Bug Squashing Party in Munich some years ago. During this weekend I saw that fixing RC bugs can be fun, is often not that difficult, and gives a warm fuzzy feeling :) I can definitely recommend attending a BSP if one happens to be organized near you. After tasting blood at this first BSP I tried to continue looking at RC bugs, and I guess I spend something around half an hour per day on it. I usually blog about it once a week, in order to motivate others to join in. And joining is easy: just take a look at the tips people like Zack, Vorlon, or me have written. You don t have to be a DD to help, many of my NMUs are based on patches that others kindly prepare and send to the BTS kudos! Another nice aspect is that the RC bug list contains problems from different fields: general packaging problems, language-specific issues, policy violations, etc. So there s something for everybody, and you don t have to be an expert in all fields to fix a specific bug. What s rewarding about fixing RC bugs is not only the feeling of accomplishment and the knowledge about having helped the next release I also received quite a few Thank you mails from maintainers who were busy at that time and appreciated the help. Raphael: Do you have wishes for Debian Wheezy? Gregor: Well, there s not so much left of the Wheezy release cycle if we manage to freeze in June :) Some quick thoughts for Wheezy and Wheezy+1: Raphael: Is there someone in Debian that you admire for their contributions? Gregor: There are many people in Debian I admire, too many to name them all. The first one that comes to my mind is Russ Allbery who not only does great work from lintian to Debian policy but who also sets a great example of communicating in a perfectly polite and respectful way even in heated discussions.
Thank you to Gregor for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Note that older interviews are indexed on wiki.debian.org/PeopleBehindDebian.

Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Google+, Twitter and Facebook.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

4 March 2012

Gregor Herrmann: RC bugs 2012/09

as usual, here's my RCBW report. like in the last weeks, a mixture of NMUs and mails to the BTS, grabbing and writing patches, ...

Next.

Previous.