Search Results: "nate"

27 May 2012

Petter Reinholdtsen: Debian Edu interview: Ralf Gesellensetter

In 2003, a German teacher showed up on the Debian Edu and Skolelinux mailing list with interesting problems and reports proving he setting up Linux for a (for us at the time) lot of pupils. His name was Ralf Gesellensetter, and he has been an important tester and contributor since then, helping to make sure the Debian Edu Squeeze release became as good as it is.. Who are you, and how do you spend your days? I am a teacher from Germany, and my subjects are Geography, Mathematics, and Computer Science ("Informatik"). During the past 12 years (since 2000), I have been working for a comprehensive (and soon, also inclusive) school leading to all kind of general levels, such as O- or A-level ("Abitur"). For quite as long, I've been taking care of our computer network. Now, in my early 40s, I enjoy the privilege of spending a lot of my spare time together with my wife, our son (3 years) and our daughter (4 months). How did you get in contact with the Skolelinux/Debian Edu project? We had tried different Linux based school servers, when members of my local Linux User Group (LUG OWL) detected Skolelinux. I remember very well, being part of a party celebrating the Linux New Media Award ("Best Newcomer Distribution", also nominated: Ubuntu) that was given to Skolelinux at Linux World Exposition in Frankfurt, 2005 (IIRC). Few months later, I had the chance to join a developer meeting in Ulsrud (Oslo) and to hand out the award to Knut Yrvin and others. For more than 7 years, Skolelinux is part of our schools infrastructure, namely our main server (tjener), one LTSP (today without thin clients), and approximately 50 work stations. Most of these have the option to boot a locally installed Skolelinux image. As a consequence, I joined quite a few events dealing with free software or Linux, and met many Debian (Edu) developers. All of them seemed quite nice and competent to me, one more reason to stick to Skolelinux. What do you see as the advantages of Skolelinux/Debian Edu? Debian driven, you are given all the advantages of a community project including well maintained updates. Once, you are familiar with the network layout, you can easily roll out an entire educational computer infrastructure, from just one installation media. As only free software (FOSS) is used, that supports even elderly hardware, up-sizing your IT equipment is only limited by space (i.e. available labs). Especially if you run a LTSP thin client server, your administration costs tend towards zero. What do you see as the disadvantages of Skolelinux/Debian Edu? While Debian's stability has loads of advantages for servers, this might be different in some cases for clients: Schools with unlimited budget might buy new hardware with components that are not yet supported by Debian stable, or wish to use more recent versions of office packages or desktop environments. These schools have the option to run Debian testing or other distributions - if they have the capacity to do so. Another issue is that Debian release cycles include a wide range of changes; therefor a high percentage of human power seems to be absorbed by just keeping the features of Skolelinux within the new setting of the version to come. During this process, the cogs of Debian Edu are getting more and more professional, i.e. harder to understand for novices. Which free software do you use daily? LibreOffice, Wikipedia, Openstreetmap, Iceweasel (Mozilla Firefox), KMail, Gimp, Inkscape - and of course the Linux Kernel (not only on PC, Laptop, Mobile, but also our SAT receiver) Which strategy do you believe is the right one to use to get schools to use free software?
  1. Support computer science as regular subject in schools to make people really "own" their hardware, to make them understand the difference between proprietary software products, and free software developing.
  2. Make budget baskets corresponding: In Germany's public schools there are more or less fixed budgets for IT equipment (including licenses), so schools won't benefit from any savings here. This privilege is left to private schools which have consequently a large share among German Skolelinux schools.
  3. Get free software in the seminars where would-be teachers are trained. In many cases, teachers' software customs are respected by decision makers rather than the expertise of any IT experts.
  4. Don't limit ourself to free software run natively. Everybody uses free software or free licenses (for instance Wikipedia), and this general concept should get expanded to free educational content to be shared world wide (school books e.g.).
  5. Make clear where ever you can that the market share of free (libre) office suites is much above 20 p.c. today, and that you pupils don't need to know the "ribbon menu" in order to get employed.
  6. Talk about the difference between freeware and free software.
  7. Spread free software, or even collections of portable free apps for USB pen drives. Endorse students to get a legal copy of Libreoffice rather than accepting them to use illegal serials. And keep sending documents in ODF formats.

22 May 2012

Gunnar Wolf: e-voting: Bad when it's near, worse when it's far.

Note: All of the information linked to from this post is in Spanish and related to Mexico Part of it will be translatable via automated means, some will not. Sorry, that's what I have, and it's too much text to invest the effort to hand-translate
I have been following the development of the different e-vote modalities in Mexico for several years already, although I have only managed to do so methodically in the last half year or so. If you are interested in my line of reasoning as to why I completely oppose e-voting, you can look at the short article I published in 2010 or the slightly longer and more updated version published in our book in 2011. Currently, in Mexico there are two different venues of e-vote that are being pushed: Bad and worse. The bad one will be carried out for about 10% of the population of the state of Jalisco and somewhat less for the state of Coahuila (Distrito Federal was also to be in this list, but the contract was cancelled due to the provider company delivering booths with too many problems and unable to deliver in the due time). The worse one is, fortunately, likely to have the least impact. Why? Because it regards votes cast by Distrito Federal residents (the capital entity, where part of Mexico City is located) living abroad. And it will have less impact because of the amount of the population registered for it: We are about 9 million residents in DF, and in the last election (first time IIRC there was the right to vote from abroad) there were only about 10,000 people registered for casting a (enveloped and sent by post) vote. Even if this year we the campaign for this was better (and I'm not yet sure about it), the number of voters will not be enough to make a dent on the results. I'm not going into details as to why it is bad in this post I requested information from the DF Electoral Institute (IEDF) with academic interest, to try to find more information about it, and I want to share my results with you and, of course, to request for your input on how to continue with this. On May 3rd, I sent the following request (this I am translating to English :) You can look at the receipt for the request for the original redaction) to the official contact address, oficinadeinformacionpublica@iedf.org.mx: Of course, I wasn't very optimistic when receiving this information. Still, I have to share my results: My information request was largely denied:
( )
III. The divulgation of this information harms the interest it protects
Given that, were it to be divulged it would affect the informatic security of the refered system. Anyway, we have to point out that said systems have enough measures and security provisions, so that the citizen can emit his vote in a universal, free, secret and direct way.
IV. The damage that can be produced by making this information public is larger than the public interest to know it
This is so because making this information public puts at risk the correct development of the Internet-based voting, because were the technical, purpose-specific information be made public, it could be misused to carry out informatic attacks.
It is also important to mention that a confidentiality agreement was signed with the company that developed said systems.
( )
VI. The time for the information to be reserved
It will be seven years starting at the present resolution, this information will be made public when the reserve period is over or when the target is reached, except for the confidemtial information that it could contain. ( )
In case some other person is interested in following this information, the other two points were answered, and I'll try to get some relevant information from it: So, I don't have any real conclusions yet. I'm just reporting how work is unfolding. Tomorrow evening (Wednesday May 23) I'll give a talk on the "e-voting in Mexico 2012" subject in Congreso Internacional de Software Libre in Zacatecas, Mexico. I'll talk on the situation on this and the other topics I have been able to work on.
AttachmentSize
Receipt of my information request11.64 KB
Transparency Committee resolution354.66 KB
Information request answered by UTCSTyPDP (IEDF)85.75 KB

Russell Coker: Another USB Flash Failure

I previously wrote about a failure of a USB flash device in my Internet gateway [1]. I have since had another failure in the same system, so both the original 4G devices are now dead. That s two dead devices in 10 weeks. It could be that the USB devices that I got for free at an exhibition were just really cheap, I m sure that they weren t expecting them to be used in that way. The devices from the same batch which are used for their intended purpose (sneaker-net file sharing) are still working well. But in any case I m not going to resume this experiment until warmer weather. At this time of year some extra heat dissipation from computer gear in my home is more like a feature and less like a bug. The second USB device to fail appeared to have it s failure in the Ext4 journal (the errors were reported at around sector 2000), I didn t keep a record of the problem with the first device, but from memory I think it was much the same. Rumor has it that cheap flash storage devices don t implement wear-levelling to avoid patent infringement. If that rumor is correct then any filesystem that uses a fixed journal in the same way as Ext3/4 is probably unsuitable for any serious use on such devices, while a filesystem based on Copy On Write will probably perform better. In Spring I ll try using BTRFS on cheap USB flash devices and see if that works better. I have another spare device from the same batch to test so I can eliminate hardware differences. I can t do enough tests to be a good statistical sample, but if a device lasts from Spring to Autumn using BTRFS with the same use that caused failures with Ext4 in a few weeks then I will consider it a strong indication that BTRFS is better than Ext3/4 for such uses. For the next 5 months or so I ll be using a hard drive in my Internet gateway system again. Related posts:
  1. Flash Storage Update Last month I wrote about using USB flash storage devices...
  2. flash for main storage I was in a discussion about flash on a closed...
  3. USB Flash Storage For some years I have had my Internet gateway/firewall system...

20 May 2012

Gregor Herrmann: RC bugs 2012/20

I was on vacations for a few days last week, therefor my list of RC bugs is a bit shorter this time. thanks again to everyone who sent patches to the BTS that I could just use.

19 May 2012

Thorsten Glaser: MirBSD goes webservice manpages and WTF

While wtf(1) always has been a bit central to MirBSD, and the acronym database has been accessible by CVSweb, what we never had was a DAU compatible (and shellsnippets compatible) lookup. This has now changed: the above link to the acronyms file is a persistent link to its latest version (well, latest when the website was last recompiled), tooltips may very well follow soon, and we ve got an online WTF lookup service.
Contributions to the acronym database are welcome, of course; just eMail them to tg mirbsd org. Not to stop there, our online HTML manpage search is also new, shiny, and should replace the !mbsdman DuckDuckGo hash-bang shortly. (Both of these services offer a DDG search as fallback. Note that DDG is an external service included herein by linking, under their request to spread it, and not affiliated with The MirOS Project. They do, however, donate some advertising money to Debian.)
For all those who didn t know: only manpages for software in the MirOS BSD base system and for the MirPorts Framework package tools are listed, not for third-party applications installable using ports or, recently, pkgsrc . Still, if you want to have a peek at a modern classic BSD s documentation, you re welcome. (Not to mention content like re_format(7) and style(9) and that some of our documentation is much more legible than others.) And because writing all that perl(1) made me ill, not to mention I don t even know that language, I ve hacked a bit more in the mir make(1) and mksh(1) parts of the MirWebsite, finally implementing pointing out where in the navigation sidebar the visitor currently is. We also have exciting mksh porting news involving RT trying a larger number of ancient platforms than I dare count, me fixing bugs in Linux klibc and diving into other things, learning more about why I consider me lucky for hacking a BSD operating system sorry, I want to keep this short as it s mostly an announcement. The MirWebsite source code is, of course, also available. Improvements welcome. Except for these three CGIs, our website is fully statically precompiled, and that s a good thing. Please help in making the CGIs secure.

Andrew Pollock: [life] Maker Faire 2012 trip report

The Maker Faire is one of those awesome Bay Area things that always fills me with excitement and gets my imagination going. Zoe and I went again this year to check it out, as best we could within the time constraints we had to work within (opening time and her nap time, minus travel time). She definitely enjoyed herself. We took the Caltrain, because historically driving and parking has been a bit of a nightmare. The optimal train to get to get there before it opened (at 10am) was the 9:19 train from Mountain View, which was scheduled to get in at Hayward Park a little before 10am. It just so happened that there was a Giants game on in San Francisco today as well, and the train was absolutely packed. We only got a seat because one kind gentleman was getting off and explicitly gave his seat to us. One lesson learned: don't try and take the BOB stroller on the train. Even when collapsed, it's way too bulky. For future Caltrain outings, I'll take our City Mini stroller instead, as it folds much flatter. I also took our macpac Possum child carrier backpack, and Zoe was pretty happy to just sit in it for the bulk of the time. I think it had novelty value for her, as we haven't used it for a while. I probably could have gotten away without taking a stroller at all. I was very glad I took the backpack, as it gave her a much better vantage point for everything that was going on than she would have gotten from sitting in the stroller. There was supposed to be a free shuttle from the Hayward Park station to the Maker Faire, but there was a huge crowd waiting for it, so I decided to just walk. It didn't take too long. For the return trip, I think I exited from the wrong side of the fairgrounds, and couldn't figure out the shuttles, so I just walked to Hillsdale station. At least the return train wasn't crowded. Overall, using Caltrain to get in and out was successful. Zoe was very well behaved for the ~30 minute train ride each way. The Faire was quite a bit bigger this year, and has spilled out into the parking lot on one side. I'd heard stories that O'Reilly had quadrupled booth prices as well. Trying to abide by the program was too difficult, so we mostly just wandered through the main Expo hall and looked at various booths. I just did a full read through the website of all the exhibitors to see what I missed out on. Here's some of the stuff I saw in person, or discovered via the website: Kickstarter is really becoming huge in the maker community. There were heaps of exhibitors there with (mostly robotics) projects that were past the initial prototyping phase and were seeking funding on Kickstarter to go into mass production. Some of the talks I'd have liked to have seen: Zoe was really well behaved for the entire expedition. I don't think she really gave me any grief at all. There was a brief period where she wanted me to carry her, but I managed to negotiate her back into the stroller after not long. I think her favourite was ArcBotics, which had a robot insect that would dance and wave at her. She kept asking for it to do more dancing.

Richard Hartmann: Mongolia

I will start linking to places we stayed at etc, from now on. On the one hand, we already left those places so my paranoid self can rest assured that no one will come hunt us down, on the other hand, I decided to only to name and link to those places which really deserve a mention. If they are linked here, I can whole-heartedly recommend them. Mongolia After leaving through the Russian border fortifications, Mongolia was an instant and welcome change. Trees growing right next to the tracks, branches forming a bit of a natural tunnel; the trains passing by keeping branches at bay, but the trees free to grow wherever else they pleased. A few hundred meters later below a rocky ledge, a cow had apparently fallen down and died. Partially decomposed flesh still clinging halfway to its rib-cage, it proved to be a sign of the (mostly) more laid-back attitude Mongolians tend to take. After finally getting our passports back and being allowed to leave the train, we first headed to the station to get some money. At ~1800:1, the exchange rate for Tugruk against Euro speaks volumes about inflation; imagine my surprise that the MNT 20,000 note is the largest one. It's also the one and only amount any and all ATMs will default to. We were quite surprised by how cheap everything was, even in the one single store within the station directly behind the border. In Russia, travellers expect to pay hefty markups when shopkeepers know they can charge them, not so in Mongolia. The woman in the trash Upon returning from the station, our waggon was just merrily being driving away on its own by a lone engine, something we had not anticipated or appreciated a whole lot. A woman (we hadn't noticed her before) who was busy scavenging a trash container for food for herself and her scrawny-looking son happily stopped what she was doing and, with a big smile, moved her hands around and smacked them back together, signalling us that our waggon would be attached to a different train for the subsequent journey. To put it mildly, I was stunned; we had met people eating from the trash in Russia as well (yes, I always make a point of giving them enough for a few decent meals; anyone eating out of the trash actually needs and definitely deserves something good happening to them), but they were glum and disheartened. Here, there was someone who not only had to look out for herself, but for a three or four year old boy as well, and still was just so... positive... I don't know if I would have it in me to act the same way... As an aside, even though we were just a few kilometers across the border from Russia, she did not actually take the money from me, she cupped her hands to receive it by having me place it into her hands; this is a very Asian thing in my experience and something we would see both in Mongolia and China consistently. Later, when we were back in our waggon, she passed by and happened to see the Spot Messgenger 2 dangling in front of our window, its lights blinking every five seconds. For a solid twenty minutes, until the train started moving again, she squatted down with her son, oggling the Spot. I have no idea what she was thinking it was and I had no chance to explain it to her, but she was really fascinated and just kept staring and staring. When we finally left the station, we waved to and smiled at each other, she ran after us, and then she was gone. All in all, this was one the more memorable things which happened to us. Mongolian Infrastructure When I say that Mongolians as a whole tend to be a lot more laid back than Russians, this extends to infrastructure as well, but not in the good way. Ulan Bator Little Dhingis Khan We arrived in Ulan Bator at six in the morning and started haggling with taxi drivers. Our hostel had (stale, it turned out) information about acceptable fare from station to their place on their website, 5,000 MNT was supposedly OK. Our driver tried to get 20,000, then 15,000, then 12,000, then 10,000, went down to 7,000 and finally settled for 5,000, all this interspersed with frequent "no" by us and walking away. Once on the road, he wanted to convince me that we had agreed upon 5,000 per person, something that did not quite work out for him. When we arrived, he started to tell me that he was good friends with the hostel manager and that said manager would surely want us to pay more; again, that did not work too well. He made the mistake of getting our luggage out of his trunk immediately (we always kept all our stuff in the passenger area afterwards...) before I paid and even though I only had a 10,000 or 20,000 note, I insisted on correct change. The same old dance began anew, with him edging closer to the correct change in steps of 1,000 MNT. He tried to get away several times, but I stood in his door and held it open; if he had just driven off, I couldn't have stopped him, but that thankfully didn't occur to him. After he shoved all small change (MNT is bill only, no coins) he had with him at me and wanted to make off yet again, I was close to throwing all the small crap at him, but after he, a man of maybe 1.65 meter and sitting down, tried to signal his willingness for a fist fight and me, a man of 1.94 meter and standing started laughing, he gave me 2,000 more and drove off cursing. In case you forgot, at that exchange rate, we argued about two to three Euro, but as he tried to cheat, it became a matter of principle. The Hostel, part one All parking lots and other property in Ulan Bator have a small watchtower with private 24/7 guards; said guard ambled down, let us in and woke the poor woman who was forced to let us check in. As she was Mongolian as well, we were a bit surprised when she switched from broken English to perfect German, but the owners of the OASIS are a German-Austrian couple so I guess that makes sense. Fighting through the war Traffic in Ulan Bator is war. Lonely Planet talks about the city being too newly motorized and that it did not have to develop a culture of driving, yet. This is a euphemism for "no one has any qualms maiming or killing you". Everyone is driving like mad, nudging and racing their way in front of each other. If a driver has an oppurtunity to get ahead by a few centimeters with his small car, thereby blocking the way for several buses, making a whole parking lot grind to a halt and thereby deadlocking themselves, they will take said opportunity, and gladly. Red lights are gentle suggestions, there to be mainly ignored; police will not drive over red lights, but they will not stop anybody else from doing so, either. A hearse will have a dozen or more cars following it, all driving in a tight pack, aggressively attacking anyone who dares to wedge in between them and expecting all other traffic to make way even if they are driving over a red light. Crossing a main street in Ulan Bator as a pedestrian is hell. There are a few traffic lights and they help to some extent, but you still need to be extremely careful when crossing a street. Thailand, India, Russia, China, no matter where it's supposedly hard to cross a street by foot, I never had any issues weaseling through. In UB, it takes planning, determination, and a lot of attention, especially as there are stretches where there's not a traffic light in sight. While maintaining eye contact with a driver usually ensures that they will try not to hit you, this does not work in the least in UB. In the evening, I took to shining a high-powered flashlight at the ground to the oncoming traffic's side to mark ourselves and decrease the likelihood of being hit. This worked well in combination with sprinting through gaps in the traffic, except for one guy who actively aimed for us and did not slow down. He even changed lanes, just to keep us in front of his car... Shining the flashlight directly at him in highest mode made him break quite violently though; a good thing, as he may have hit us had he not slowed down. UB itself Ulan Bator is weird. It has clear signs of westernization, but it's still very much an old city. There is one large store which would fit into any random shopping center in Germany (which are tiny when compared to most other countries) and which carries most goods, if with a limited selection. Its supermarket is extremely well stocked with food and drink from all other the world and laughably cheap (for us) prices. While its infrastructure is definitely crumbling, the city is clean, especially when compared to Russia. Two girls from Stuttgart who we met during our Trans-Siberian travels commented on how clean Russian cities were when compared to home, but that may be more of a statement about Stuttgart than about Russia... Anyway, in Mongolia in general and in UB in particular, there's a concise effort to make what they have look nice and clean. There is no trash in the streets, loose gravel and earth is removed from the curb by dedicated workers, all areas of packed earth are sieved and all large stones removed. All in all, it's a lot cleaner (if more dusty and sandy) than Russian cities if somewhat older and more shabby-looking. That being said, Mongolia's countryside is littered There were several street stalls consisting of nothing more than a cardboard box on a stool selling single chewing gums and cigarettes from the original packaging; it's normal for people to buy one cigarette or one piece of chewing gum and walk on. The sights of UB were nice, but nothing too special. Outlook In the next part, I'll cover "The Hostel, part disaster" and generally talk more about excrement than I thought one could write before moving on to China (were I am located at the time of this writing).

Richard Hartmann: Motherland's bosom

I read a translated poem about Russia being "the Motherland" and its vast bosom years ago. Having driven through a significant part of it, I can agree on the "vast" part... Also, as I am on a train and without access to the Internet, I will refrain from linking to a lot of pages; sorry. (Turns out I am posting this a week later, but I will still not link to stuff now; no time). Russia in general Moscow Sights Kreml Our remaining time in Moscow was spent with touring the usual suspects; the Kreml is a lot less impressive in real life, the Red Square is tiny when compared to the stories I heard about it and the Chapel ofi St. Basil is even more colorful and impressive in real life. Lenin's body was inaccessible because workers built seats for the May 9th parade to the left and the right of it and they apparently thought it would be a good idea to block access to one of the main tourist attractions while doing so. A river tour of Moscow was a nice cool-off and we got to see quite a few things. We managed to see the weekly military parade within the Kreml grounds, but it was mostly pomp and little substance. The National Treasure which you can access with an extra ticket within the Kreml grounds is nice, but less impressive than the tourist guides would make you believe. That being said... There's another museum within the museum and.... Whoah... Tourists pay extra, visitors go through the only non-security-theater check I encountered in Russia, guards are armed, people can only enter and leave in batches, and the stuff which is presented is mind-boggling. Disregarding the fist-to-calf-sized chunks of gold and platinum which are still in their original form directly from the mine, there is real, actual treasure galore. Little heaps of uncut and cut diamonds, an outline of Russia filled with cut diamonds and other random "we have this stuff" displays can be found as well. Then, you have various tiaras and other jewellery made from various gems. Not incorporating, but largely made of. All that pales in comparison to the crown, royal apple, scepter, etc. It's hard to put the amount of tiny multi-colored light points that shine at you into words. I was just standing there, swaying back and forth to catch the moving pattern of pinpoints. It's said that this collection is equalled only by the ones in the Tower of London and the one Shaw of Iran had and boy do I believe it. TV Tower Getting up there was funny. The old-style Soviet queuing system was used: "Security" for approaching the tower was multi-level, the guards see you approach along a long walkway way in advance and the main guard shed had several small cabins separated by thick glass. So good so menacing. But in a twist that would make Bizarro and Garry Larson proud, I was required, by means of metal detector gate, metal detector wand and even an x-ray machine to remove every shred of metal and other hard objects from myself and the camera bag and put them onto a table. Once I was without anything except my clothes and the bag was completely empty, I could pass. Everything I had had to remove was just laying there, not inspected in the least, for me to stuff back into pockets and bag and to take with me. This "everything" included a Spot Messenger 2 with lots of green and red blinky lights. The guard did not even glance and it. Security theater? Security theater. The view from 364 meters down on Moscow was nice, but there was a lot of Smog so I couldn't see very far. Jumping on the glass floor while looking down was a lot of fun, though. Subway to Thiefing I bet Christopher Nolan rode the subway in Moscow at least once. That unnerving sound you hear during several key scenes in "The Dark Knight"? Two thirds of all subways make the same sound while moving. Also, I had an encounter with a pickpocket down there; very classical, too. Guy approaches quickly, talks loudly and sounds as if it's really important (in Russian... duh... that's sure to keep me interested). His approach made me turn and protect my left leg pocket automatically, most likely marking the target for the tiny woman standing behind me. Now, I have to tell you something about my usual travel layout. As my normal pockets are very deep, it looks as if their content was in the leg pocket. Plus, there's an extra, hidden leg pocket where I keep the passports and train tickets. The outermost leg pocket is protected by a velcro flap, but it contains nothing of value; usually the appropriate phrasebook, local map, maybe a tissue or chewing gum. Due to this layering, the outermost pocket looks as if it's full to the brim with stuff. Also, I took pains to make it a habit to protect said leg pocket with my hand, nothing else. This looks as if that's the target, but what I am actually doing is protect my normal pocket with my forearm. The right side is different, but the most easily accessibly pocket always holds some small change. I pay from that stash but my actual wallet is well out of reach. Anyway, once the guy ran off, talking to several others, most likely marking all them for the actual pickpockets, I wanted to enter the subway. While the Russian-style queuing took place, I felt an unusual tug at the velcro flap. I looked down and saw a tiny woman to the left of me with a jacket held over her right side with the left arm; I look up to check no one is trying to steal from my permanently assigned female, feel another tug, look the woman into the eyes, look up again and around me, look down again and she is gone. All that took maybe three seconds and I had boarded the subway after an additional two. In hindsight, it makes sense to choose the time of entry for attack. It's crowded, you are being pushed around, and once you are in the subway, it will start moving more or less immediately while the thief remains in the station. In this case, she would only have gotten a grubby map of Moscow's subway and an English-Russian phrasebook, but she got nothing at all. Moscow-Novosibirsk Where to begin... If you think a few hours on a train are a long time, try over fifty hours. Things get so bad, you start getting land-sick while not in a moving train. You even start missing the familiar tunk-cachunk, tunk-cachunk, tunk-cachunk... of driving over rails with gaps in them when you are not moving. The defining element of the Trans-Siberian Railway are birch trees. And birch trees. And then more birch trees. You would not believe how many birch trees there are. This is made "worse" by the way the Russian Railway protects their rails. Left and right of the track, there's a cleared area of maybe ten to twenty meters, sometimes as little as three. Outside of that, they plant ten to twenty meters of birch trees, presumably to catch snow during winter. Beyond that protective perimeter, there's the normal landscape.As a result, on top of the near endless stretches of birch woods, you see most if not all scenery through a layer of birch trees. You get sick sick of birch trees after a few hours and you see them for days on end. Bullet points to save myself some typing and you some reading... Novosibirsk The non-existent hostel We arrived at ~0200 local and made our way to the hostel we had booked a room with. Walking to the correct address, we saw several signs but they all turned out to be for a police station and some other state agency. We walked back, forth, double-checked, triple-checked: no hostel. We then walked around the building through some not-quite-nice back alleys, but other than a few entries to private flats, there was nothing. Thankfully, the booking slip included a number which we called and after at least twenty rings (no kidding), when I had given up and wanted to hang up, it stopped ringing. Dead silence. After maybe ten seconds, someone started talking in Russian. I asked him if he spoke English and told him that we could not find the hostel. He mumbled something about being sorry and that we should wait, he would come down. Fast forward a minute or two and someone walked towards us. Again, he mumbled about being sorry, that the hostel "did not work" at the moment and that we would need to sleep in his private apartment. He ushered us into some back alley entrance, into his flat, and proceeded to remove the sheets from the couch on which he had slept; after putting on new sheets, we had our "hostel" bed, ready to sleep on. We briefly considered if he would murder us in our sleep, but him and me even got to talking a bit. Over cheese, sausage and rum (at 0300), he admitted that the hostel did not exist and he merely planned to turn his flat into a hostel for the summer while he and his family moved into their summer house (the Russian term of which escapes me, at the moment) in the countryside. He had accepted our reservation as he thought he would be finished by that time. He did not even get started, though. While he sent us an overbooking notice through booking.com two days before, we were on the train at that time, so... booking.com even called him to check what happenend to us as we did not book another place through them. Good customer service/protection, that. Next morning, he didn't even want to take our money (we paid anyway) and, as a means of compensation, drove us into the city in the morning and to a train museum well outside the city limits, one of the fabled scientist cities, and a large lake which everyone in Novosibirsk claims is an ocean, in the afternoon. Foreigners, foreigners! All in all, Novosibirsk was relatively uneventful, safe for one bizarre episode. We took our lunch in a local fast food joint (why do all the good stories happen there, and not at the various truly local places?) and threw the cashier our well-rehearsed "Niet Russkie; anglisky?" with phrasebook in hand and he actually understood a few words of English (beef, chicken, fries). We told him, in our worst Russian, that we are from Germany wished him a nice day and went to sit down. A few minutes later, a girl approached us, literally hopping from one foot to the other and wringing her hands. She told us that the cashier had told her that we spoke English and if it would be OK if she talked to us. We suspected some sort of elaborate ruse, but went with it. Turns out, she had English at school and really wanted someone to practice English on. Two young men passed our table and exchanged a few words with her, sitting down out of sight. When she told us that she had to leave now but if it would be OK if the two boys joined us we suspected a ruse yet again. But those two were law students, one with a minor in English and one with a minor in German; both of them also extremely nervous, asking us if we would talk to them. When they had to leave, they told us that the three of them worked at the burger joint and that their shift was just about to start when the news that foreigners were here spread amongst staff like wildfire. The girl stopped by several times in between cleaning tables, getting in a sentence or two before being cussed at by her supervisor. All in all, this took about twenty minutes and seeing three people so nervous and grateful to talk with us felt beyond absurd. On the other hand, not a single traveller we met even considered stopping in Novosibirsk during their transit so there really does seem to be a shortage of non-Russians there. Weird, and memorable. Novosibirsk-Irkutsk Irkutsk / Listvianka / Lake Baikal Listvianka Aah, lake Baikal... the oldest and deepest lake on Earth which holds a fifth of the global non-salt water reserves; a must-see in my book. Quad tours at break-neck speeds, dry-suit diving with Russian regulators, walking barefoot in between and across drift ice that made its way onto the shorei, and extended hiking around the lake's coast... All of which I could not do because I was ill and had to spend two solid days in bed. The draft from the open window in between Novosibirsk and Irkutsk was enough to give me a rather bad cold which peaked at Lake Baikal. Still, the area was lovely and we were glad to be out of a train and able to unpack our stuff without having to repack immediately for once. I am not sure where my current losing streak with regards to diving is coming from (Grimsey, diving north of the Arctic circle with birds that plummet into the water and hunt fish: Only guy who does this is on the Icelandic mainland that day; Svalbard, diving north of the Arctic circle in permanent darkness: The few people who do this privately did not reply while I was there; Baikal, oldest, deepest, largest lake on Earth: ill), but I will most likely return to Russia for a week of ice diving in Lake Baikal next winter or the one after that. As an aside, I saw several people walking to Lake Baikal with buckets to get their water. Other people got it from a well which was still half frozen. If you have running water consider yourself lucky... Irkutsk Nice city, largely uneventful. The farther east you get within Russia, the more normal women look. In Moscow, just as in Paris, they are way over-dressed and even service personnel will walk with high heels. Thankfully, I don't have to wear heels, but for the other males out there: Walking and standing in these things hurts and thus most if not all people who stand and walk for a living have flat shoes. We happened upon preparations for a military parade, complete with cordon, viewing podests, at least half a dozen TV cameras etc, but were not sure if it would start soon enough for us to catch our train.We asked someone who told us it would start at 2100 local, at 1945 local it seemed about to start, and sure enough at 1955 sharp, the whole thing went under way. About a dozen groups of 50-100 people each, all in their own, respective uniforms stood against one side of a cordoned-off street and several higher-ups on the other side. Two highest-ups shouted into microphones and the throng of people on the other side shouted back answers. Then, the two highest-ups stood in the back of a jeep each and drove past said throng, stopping in front of each group, shouting into microphones mounted in the back of the jeeps and the groups shouted back once again. After that, all groups marched around the make-shift plaza once, saluting the higher ups. Once they were done, and they took ages, two trucks drove by with soldiers jumping out of the moving trucks and moving into crouching positions. They ran around in a circle a few times and engaged in pretend hand-to-hand combat. I am sure they are skilled at whatever style they wanted to show, but they were overdoing things so badly, they were funny, not imposing. When they jumped over some barriers, the barriers fell to pieces and everyone scrambled to make it look as if that was part of the show. While carrying off the gear, it fell into further pieces which was even more funny. An armoured personnel carrier ended the show; several tougher looking guys jumped off of that one and their mock combat involved fully automatic fire (of blanks), several flashbangs, smoke grenades and, to top things off, the machine gun mounted on the APC moving down the opposing team with blanks. I never witnessed a "real" military parade in person but this one was somewhat disappointing. On the one hand, there was a distinct lack of ballistic missile carriers and tanks like you see in movies, documentaries and games, on the other hand, the whole thing had a make-do feeling to it. The cordoning police had designated spots to stand on, yet walked around. They were standing to attention, yet checking their cell phones. Several people in one uniformed group were wearing track suits and jeans. Another uniformed guy had a grocery bag with him; yet another one was carrying a huge water bottle. Bikers zig-zagged through the cordon and when the whole show was just about to wrap up the police finally started putting up barriers around the unmoving pedestrians, not blocking the bikers. One little girl was standing well within the cordoned area, watching with big eyes and after she did not react to the police talking to her, they just built the barriers in a curve around her. And to top it all off, some guy with a cane walked all through the parade with his personal camcorder, trying to direct the whole show while being ignored by everyone. Still, I am sure he managed to mess up some otherwise perfectly good TV scenes. Irkutsk-Russian border TL;DR 3000 kilometers of birch trees

Pietro Abate: how to create VMs with ganeti / xen and dnsmaq

I'll start here a small series of posts about ganeti, xen and puppet. For my work I run few servers sitting on xen and it has always been a bit of a pain to create a new instance and keep it up to date. Up to now I've used the excellent xen-create-image tool to create my VMs, but I wanted to try something new and more sexy... Last week I finally found some time to learn (and a spare box to run my experiments) how to use ganeti. Ganeti is the only tool I tried out, but it seems to fit the bill for my use and it seems polished and mature project to me... Moreover I've seen a presentation about it in every FLOSS conference I've attended in the last few years and I thought it was time to give it a try. Installing and configuring ganeti is fairly easy, there is a lot of documentation available and this post is not going to be about installing it, but rather how to create a new bare instance with ganeti-deboostrap-instance. There is also a way to create a new instance from an image, but I didn't go that way yet. This first post is about the first problem I've encountered, that is, how to automatically assign a network address and a name to each new instance created by gnt-instance add. Since all my instances should be able to communicate together on the same subnet, I've decided to configure xen to create a NATted private network and add each new instance to this network. The first step is to create an interface in /etc/network/interfaces .
auto xen-br0
iface xen-br0 inet static
address 10.0.0.1
netmask 255.255.255.0
pre-up brctl addbr xen-br0
post-down brctl delbr xen-br0
bridge_stp off
bridge_fd 0
This is the standard debian way but since xen uses a different naming convention (here I'm using ganeti naming convention xenbr0 vs xen-br0), I need to convince tell xen what I intend to do by adding these lines in /etc/xen/xend.config :
(network-script 'network-virtual bridgeip="10.0.0.1/24" brnet="10.0.0.0/24" bridge="xen-br0"')
(vif-script vif-bridge)
Next I have to connect my real network interface to the private network using few iptables rules in /etc/rc.local (probably there is a better place to do this...):
echo 1 > /proc/sys/net/ipv4/ip_forward
/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
/sbin/iptables -A FORWARD -i eth0 -o xen-br0 -m state --state RELATED,ESTABLISHED -j ACCEPT
/sbin/iptables -A FORWARD -i xen-br0 -o eth0 -j ACCEPT
The xen setup is complete and every new image should have a vif connected to the subbet 10.0.0.0. The xen setup corresponds to the physical wiring of the network. The next step is to configure each instance so to allow them to communicate on this subnet. Since I build my VMs using ganeti-debootstrap-instance, and by default debootstrap does not configure the network, we need to add a new hook in the directory /etc/ganeti/instance-debootstrap/hooks.
#!/bin/bash
if [ -z "$TARGET" -o ! -d "$TARGET" ]; then
echo "Missing target directory"
exit 1
fi

if [ ! -d "$TARGET/etc/network" ]; then
echo "Missing target network directory"
exit 1
fi

if [ -z "$NIC_COUNT" ]; then
echo "Missing NIC COUNT"
exit 1
fi

if [ "$NIC_COUNT" -gt 0 ]; then

cat > $TARGET/etc/network/interfaces <<EOF
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

EOF


fi

DAEMON_PID_FILES="/var/run/dnsmasq.pid /var/run/dnsmasq/dnsmasq.pid"

instance=$INSTANCE_NAME
[ -n "$instance" ] exit 1
nic_count=$((NIC_COUNT - 1))
mac_var="NIC_$ nic_count _MAC"
echo $mac_var
echo $nic_count
mac=$ !mac_var
echo $mac
echo "dhcp-host=$mac,$instance" > /etc/dnsmasq.d/$instance.conf
This hook will do two things. First it will configure the interfaces of the new instance to get configured using dhcp, second, it will add an entry to the dnsmasq configuration file to make this instance known to the world. This basically boils down to add a file in /etc/dnsmasq.d/ with the mac address of the new instance and its designated name. Dnsmasq will then provide an ip address for this instance and add it to the dns. dhcp-host=aa:00:00:24:6c:8a,node1 Configuring dnsmasq is pretty easy as well. First I want it to answer dhcp queries only on the internal network, second I want to configure my clients passing 10.0.0.1 as nameserver and gataway. You can just add the following lines in /etc/dnsmasq.d/general to get it going.
interface=xen-br0
interface=lo
dhcp-range=10.0.0.128,10.0.0.250
domain=localnet.org,10.0.0.128,10.0.0.250
dhcp-option=3,10.0.0.1
bogus-priv
#expand-hosts
local=/localnet.org/
To create your new instance you can just run the following command :
gnt-instance add -t plain -s 5g -B memory=1024 -o debootstrap+unstable --no-ip-check --no-name-check node1
I use --no-ip-check and --no-name-check to skip ip and dns check performed by ganeti and to avoid a sort of chicken-egg problem, where the name and address of this new instance is yet unknown to dnsmasq and that node1 is the name that will be used by the hook to add an entry in the dnsmasq configuration. debootstrap+unstable is a variant of the default configuration and you need to add it to the list of variants used by ganeti-deboostrap-instance. That should be it. The new instance should come up with a dynamically assigned ip address, able to talk to the outside world and automatically known by all the other machine on the subnet via dns. Next post will be about how to add a swap hook for ganete-debootstrap-istance.

Pietro Abate: how to create VMs with ganeti / xen and dnsmasq

I'll start here a small series of posts about ganeti, xen and puppet. For my work I run few servers sitting on xen and it has always been a bit of a pain to create a new instance and keep it up to date. Up to now I've used the excellent xen-create-image tool to create my VMs, but I wanted to try something new and more sexy... Last week I finally found some time to learn (and a spare box to run my experiments) how to use ganeti. Ganeti is the only tool I tried out, but it seems to fit the bill for my use and it seems polished and mature project to me... Moreover I've seen a presentation about it in every FLOSS conference I've attended in the last few years and I thought it was time to give it a try. Installing and configuring ganeti is fairly easy, there is a lot of documentation available and this post is not going to be about installing it, but rather how to create a new bare instance with ganeti-deboostrap-instance. There is also a way to create a new instance from an image, but I didn't go that way yet. This first post is about the first problem I've encountered, that is, how to automatically assign a network address and a name to each new instance created by gnt-instance add. Since all my instances should be able to communicate together on the same subnet, I've decided to configure xen to create a NATted private network and add each new instance to this network. The first step is to create an interface in /etc/network/interfaces .
auto xen-br0
iface xen-br0 inet static
address 10.0.0.1
netmask 255.255.255.0
bridge_stp off
bridge_fd 0
bridge_ports none
This is the standard debian way but since xen uses a different naming convention (here I'm using ganeti naming convention xenbr0 vs xen-br0), I need to convince tell xen what I intend to do by adding these lines in /etc/xen/xend.config :
(network-script 'network-virtual bridgeip="10.0.0.1/24" brnet="10.0.0.0/24" bridge="xen-br0"')
(vif-script vif-bridge)
Next I have to connect my real network interface to the private network using few iptables rules in /etc/rc.local (probably there is a better place to do this...):
echo 1 > /proc/sys/net/ipv4/ip_forward
/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
/sbin/iptables -A FORWARD -i eth0 -o xen-br0 -m state --state RELATED,ESTABLISHED -j ACCEPT
/sbin/iptables -A FORWARD -i xen-br0 -o eth0 -j ACCEPT
The xen setup is complete and every new image should have a vif connected to the subbet 10.0.0.0. The xen setup corresponds to the physical wiring of the network. The next step is to configure each instance so to allow them to communicate on this subnet. Since I build my VMs using ganeti-debootstrap-instance, and by default debootstrap does not configure the network, we need to add a new hook in the directory /etc/ganeti/instance-debootstrap/hooks.
#!/bin/bash
if [ -z "$TARGET" -o ! -d "$TARGET" ]; then
echo "Missing target directory"
exit 1
fi

if [ ! -d "$TARGET/etc/network" ]; then
echo "Missing target network directory"
exit 1
fi

if [ -z "$NIC_COUNT" ]; then
echo "Missing NIC COUNT"
exit 1
fi

if [ "$NIC_COUNT" -gt 0 ]; then

cat > $TARGET/etc/network/interfaces <<EOF
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

EOF


fi

DAEMON_PID_FILES="/var/run/dnsmasq.pid /var/run/dnsmasq/dnsmasq.pid"

instance=$INSTANCE_NAME
[ -n "$instance" ] exit 1
nic_count=$((NIC_COUNT - 1))
mac_var="NIC_$ nic_count _MAC"
echo $mac_var
echo $nic_count
mac=$ !mac_var
echo $mac
echo "dhcp-host=$mac,$instance" > /etc/dnsmasq.d/$instance.conf
This hook will do two things. First it will configure the interfaces of the new instance to get configured using dhcp, second, it will add an entry to the dnsmasq configuration file to make this instance known to the world. This basically boils down to add a file in /etc/dnsmasq.d/ with the mac address of the new instance and its designated name. Dnsmasq will then provide an ip address for this instance and add it to the dns. dhcp-host=aa:00:00:24:6c:8a,node1 Configuring dnsmasq is pretty easy as well. First I want it to answer dhcp queries only on the internal network, second I want to configure my clients passing 10.0.0.1 as nameserver and gataway. You can just add the following lines in /etc/dnsmasq.d/general to get it going.
interface=xen-br0
interface=lo
dhcp-range=10.0.0.128,10.0.0.250
domain=localnet.org,10.0.0.128,10.0.0.250
dhcp-option=3,10.0.0.1
bogus-priv
#expand-hosts
local=/localnet.org/
To create your new instance you can just run the following command :
gnt-instance add -t plain -s 5g -B memory=1024 -o debootstrap+unstable --no-ip-check --no-name-check node1
If you are running your dom0 on debian squeeze before running this command you should configure ganeti to pass the right xen parameters to the newly created instance :
gnt-cluster modify --hypervisor-parameter xen-pvm:root_path='/dev/xvda1'
gnt-cluster modify --hypervisor-parameter xen-pvm:initrd_path='/boot/initrd-2.6-xenU'
I use --no-ip-check and --no-name-check to skip ip and dns check performed by ganeti and to avoid a sort of chicken-egg problem, where the name and address of this new instance is yet unknown to dnsmasq and that node1 is the name that will be used by the hook to add an entry in the dnsmasq configuration. debootstrap+unstable is a variant of the default configuration and you need to add it to the list of variants used by ganeti-deboostrap-instance. That should be it. The new instance should come up with a dynamically assigned ip address, able to talk to the outside world and automatically known by all the other machine on the subnet via dns. Next post will be about how to add a swap hook for ganete-debootstrap-istance.

15 May 2012

John Goerzen: Suspicious Blog Activity any advice?

I ve been noticing a number of odd things happening surrounding my blog lately, and I thought it s about time to figure out what s going on and how to stop it. The first problem is that people are illegally copying my posts, probably using RSS scraping, and putting them up on their own ad-infested sites. It is trivial to find them using Google for any somewhat unique word or phrase in one of my posts. Lately one of them, linux-support.com, actually sends me pingbacks announcing the fact that they ve scraped me! Most of these sites seem to be nothing but content farms for selling ad impressions, and almost none of them have any identifiable names for the owners. (There is an exception: I have specifically set up sites like Planet Debian and Goodreads to copy my blog posts.) I m obviously an advocate of open content, but I do not feel it right that others should be profiting by putting photos and stories about Free Software, or photos of my family, on their ad farms. While I release a great deal of content under GPL or Creative Commons licenses, I have never done so with my blog an intentional decision. What should I do about this? Is it worth fighting a battle over, or is it about as useless as trying to block every spam follower on my twitter account? So that s the first weird thing. The second weird thing just started within the last few weeks. I have been getting a surprising amount (a few a week) of email addressed to me. It does not bear the appearance of being 100% automated spam, though it is possible that it is. It s taken a few forms: The profit motive in all of these is high, and in at least the second and third, so is the sleaze factor. I ve gotten two emails lately of this form:
Hi John, I am curious if you are the administrator for this site: changelog.complete.org/archives/174-house-outlaws-fast-forwarding-senate-pres-next I am a researcher / writer involved with a new project whose mission it is to provide accurate and useful information for those interested in the practice of law, whether as a lawyer or paralegal. I recently produced an article detailing the complex relationship between law and technology and the legal implications on personal privacy and free speech. I would love to share this resource with those who might find it useful and am curious of you are the correct person to contact about such a request? Thank you! All my best,
The details vary the URLs appear to be random (the one cited above was little more than a link to an article), the topics the website claims to discuss range from law to schizophrenia (that one actually came with a link to the site, which again seemed to be a content farm). I am slightly tempted to reply to one of these and ask where the heck people are getting my name. It seems as if somebody has put me into a mailing list they sell containing sleazebag bloggers. Frankly, I am puzzled at this attention. I guess I haven t checked, but I can t imagine that my blog has anything even remotely resembling a high PageRank or anything else. It s not high-traffic, not Slashdot, etc. Either people are desperate, naive, failing to be selective, or maybe working some scam on me that I don t know yet. In any case, I m interested if others have seen this, or any advice you might have.

14 May 2012

Benjamin Mako Hill: Date Arithmetic

When I set an alarm, my clock, now running on the computer in my pocket, is smart enough to tell me how much time will pass until the alarm is scheduled to sound. This has eliminated the old problem of sleeping past meetings before being surprised by an alarm precisely half a day after I had originally planned to wake. The price has been having to know exactly how little I will sleep: a usually depressing fact that had previously been obscured by my difficulty doing time arithmetic in my most somnolent moments.

13 May 2012

Jamie McClelland: Sparkleshare

Sparkleshare is a file sharing utility that keeps folders in sync on multiple computers in a similar fashion to the commercial and proprietary Dropbox. Sparkleshare recently released a Windows client, finally allowing me to start switching my co-workers at the Progressive Technology Project away from Dropbox. Overall, I'm very impressed. In particular, I appreciate the Sparkleshare authors' decision to build on top of existing tools (git for storage and revision control, ssh for transport, and ssh public key infrastructure for authorization and authentication). That means I don't have to learn new tools and protocols to debug and it means Sparkleshare can focus on the file sharing pieces. Despite my overall enthusiasm, I do have some serious concerns. Protecting your credentials A researcher found a startling security flaw in Dropbox - if you can copy a particular file from a user's computer to your own computer, you can impersonate them flawlessly, accessing all their Dropbox files without having to know the user's password. Unfortunately, Sparkleshare suffers from this same vulnerability if you use the default configuration. When you first install Sparkleshare, it creates a password-less ssh private/public key pair and then makes the public part easily accessible to you so you can add it to your server (or gitorious account, etc). Very convenient. But it also means that all an attacker needs to do is copy your private key (and your Sparkleshare configuration file) and they get complete control over your files. This problem is easily avoided. If you are running Linux or Mac OS X and you have your own key loaded in your ssh agent, Sparkleshare will happily use that key. So - simply by using your existing (presumably password-protected key) on your remote Sparkleshare servers, you can mitigate this problem. Sparkleshare will still load your Sparkleshare generated key, but if you don't provide that key with access to anything, no harm is done. Confirming each use of your key However... that leads to a new problem. If you are like me, your ssh agent is configured to ask for a confirmation every time your key is used. And, Sparkleshare regularly polls the remote git repository for changes. At best repeatedly clicking to confirm is tedious. At worst, it prevents you from intelligently rejecting malicious requests, thus defeating the whole purpose of the check. It is possible to launch Sparkleshare via ssh-agent in an environment in which you are not requiring the confirmation when Sparkleshare uses your key, but still requiring it for all other uses. However, given the trade offs, I've decided to add a password to my Sparkleshare provided ssh key rather than using my existing key: ssh-keygen -p -f ~/.config/sparkleshare/sparkleshare.jamie@progressivetech.org.key Now, I am prompted to enter my passphrase when I start Sparkleshare and don't have to confirm every use of the key. And, I continue to confirm each use of my main key. git was designed to store code, not documents Just because something is designed for one purpose doesn't mean it can't be used for another. However, there are a few limitations. Large files are one. git can handle files over 100MB, but may run into memory problems. I had to alter the git windowMemory setting, raising it higher than the size of the largest file. Another problem is disk space. Since git keeps full revision history on every machine, you have to download more data than just the files that are checked out. The more edits you make to a repository, the more disk space beyond what is checked out is needed (and deleting files won't help). Lastly, you can't use file modification times. With git, the file modification time will depend on when you checkout the files. With some fancy hook writing, you could tweak things so that the file modification date is the same as the commit date, but that still won't help you if you add an existing directory to Sparkleshare because all the files will have the same commit time. This "bug" has been reported to git and it has been rejected because mucking with the modification time of files can have bad results when you are using make to compile code. As Linus colorfully put it:
I'm sorry. If you don't see how it's WRONG to set a datestamp back to something that will make a simple "make" miscompile your source tree, 
I don't know what definition of "wrong" you are talking about.
It's WRONG.
It's STUPID.
And it's totally INFEASIBLE to implement.
Well, did I mention that git was designed for source code? No server validation Of all the problems I encountered, this one is by far the most serious. Perusing ~/.config/sparkleshare/debug.log is very informative. On a default installation, after you have added a project, you'll see:
19:32:27 [Fetcher][/home/jamie/SparkleShare/.tmp/bar] Fetching folder: ssh://foo@bar.org/foo/bar
19:32:27 [Fetcher] Disabled host key checking for bar.org
Wah. Once the project is added, whatever key fingerprint is given is stuffed into your ~/.ssh/known_hosts file. Host key checking is only disabled when you initially create the project, so if you connect to the proper server the first time, you are protected from subsequent man-in-the-middle attacks. However... if the initial fingerprint is wrong, your personal ssh configuration is now poisoned. This seems like a very bad idea. I've opened a debian bug to address it.

Matthew Palmer: vmdksync helps you escape from VMware

When I wrote lvmsync late last year, I didn t realise I was being typecast. Before too long, I realised that the logic that I d implemented for lvmsync would also help me with a separate migration project I d been dreading getting the day job off VMware. Back in the early days of virtualisation, management made the decision to run VMware, for all the usual reasons ( commercially supported! , industry standard! , and so on). Unsurprisingly (to me, anyway) it didn t take too long for management to realise that it wasn t the best choice for us. When you ve got umpty-billion dollars to spend on hardware, software, and support, VMware might be the right option (although Amazon doesn t seem to think so). Anchor s company culture, on the other hand, is build around smart staff, simple systems over dumb staff, smart vendors , because no vendor is ever going to care about our customers as much as we do. So VMware was never going to work for us. Unfortunately, as happens all too often, once VMware was in place, there was very little motivation to get rid of it and move those customers onto the chosen replacement (that we were deploying all new customers on). I happen to think this is a terrible attitude in general one that makes life so much harder in the long term. I believe strongly in retrofitting old systems to keep them up-to-date with the current state of the art, and keeping technical debt under control. But, I wasn t running the show back when we stopped putting new customers on VMware, so the few VMware servers we had stayed around far longer than they should have. Recently, though, bad things started to happen. The VMware servers were starting to fall apart. The Windows machine we had to keep around to use the VMware management console started crapping out, and when the choice was between doing unspeakable things to Windows, and just ditching VMware well, it wasn t much of a choice. The only remaining question was how to do the migration off VMware with the least amount of downtime to our customers. I was really quite surprised that nobody out in Internet land appeared to have come up with a simple, robust tool to do this. Sure, some vendors had all-singing, all-dancing toolkits that cost ridiculous amounts of money, required you to install their agent on the machine involved, and promised the earth, but it all smelt of snakeoil and bullshit. In true hacker style, then, I decided to write something myself. The model I came up with mirrored lvmsync s quite closely because that one worked, and it turned out to be surprisingly easy to implement once I managed to reverse-engineer the file format (VMware has a PDF spec of a bunch of it s file formats, but whoever wrote it was enough of an evil genius to make it utterly incomprehensible to anyone who doesn t already know the file format, whilst making perfect sense to anyone who already does). The result: vmdksync. It is nothing but 80-odd lines of ruby whose sole purpose is to take a delta.vmdk file and write the changes that are stored in that file to a file or block device that is a copy of the flat.vmdk file that you can copy while the VM is still running (after you ve made a snapshot, of course). It helped me provide a painless migration path away from VMware, and I d be really pleased if it helped some other people do the same. Share and enjoy!

11 May 2012

Andrew Pollock: [life] Zoe at 24 months

Once again, another 3 months have flown by. Honestly, it's been a total blur. What on earth has happened since February? Well, heaps more travel of course. There was the trip to New Orleans, which I already wrote about. The other big trip we did was to Washington D.C. for a week, which I'm yet to write up. We stayed with friends in Alexandria for the week. Unfortunately, Zoe came down with conjunctivitis before Sarah's very eyes on the flight over (I'd been in North Carolina for work the week before, so they flew without me and I flew up to D.C. to meet them). She also developed an ear infection and a general cough/cold. Fortunately, we caught it all pretty much as soon as it happened, and spent Easter Sunday at an Urgent Care in Virginia getting antibiotics for her, but her sleep was not so great the whole time we were in D.C. Other than that, we've just been doing the normal stuff. Swimming classes have continued (she's very confident in the water now). Her speech continues developing well. Toilet training is continuing to go very well. There haven't been very many accidents at all. She's really loving the "new" day care. It was definitely the right thing to do to move her. Physically, she's in the 80th percentile for height and the 50th percentile for weight, so I guess she's currently tracking to be on the tall and thin side of things. We had another birthday party in the park behind our home, and this year the weather was much nicer, so we had people hanging around until about 7pm, and a good time was had by all. Zoe even successfully blew out her candle on the first attempt. Her two-year molars still haven't come through. They seem to go through various stages of giving her a lot of grief, and then they stop messing with her sleep. I was beginning to think we'd never get back to an uninterrupted night's sleep ever again. It's still a bit hit and miss, but she usually settles down again fairly quickly if she does wake up, and doesn't always require intervention from us. I've arrived at the theory that the "terrible twos" are largely the fault of trying to cut these two year molars. Her tantrums are so much worse when her teeth are actively giving her trouble. If she's had a good night's sleep and her teeth aren't driving her crazy, she's still pretty well behaved. If she's been awake half the night and her teeth are hurting during the day, she can be quite difficult. We seem to be currently at a point where they're not causing too much trouble at night, but she'll chew on her fingers like crazy if given the chance during the day. I think seasonal allergies are also messing with her a bit. After she got over the cold she came down with in Washington D.C., she's still had a runny nose and a bit of a cough, but I don't think she's actually "sick" I think it's more related to allergies. She passed on whatever she had to me, and since I've recovered from it I'm also not feeling quite right. I read somewhere that pollen counts are off the charts this year due to the weird weather, and Sarah's having a bit of a hard time as well, so I think we're all going to just have to ride it out with the judicious use of antihistamines. Speaking of allergies, it's about time to re-test her for her egg allergy. We'll have to try that at a point when she's not regularly taking antihistamines for seasonal allergies, so it might be a while yet before we can do that. I guess the next big development will be graduating from the crib to a toddler bed. I'm not in any hurry to do that though, so as long as she's not escaping the crib, her sleeping arrangements can stay the way they are. It'd be nice if she can cut her teeth first and then once she's sleeping properly, we can try converting the crib into a toddler bed. Zoe walking down Castro Street in Mountain View

10 May 2012

Lars Wirzenius: Obnam 0.28 (backup program)

I've just released version 0.28 of Obnam, my backup application. The relevant part of NEWS:

DebConf team: Submit your greatest ideas as a DebConf12 talk (read: Call for Papers reminder) (Posted by Gunnar Wolf)

DebConf12 is drawing closer and closer! Have you ever been to a DebConf? You will surely agree with us: It s a life-changing event! It s one of the greatest geek gatherings in the world! It s something you can t afford on missing out! It s wait.. Oh, sorry, I had my spam mode set to highest. But still, that does not make it any less true. And one third of May has already passed. This means that the first deadlines are just around the corner: next Tuesday (15-may), sponsored registration will close, so make sure to register before it s too late. But wait wasn t this about the call for papers? Right, that s not the point of my post, I wandered off again. I m sorry, I just visited Managua last weekend (Holger will soon tell you about our visit), and it s impossible to keep the mind off the excitement. I wanted to ask you to share your greatest, latest ideas or projects. DebConf is about many things But one of its most important features is its rich set of talks, usually with very high and interesting contents in the technical, social, organizational levels Or several of them at once. As always, you will be able to submit talks until a day before you present it. However, it s best for your intended audience to know bits about your proposed topic, possibly even to share some information beforehand, so that the exposition can be as smooth as possible and the interaction during the talk even more And this is specially true for BoF sessions. However, we do have one advertised deadline: Talks submitted up to June 1st will be considered to be pre-scheduled. This means, to be part of the officially announced program, and have guaranteed coverage of our (always great) video team. DebConf Tracks But maybe you don t feel comfortable presenting a topic. Or you are, but you think it would be important to have several thematically related talks be grouped together. And if you are interested in this group of talks, you will probably know who would be more interesting to invite to present the topics - And will be able to prod them to do so. You can request to coordinate a track. That will ensure you get a contiguous block of timeslots for the talks you schedule, and do a great contribution to DebConf. To add your track, please mail talks@debconf.org. Please, specially if you do so in the few days following May 9, send me a copy to gwolf@gwolf.org, as it seems the alias is not yet set up. Tutorials and mentoring This year, we will have a high number of very motivated attendees from the many countries in the Central American region who want to get started in helping Debian, but have not yet found how to do it. Usually we would see local communities as prime material for DebianDay However, many of them are more interested in getting hands-on tutorials and be mentored into contributing to Debian. So, if you are part of a team working on a given aspect of Debian, or want to get people involved into your project, register it in the Tutorials and mentoring track. See you all soon, very soon, in tropical Managua for the hottest DebConf ever! (oh For those who remember days long past in Helsinki: There is even a sauna! ;-) And it s GREAT)

6 May 2012

Stefano Zacchiroli: overlapping bits from the DPL for April 2012

Just posted to d-d-a, here is the monthly report about my DPL activities.
Dear project members,
last bits of the past DPL term and first bits of the current term, all in one. Here is a report of what has happened in DPL land last April. Highlight: call for DPL helpers Before the report, though, let me point out that your friendly neighborhood DPL could use some help. As discussed during campaign, there are some intrinsic transparency and scalability limits in the DPL institution, when run by a single person. Before trying something new to fix that, I'd like to give a last try to an old "tactic": calling for help. If you're considering running for DPL or if you're simply interested in the job the DPL does and willing to help with that, please let me know. Ideally, if I find a group of people I'm happy to work with, I'd like to set up periodic IRC meetings with all "DPL helpers" to publicly discuss items in the DPL agenda and share the work-load. Ongoing discussions A big topic of last month has been the proposal by Francesca Ciceri to publish a diversity statement for the Debian Project. After a lively discussion on -project, we reached consensus on a text, and I've been happy to help with that. To finalize statement publication we now need to vote on it with a GR. I've helped drafting a corresponding GR proposal that has already been posted to -vote by Francesca. A final one, looking for seconds, will be posted there soon. Wrapping up March discussions on a revenue sharing agreement with DuckDuckGo, I've announced my intention to finalize the agreement and have done so shortly thereafter. The Iceweasel maintainer has deployed the corresponding search engine query string and other web browser maintainers could do the same, if they want to. In April I've also spent some time to move forward the long running conflict on Python maintenance, reported to the tech-ctte more than 2 years ago. With the help of people on the -python mailing list, I've now submitted to the tech-ctte an up to date list of potential maintenance teams. I hope the tech-ctte now have all the information needed to come to a decision. Speaking of which, I'm also discussing with tech-ctte members the possibility of having periodic ctte meetings; the idea is to ensure that outstanding issues are periodically reassessed, improving the reliability of tech-ctte decision times. I've also discussed at length with members of the pkg-multimedia-maintainers team the relationships with the unofficial debian-multimedia.org (d-m.o) repository, that have been a cause of tension for Debian multimedia users and maintainers for quite some time. On behalf of the team and of the Project I've now reached out to the d-m.o maintainer, hoping to come to some sort of amicable agreement on which packages belong where. Hardware replacement As anticipated in last report, I've started approving hardware purchases to implement the yearly hardware replacement plan prepared by DSA. During April I've approved requests to buy servers to replace the machines running the bugs-master, bugs-mirror, and UDD services. The total expected expenditure is about 15'000 USD. Communication I've delivered my classic Debian "18^W 19 years" talk at UNIVPM, a polytechnic university in center Italy; slides are available. I've then been contacted by people from the European Synchrotoron in Grenoble who, beside having recently migrated their infrastructure to Debian, are looking into organizing a workshop on Debian usage for large science facilities. I've been happy to help out providing a list of potential topics and speakers for the event. Also as anticipated last month, the Debian Project has been present at the OpenStack summit. Loic Dachary has represented Debian at the event and provided a nice report about his experience there. Speaking of which, I've also coordinated a news release about the availability of cloud technologies in Wheezy, taking the chance to point out the relationships between what Debian stands for and the ability to deploy your own private cloud. Sprints April has been a rather calm month on the sprint front, with the notable exception of the I18n team who is organizing a sprint for June in Paris. Miscellanea Thanks for reading thus far,
HDH! (Happy Debian Hacking)
PS the boring day-to-day activity log for April is available at master:/srv/leader/news/bits-from-the-DPL.txt.201204

3 May 2012

Wouter Verhelst: Debconf

... is drawing closer again, and today I've booked my flights. I'll be arriving in the early evening on the 1st, and leaving again on the 15th. The flights will be Brussels->London->Miami->Managua, which unfortunately means I'll have to deal with US border control, but hey. Also, in order to avoid having to get up at early-o-clock on the morning of the 1st, I'll be flying from Brussels to London on an afternoon flight on the 30th, which means I'll have an evening to spare in London. While I've been there a few times before, it's not a city that I've ever actually visited; my previous three visits were limited to Heathrow, King's Cross (IIRC; could be another train station) and driving through the city on my way from (and to) Steve's barbecue last summer. If anyone will be there as well that night and would like to meet for beer and/or keysigning, I'd be interested.

2 May 2012

Matthew Garrett: Anatomy of a Fedora 17 ISO image

My spare time has been limited lately, so progress on producing Mac-bootable Fedora install images has been a bit slow. Thankfully it looks like everything's gong to land in time for Fedora 17[1] so hurrah for that - all that's missing right now are the last couple of patches that make sure the boot picker displays useful labels on the drives.

So how is this accomplished? Here's a hex dump of the first few K of the image.
00000000  45 52 08 00 00 00 90 90  00 00 00 00 00 00 00 00   ER.............. 
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00000020  33 ed fa 8e d5 bc 00 7c  fb fc 66 31 db 66 31 c9   3...... ..f1.f1. 
00000030  66 53 66 51 06 57 8e dd  8e c5 52 be 00 7c bf 00   fSfQ.W....R.. .. 
00000040  06 b9 00 01 f3 a5 ea 4b  06 00 00 52 b4 41 bb aa   .......K...R.A.. 
00000050  55 31 c9 30 f6 f9 cd 13  72 16 81 fb 55 aa 75 10   U1.0....r...U.u. 
00000060  83 e1 01 74 0b 66 c7 06  f1 06 b4 42 eb 15 eb 00   ...t.f.....B.... 
00000070  5a 51 b4 08 cd 13 83 e1  3f 5b 51 0f b6 c6 40 50   ZQ......?[Q...@P 
00000080  f7 e1 53 52 50 bb 00 7c  b9 04 00 66 a1 b0 07 e8   ..SRP.. ...f.... 
00000090  44 00 0f 82 80 00 66 40  80 c7 02 e2 f2 66 81 3e   D.....f@.....f.> 
000000a0  40 7c fb c0 78 70 75 09  fa bc ec 7b ea 44 7c 00   @ ..xpu.... .D . 
000000b0  00 e8 83 00 69 73 6f 6c  69 6e 75 78 2e 62 69 6e   ....isolinux.bin 
000000c0  20 6d 69 73 73 69 6e 67  20 6f 72 20 63 6f 72 72    missing or corr 
000000d0  75 70 74 2e 0d 0a 66 60  66 31 d2 66 03 06 f8 7b   upt...f f1.f... 
000000e0  66 13 16 fc 7b 66 52 66  50 06 53 6a 01 6a 10 89   f... fRfP.Sj.j.. 
000000f0  e6 66 f7 36 e8 7b c0 e4  06 88 e1 88 c5 92 f6 36   .f.6. .........6 
00000100  ee 7b 88 c6 08 e1 41 b8  01 02 8a 16 f2 7b cd 13   . ....A...... .. 
00000110  8d 64 10 66 61 c3 e8 1e  00 4f 70 65 72 61 74 69   .d.fa....Operati 
00000120  6e 67 20 73 79 73 74 65  6d 20 6c 6f 61 64 20 65   ng system load e 
00000130  72 72 6f 72 2e 0d 0a 5e  ac b4 0e 8a 3e 62 04 b3   rror...^....>b.. 
00000140  07 cd 10 3c 0a 75 f1 cd  18 f4 eb fd 00 00 00 00   ...<.u.......... 
00000150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
This is the boot sector. One of the fun things about ISO9660 is that it doesn't have a boot sector in the usual x86 sense - firmwares are expected to read enough of the filesystem that they can find the bootloader in it and run it directly. x86 isn't really smart enough to manage that, so it uses something called El Torito[2]. In any case, this is usually empty space on an x86 CD. But with an increasing number of systems shipping without optical drives, real CD installs are decreasing in popularity. We use a piece of software called isohybrid that allows the building of ISO9660 images that can be written directly onto a USB stick. Asking a machine to boot off them will execute the boot sector located here, which then loads a real bootloader that's capable of reading ISO and booting the OS. It'll just be ignored when it's on a real CD.

There's one difference here, though. The isohybrid bootsector has been modified so that the first 32 bytes are just noops, and this has been inserted:
00000000  45 52 08 00 00 00 90 90  00 00 00 00 00 00 00 00   ER.............. 
45 52 ("ER") indicates the presence of an Apple partition map. This will become important later. 0800 indicates that it's using 2048-byte sectors. The 9090 is a lie to convince the firmware that it's not a zero-sized disk without representing dangerous code. We need an Apple partition map because some older Macs won't boot off CD unless the CD has one. But it's sitting right at the start of the boot sector, and this code is going to be executed. Thankfully, the entire Apple header decodes to
00000000  45                inc bp
00000001  52                push dx
00000002  0800              or [bx+si],al
00000004  0000              add [bx+si],al
00000006  90                nop
00000007  90                nop
00000008  0000              add [bx+si],al
0000000A  0000              add [bx+si],al
0000000C  0000              add [bx+si],al
0000000E  0000              add [bx+si],al
00000010  0000              add [bx+si],al
00000012  0000              add [bx+si],al
00000014  0000              add [bx+si],al
00000016  0000              add [bx+si],al
00000018  0000              add [bx+si],al
0000001A  0000              add [bx+si],al
0000001C  0000              add [bx+si],al
0000001E  0000              add [bx+si],al
which is completely harmless - we can execute all these instructions without any interesting changes in state. They'll all be undone by the rest of the boot sector.
000001b0  14 05 00 00 00 00 00 00  4c 1e ed 76 00 00 80 00   ........L..v.... 
000001c0  01 00 00 3f a0 89 00 00  00 00 00 50 14 00 00 fe   ...?.......P.... 
000001d0  ff ff ef fe ff ff a4 00  00 00 70 04 00 00 00 fe   ..........p..... 
000001e0  ff ff 00 fe ff ff 44 05  00 00 c0 08 00 00 00 00   ......D......... 
000001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa   ..............U. 
This is the MBR partition table. The aim here was to produce media that would boot via BIOS just as well as it does via EFI, and it turns out that there are some systems that refuse to BIOS-boot off GPT media. So we need an MBR partition map. The first entry covers the entire disk and is of type 0. This is important, because some EFI implementations are strict about MBR parsing - if there's an MBR with overlapping partitions, the entire MBR will be ignored. Inconvenient. Fortunately, we've got the source code to this code, and it turns out that partitions of type 0 are ignored when performing this check. So, there's a partition of type 0.

Why does it cover the entire disk? Once we've booted the OS, we need to be able to read the ISO image contained on the USB stick. That means the kernel needs to be able to mount it, which means we need a partition entry that covers the entire disk. Linux is perfectly happy mounting a filesystem from a partition of type 0.

So, the next partition. This is an EFI boot partition that points at the embedded VFAT El-Torito image. Some firmware will decide that we've got an MBR and so will only boot off a partition that exists in it. So, here's an MBR partition to keep them happy.

The final partition covers the embedded HFS+ image. It's there purely for convenience - it means we can access it under Linux in order to update its contents for testing purposes.
00000200  45 46 49 20 50 41 52 54  00 00 01 00 5c 00 00 00   EFI PART....\... 
00000210  09 27 93 7e 00 00 00 00  01 00 00 00 00 00 00 00   .'.~............ 
00000220  fe 4f 14 00 00 00 00 00  30 00 00 00 00 00 00 00   .O......0....... 
00000230  de 4f 14 00 00 00 00 00  2a 26 0f 37 23 c9 7f 4f   .O......*&.7#..O 
00000240  90 df 5e fe 0a 60 1c b0  10 00 00 00 00 00 00 00   ..^.. .......... 
00000250  80 00 00 00 80 00 00 00  ec 70 63 c1 00 00 00 00   .........pc..... 
00000260  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
And here we have the GPT header. Nothing complicated here, other than it indicates that the first partition descriptor is at LBA 0x10, ie byte 0x2000 - that's further into the disk than normal. That leaves us enough space to fit the Apple partition entries.

Why have a GPT at all? The EFI spec says that machines should boot fine from an MBR partition. Sadly, not all seem to. It also lets us represent the HFS+ partition in a way that makes the Mac firmware happy.
00000800  50 4d 00 00 00 00 00 03  00 00 00 01 00 00 00 10   PM.............. 
00000810  41 70 70 6c 65 00 00 00  00 00 00 00 00 00 00 00   Apple........... 
00000820  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00000830  41 70 70 6c 65 5f 70 61  72 74 69 74 69 6f 6e 5f   Apple_partition_ 
00000840  6d 61 70 00 00 00 00 00  00 00 00 00 00 00 00 00   map............. 
00000850  00 00 00 00 00 00 00 0a  00 00 00 03 00 00 00 00   ................ 
00000860  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
2K into the disk, here's an Apple partition map. It's 2K into the disk because we set the sector size to 2K in the partition map header. Why did we do that? Because the default is 512 bytes, and if we'd put the partition map at 512 bytes it would have been on top of the GPT header. Why 2K rather than 1K? A couple of reasons. First, the Apple partition map is only used when we boot off an actual CD, and the CD sector size is genuinely 2K. Secondly, because CDs have a sector size of 2K, 2K is a value that's actually been tested in the real world. It's nice to avoid tempting fate.
00001000  50 4d 00 00 00 00 00 03  00 00 00 29 00 00 04 70   PM.........)...p 
00001010  45 46 49 00 00 00 00 00  00 00 00 00 00 00 00 00   EFI............. 
00001020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00001030  41 70 70 6c 65 5f 48 46  53 00 00 00 00 00 00 00   Apple_HFS....... 
00001040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00001050  00 00 00 00 00 00 04 70  00 00 00 33 00 00 00 00   .......p...3.... 
00001060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
*
00001800  50 4d 00 00 00 00 00 03  00 00 01 51 00 00 08 c0   PM.........Q.... 
00001810  45 46 49 00 00 00 00 00  00 00 00 00 00 00 00 00   EFI............. 
00001820  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00001830  41 70 70 6c 65 5f 48 46  53 00 00 00 00 00 00 00   Apple_HFS....... 
00001840  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00001850  00 00 00 00 00 00 08 c0  00 00 00 33 00 00 00 00   ...........3.... 
00001860  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
The Apple partition map entries. One partition covers the entire disk, the other the embedded HFS+ filesystem. Why do this? Because not all Macs understand EFI El Torito, so won't EFI boot a CD unless there's an Apple partition map. Why have an HFS+ partition at all? Because the same Macs won't boot off FAT.
00002000  a2 a0 d0 eb e5 b9 33 44  87 c0 68 b6 b7 26 99 c7   ......3D..h..&.. 
00002010  d0 84 a5 d4 aa 89 e0 40  81 f9 fc e4 04 e9 c2 99   .......@........ 
00002020  00 00 00 00 00 00 00 00  10 4b 14 00 00 00 00 00   .........K...... 
00002030  00 00 00 00 00 00 00 00  49 53 4f 48 79 62 72 69   ........ISOHybri 
00002040  64 20 49 53 4f 00 49 53  4f 48 79 62 72 69 64 00   d ISO.ISOHybrid. 
00002050  41 70 70 6c 00 00 00 00  00 00 00 00 00 00 00 00   Appl............ 
00002060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
*
00002080  a2 a0 d0 eb e5 b9 33 44  87 c0 68 b6 b7 26 99 c7   ......3D..h..&.. 
00002090  68 d5 e0 27 72 83 c6 4c  a7 ae 92 dd ed 6c 89 71   h..'r..L.....l.q 
000020a0  a4 00 00 00 00 00 00 00  13 05 00 00 00 00 00 00   ................ 
000020b0  00 00 00 00 00 00 00 00  49 53 4f 48 79 62 72 69   ........ISOHybri 
000020c0  64 00 41 70 70 6c 65 00  41 70 70 6c 00 00 00 00   d.Apple.Appl.... 
000020d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
*
00002100  00 53 46 48 00 00 aa 11  aa 11 00 30 65 43 ec ac   .SFH.......0eC.. 
00002110  68 d5 e0 27 72 83 c6 4c  a7 ae 92 dd ed 6c 89 71   h..'r..L.....l.q 
00002120  44 05 00 00 00 00 00 00  03 0e 00 00 00 00 00 00   D............... 
00002130  00 00 00 00 00 00 00 00  49 53 4f 48 79 62 72 69   ........ISOHybri 
00002140  64 00 41 70 70 6c 65 00  41 70 70 6c 00 00 00 00   d.Apple.Appl.... 
00002150  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
Three GPT entries - one covers the entire CD, one covers the embedded FAT filesystem, one covers the embedded EFI filesystem. That ensures that they're all accessible to the firmware.
00008000  01 43 44 30 30 31 01 00  4c 49 4e 55 58 20 20 20   .CD001..LINUX    
00008010  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20                    
00008020  20 20 20 20 20 20 20 20  46 65 64 6f 72 61 2d 4c           Fedora-L 
00008030  69 76 65 43 44 20 20 20  20 20 20 20 20 20 20 20   iveCD            
00008040  20 20 20 20 20 20 20 20  00 00 00 00 00 00 00 00           ........ 
00008050  c4 12 05 00 00 05 12 c4  00 00 00 00 00 00 00 00   ................ 
00008060  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00   ................ 
00008070  00 00 00 00 00 00 00 00  01 00 00 01 01 00 00 01   ................ 
00008080  00 08 08 00 40 00 00 00  00 00 00 40 15 00 00 00   ....@......@.... 
00008090  00 00 00 00 00 00 00 17  00 00 00 00 22 00 1d 00   ............"... 
000080a0  00 00 00 00 00 1d 00 08  00 00 00 00 08 00 70 05   ..............p. 
000080b0  01 09 30 36 f0 02 00 00  01 00 00 01 01 00 20 20   ..06..........   
000080c0  20 20 20 20 20 20 20 20  20 20 20 20 20 20 20 20                    
The ISO9660 superblock. This is completely standard. From here on there's nothing terribly surprising - the CD filesystem itself is entirely normal. The only slight oddity is that we have three embedded El Torito images. The first of these is a BIOS-bootable disk image. That'll be executed whenever the CD is put in a non-EFI machine. The second is a VFAT EFI boot image. That's used for the majority of EFI systems. The third is an HFS+ EFI boot image. There's no strict requirement for the HFS+ one to be an El Torito image - CD booting on Macs is already handled by the Apple partition map. Storing it as an El Torito is purely for convenience, since it guarantees correct alignment and avoids us having to parse the entire ISO9660 image to find its location when generating the partition maps.

In summary: Three partition maps, three bootable images, support for BIOS, UEFI and Mac platforms, works whether it's burned to a CD or written directly to a USB stick. I'm pretty pleased with that.

[1] Other than an awkward bug where something goes horribly wrong during Radeon graphics init, so Radeon-based Macs aren't working too well at the moment. We're working on it.

[2] The world decided that writing a real ISO9660 driver for BIOS was hard, so came up with El Torito. El Torito is a spec for embedding disk images inside ISO9660. There's a pointer to the disk image in the Cd header, so the BIOS simply reads a few blocks off CD, finds that pointer and then sets up a bunch of disk i/o interrupts to point at the embedded filesystem rather than a real disk. From that point on, the El Torito image simply behaves like a floppy or hard drive. This was a reasonable compromise given how resource limited older BIOS implementations were.

For reasons that aren't entirely clear, UEFI doesn't mandate that implementations support ISO9660. So even though our firmware is now sufficiently capable that the only thing standing between it and emacs is someone being sufficiently bored, we still use El Torito. Fortunately the spec allows multiple El Torito images to be embedded, so we can include one that's bootable by BIOS systems and another that's bootable by UEFI systems.

comment count unavailable comments

Next.

Previous.