I'm still lacking the obligatory post-DebConf7 post... These weeks had been intense... So, first of all, thanks to all the DC-team for another wonderful conference. Next year I'll feel how it's done one of the two things that everybody likes but nobody wants to know how it's done. Hint: not sausages. This is the first time I've also attended DebCamp, and it was very useful. Not that I could do any real work, as very few people are able to do that during DebConf, but I managed to do what I planned, that is talk with many people who are working on debian-cd, debian-installer and custom-cdds. I've already uploaded my pictures to a temporary place, whitout captions, until I decide how I will replace the stinko called gallery. And there you can see that I was one of the few that actually wore a kilt during the event —not a proper Debian kilt, but hey... After DebConf, and after a year of hard work, I took a seven days vacation in the UK. So I was able to know Edinburgh (I prefer to hang out with Debian people that doing tourism during the conference, so I only knew the way to the supermarket). I also went to the Arthur's Seat HIll, which was very beautiful, and was only a 1-hour walk from the hostel to the top. Then I did a small visit to Stirling and concluded that castles in Scotland are not that interesting (but go to the Smith Museum!). Then, I went to London for a couple of days (dang, that was expensive!!), visited Marx's tomb, rided near the car bombs, got to know all the nice stuff that the british stole from around the world in the British Museum (it's really great, and free), and went to Cambridge to spend a day (and punting, I even did the punting myself!). Finally I spent my last day walking around that beatiful city, in the Soho and South Bank areas. I don't want to forget to say thanks to Nattie, Kai and Ben for hosting me, you guys rock!! This last week has been very productive. I was working GSoC project, trying to recover from lateness, and staying until late. It was worth it, I have finished settling down the architecture and overall design, and wrote the main program which will coordinate all the tests to be run. It includes from the begining dependency-based scheduling and paralellism, so I hope it will scale well in SMP computers. I also wrote a draft specification for the plugins and description of the running plan in the wiki page. I've also asked for the creation of a Alioth's project, so I can start publishing the code, and get feedback from the people that will use it.

I'm still lacking the obligatory post-DebConf7 post... These weeks had been intense... So, first of all, thanks to all the DC-team for another wonderful conference. Next year I'll feel how it's done one of the two things that everybody likes but nobody wants to know how it's done. Hint: not sausages. This is the first time I've also attended DebCamp, and it was very useful. Not that I could do any real work, as very few people are able to do that during DebConf, but I managed to do what I planned, that is talk with many people who are working on debian-cd, debian-installer and custom-cdds. I've already uploaded my pictures to a temporary place, whitout captions, until I decide how I will replace the stinko called gallery. And there you can see that I was one of the few that actually wore a kilt during the event —not a proper Debian kilt, but hey... After DebConf, and after a year of hard work, I took a seven days vacation in the UK. So I was able to know Edinburgh (I prefer to hang out with Debian people that doing tourism during the conference, so I only knew the way to the supermarket). I also went to the Arthur's Seat HIll, which was very beautiful, and was only a 1-hour walk from the hostel to the top. Then I did a small visit to Stirling and concluded that castles in Scotland are not that interesting (but go to the Smith Museum!). Then, I went to London for a couple of days (dang, that was expensive!!), visited Marx's tomb, rided near the car bombs, got to know all the nice stuff that the british stole from around the world in the British Museum (it's really great, and free), and went to Cambridge to spend a day (and punting, I even did the punting myself!). Finally I spent my last day walking around that beatiful city, in the Soho and South Bank areas. I don't want to forget to say thanks to Nattie, Kai and Ben for hosting me, you guys rock!! This last week has been very productive. I was working GSoC project, trying to recover from lateness, and staying until late. It was worth it, I have finished settling down the architecture and overall design, and wrote the main program which will coordinate all the tests to be run. It includes from the begining dependency-based scheduling and paralellism, so I hope it will scale well in SMP computers. I also wrote a draft specification for the plugins and description of the running plan in the wiki page. I've also asked for the creation of a Alioth's project, so I can start publishing the code, and get feedback from the people that will use it. PS: do MIND THE GAP, please. :-)

I'm still lacking the obligatory post-DebConf7 post... These weeks had been intense... So, first of all, thanks to all the DC-team for another wonderful conference. Next year I'll feel how it's done one of the two things that everybody likes but nobody wants to know how it's done. Hint: not sausages. This is the first time I've also attended DebCamp, and it was very useful. Not that I could do any real work, as very few people are able to do that during DebConf, but I managed to do what I planned, that is talk with many people who are working on debian-cd, debian-installer and custom-cdds. I've already uploaded my pictures to a temporary place, whitout captions, until I decide how I will replace the stinko called gallery. And there you can see that I was one of the few that actually wore a kilt during the event —not a proper Debian kilt, but hey... After DebConf, and after a year of hard work, I took a seven days vacation in the UK. So I was able to know Edinburgh (I prefer to hang out with Debian people that doing tourism during the conference, so I only knew the way to the supermarket). I also went to the Arthur's Seat HIll, which was very beautiful, and was only a 1-hour walk from the hostel to the top. Then I did a small visit to Stirling and concluded that castles in Scotland are not that interesting (but go to the Smith Museum!). Then, I went to London for a couple of days (dang, that was expensive!!), visited Marx's tomb, rided near the car bombs, got to know all the nice stuff that the british stole from around the world in the British Museum (it's really great, and free), and went to Cambridge to spend a day (and punting, I even did the punting myself!). Finally I spent my last day walking around that beatiful city, in the Soho and South Bank areas. I don't want to forget to say thanks to Nattie, Kai and Ben for hosting me, you guys rock!! This last week has been very productive. I was working GSoC project, trying to recover from lateness, and staying until late. It was worth it, I have finished settling down the architecture and overall design, and wrote the main program which will coordinate all the tests to be run. It includes from the begining dependency-based scheduling and paralellism, so I hope it will scale well in SMP computers. I also wrote a draft specification for the plugins and description of the running plan in the wiki page. I've also asked for the creation of a Alioth's project, so I can start publishing the code, and get feedback from the people that will use it. PS: do MIND THE GAP, please. :-)

Busy weekend, and I haven't taken the time to blog over the last few days as I should have. On Thursday and then Friday, Maj, Amita June and Brian and I went down to the Montreal Jazz Festival, which took place last week. It was a great time -- we went and saw some free acts, including some Dixieland, some Latin jazz, and the awesome Klez Dispensers playing klesmer. We also went to the huge playground and crafts centre for children on-site. It had slides, a big moonwalk bouncy-room, tunnels, a sandbox, and a giant piano that kids can step on to play music. They also had cool drawing and crafts projects -- a neat, active addition. The Jazz Fest is a great place to bring little kids. Amita June loved dancing to the music and playing in the playground. tags: montrealjazzfest maj brian amitajune

Quebec City Yesterday and today we spent in Quebec City. Brian hadn't been before, and the old, walled city is on the list of UNESCO World Heritage Sites, so we dragged him up there. It was also the Festival d' t de Quebec while we were there, which was pretty fun. We had no idea the festival was happening, so it was a pleasant surprise. Hell, I have a hard time keeping the festivals in our own city straight. We got rooms at the really nice Hotel Ch teau Laurier just outside the walls near the festival site. It was really nice -- modern, sleek, and pretty affordable (we got a big double-queen room for $150). Then we walked around the Upper Old Town in on-again-off-again rain, and watched some of the street performers. There was a troupe of stilt-walkers dressed like stone angels and gargoyles -- an awesome performance that scared the bejeesus out of Amita June. But as they left she said, "Bye bye, monsters," so I guess they weren't all bad. We had dinner at the cozy Moine chanson, a really nice restaurant just outside the walls of the Upper Old City. Quebec's restaurant scene in this are is, well, pretty dreadful -- lots of mass-production restaurants that serve tourists mechanical food from conveyor belts. The Moine was an accidental find, and it was really a godsend -- pleasant, interesting pan-Mediterranean food. I had grilled sardines (memories of Portugal); Brian had a thoughtful paella and Maj had a roasted vegetables with polenta plate. They also had a choice selection of organic wines. I have preconceptions about organics, I admit, but we had a very tasty and complex organic Douro from Portugal last night that set me back in my seat. We walked home through the music festival just as Nickelback was coming on the main stage. Brian wandered a bit, Amita June and Maj and I went to bed. Unfortunately, that wasn't the last time we were up: AJ woke up vomiting in the middle of the night. She only barfed once, and we're not sure what caused it: whether she overate at the restaurant (she packed away a lot of food), or if she got some sausage or chicken in the couple of cupfuls of Brian's paella she ate. Either way, she was a sad toddler and we were freaked-out parents. Having your child get sick while you're traveling is really scary. You start thinking about helicopter rescue. But she was fine after one big purge, and we all slept well until this morning. Maj got up to change Amita's diaper, and we realized we didn't have any. Not in the hotel room, not in the car. I went out from the hotel in increasingly wide spirals looking for a convenience store, but those that had diapers weren't open on a Sunday morning, and those that were open didn't carry diapers. After a couple of hours (!), I came back to the hotel in defeat. Fortunately, Maj managed to find a diaper in the emergency stash in the trunk of the car. We picked up Brian and drove to another part of town, where we got a tasty brunch on the terrasse and found some diapers at the Jean Coutu drugstore. Brian kept going from Quebec City -- he's heading all the way around the Gasp Peninsula to New Brunswick, Nova Scotia and Prince Edward Island, and coming back to Montreal in about a week. Maj and I drove home on the South Shore -- Amita fell asleep as we hit the freeway and woke up when we pulled into the driveway -- and had a nice afternoon at home. tags: quebeccity summerfestival travel amitajune

Seafood windfall On Friday, our friend Sebastian called us with a problem: his sister, who lives in the les-de-la-Madeleine, had sent him too much fresh seafood. It had come out of the sea and been driven to Montreal, and there was too much for him and his family to eat. The fish was less than 24 hours out of the water -- could we help him out? So we ended up with two huge bags of succulent lobster meat and three more of scallops. Oh, man, were they good. On Saturday morning before we left for Quebec City I made up two giant lobster omelets for the four of us to share. They had just huge chunks of lobster in them, and it was the most decadent meal I've had in a while. It was so much, we all lay around on the couch moaning for about an hour. Tonight, I cooked up the scallops. They were gigantic, tender, and fresh, so I pan-seared them in butter and olive oil with salt & pepper and prepped garlic and oregano. I know, that seems too much for such a delicate meat, but it worked out great. I put them on a bed of whole-wheat linguine, arugula, and red peppers. They were really, really good. I don't think I'm going to have seafood this good again for a while. Mmmmm. Thanks, Sebastian! tags: scallops lobster seafood

News in brief from the land of the Free Two Freedom news items:

• Robin Millette has a great list of Sourceforge alternatives.
• openmoko.com is open for business. It's possible to buy the coolio new Neo 1973 developer preview through the Web site. I'm ordering one!

tags: freedom robinmillette openmoko

I (as root) have a directory hierarchy that I want a particular group to always have write access to. The files and folders inside the hierarchy are owned and manipulated by a wide variety of diffrent users. Essentially I want to delegate ‘root’ access for a portion of the filesystem to a particular group. My first attempt at implementing this was to use the standard POSIX ACLs that are available for almost every filesystem Linux supports. I recursively set an ACL on the top-level directory to give the group write access to all files and directories that currently exist and then I recursively set a default ACL to give the group write access on all the directories. This default ACL should be inherited by any new files that are created ensuring that the group keeps write access to everything. Problem solved? Unfortunately not. The intricacies of complying with POSIX means that ACLs are implemented as an ACL plus a mask. To gain access to a particular file or directory the user or group must match an appropriate ACL granting the access and the mask for that file or directory must also allow the requested permission to be granted. When you add an ACL to a file or directory, the ‘group’ bits of the standard Unix permissions magically switch from controlling group access to controlling the mask portion of the ACL, effectively providing an upper bound on the permissions that an ACL entry can grant. This prevents legacy POSIX applications that do not understand ACLs from unintentionally granting excessive permissions - arguably a good thing. Unfortunately this also makes it very hard to preserve the ACL granting write access to the ‘root’ group which I legitimately intended to have in place on this portion of the filesystem. Newly created files under the hierarchy generally inherit the ACL as intended, as most applications attempt to create files with as many permissions as possible, leaving it up to the umask to remove undesired permissions. However any file that is copied into the hierarchy without the ‘group’ write bit set, or any file that has the ‘group’ write bit removed via chmod will actually remove the write bit from the ACL mask invalidating the ACL and leaving me back at square one! After a bit of Googling I thought that NFSv4 ACLs might be the answer to this problem, as they are marketed as “very similar to Windows ACLs” and I’m sure that I vaugely recall Windows being able to properly inherit ACLs from parent directories. Unfortunately after downloading the NFSv4 ACL patches and trying all the various mount options I cannot find any combination that will offer the functionality I need. The implementation conforms to POSIX, so it still has a mask parameter and the same problems as the standard POSIX ACLs. The only benefit from using NFSv4 ACLs that I can see is that you have more permissions to grant. So once again, I’m back to square one. I’m hoping that there is some fundamental point that I’m missing as this seems like a very common use-case that I would have thought would be well supported. If a command-line example is clearer to you look at:
http://www.mattb.net.nz/blog/dump/acl-inheritance-problems.txt My current solution is to run a cronjob every X minutes to recursively ‘chmod -R g+w /dir’, however that’s far from optimal as it exposes all sorts of race conditions and just seems ugly! Any suggestions or solutions will be gratefully received.

Don’t you just love leaked corporate memos? I do. They are a window into that which we KNOW exists, yet we are not privvy to very often. Because corporations like to keep their dark, dirty secrets well hidden. Memos give us access to what I call in High Academic Jargonese “Corporate Psychological Interiority,” or to put in simpler language: memos allow us to see corporations crapping in their pants. Gross, but pleasant to see from time to time. I just got word of leaked memos from the Insurance Company I love to hate: Blue Cross Blue Shield. The four pages are chock full of interesting stuff, so take a read for yourself. Here, I will only highlight two things:

* Horizon BlueCross/BlueShield is picked out early in the film in a collage of stories citing bad
treatment of members.

And well, as most readers here know, I concur. Well at least
Blue Cross Blue Shield of New Jersey/Horizon can be proud for being one of the worst. In their concluding talking point sections, they say:

2) The Blues recognize the need for improvement of both the coverage and delivery of healthcare.
But the divisive tone set forth by Michael Moore and his movie “Sicko” is not helpful. Positive
change to our healthcare system can be best achieved through shared responsibility, not
recrimination. To ensure Americans have access to the best healthcare that is both timely,
efficient, and of high quality, requires the collective contribution of all stakeholders
consumers, providers, employers and the government.

Try NOT being angry at a 4,000 dollar bill or worse, an 80,000 dollar bill, or even worse a death. It is near to impossible to stop the rumblings of anger. We are human beings, after all. We are born with the capacity to think and feel, passionately and deeply. And some of the best change comes from the fire that is anger and I hope that enough Americans are finally feeling the fire.

Dekoor, a student choir which includes our friend Jense, from Utrecht participates in a television show Korenslag, a pop-idol like contest for choirs. As we've been devoted fans of Jense from the "Miracle Man" up to now, we attended the semi finals of this program. Being in the audience of this show that is broadcast live on national television is an experience in itself. An "applause master" instructed us when to be silent, when to applaud, and when to go absolutely crazy. To which we faithfully complied. During the show Dekoor did very well in our unbiased ears, although the others did okay too. Although we SMS'ed a fortune to vote for Dekoor, the choir "Re-Flax" got the popular vote. Quite to my suprise because they were the least original and dynamic of the four, in my opinion. Luckily the professional jury was again impressed and decided that Dekoor must go to the finals together with "Re-Flax". Unfortunately the finals are decided by popular vote (eur 0,60 per vote) which might work against them. So remember, next Thursday, 20.30 Nederland 1: the finals. Vote Dekoor! P.S. I made some pictures aswell.

Mark Greenaway writes about writes about robots being designed to remove weeds from farms. This seems like a bad idea given that we currently have an energy crisis due to CO2 emissions from power plants causing environment change (including reduced water supplies), and coal and nuclear power plants requiring water to produce electricity (pity about the reduced water supply). Introducing robots that require either electricity of petrol for operation and also maintenance (which probably requires plastic parts on occasion) seems to be more of a problem than a solution. Fortunately there is already work in progress on training sheep to eat weeds. Given that we already raise sheep for wool and food it makes more sense for them to eat unwanted vegetation (weeds in a vineyard is the example given) than to grow grass specially for them. Also Mark says that the recent increase in oil price is not good news. What would be ideal is a slow steady increase in petrol prices that allows individuals and companies to change to more efficient vehicles (smaller and slower vehicles, hybrid vehicles, and more use of public transport) and also allows governments to make changes (such as building new public transport infrastructure). Having the oil remain cheap until it starts to run out doesn’t do anyone any good (apart from the short-term interests of oil companies).

Share This

Well, the last day in Greece went swimmingly. It was mostly just relaxing, playing on the Internet, chatting with lovely Di, and working out logistics of taking a long flight back to the United States. The only strange thing about it is that I haven't slept since the last post, which is away from the norm. It's a kind of forced all-nighter caused by the bad scheduling that I made for myself. The flight from to Athens happened at 10:20pm in the evening. This was the latest flight that I could get and assure that I could get to the Athens airport in time for my 7:20 flight to Madrid, which connects through Chicago and back to Minneapolis at last.

I've spent a total of 10 days outside of the United States now, the longest time since I have been alive, according to my knowledge. It has caused some interesting results. I realized that I am inextricably connected to the people who I am familiar with in my life, and being without them decreases my mood significantly, and without contact to them decreases my mood sharply.

I am not alone in this fact. There are many studies that show that people who have more social connections lead happier lives. I am lucky in that I am technologically ept and can connect to my near and dear in a number of ways, and be resourceful in finding other ways to contact them if necessary. I feel that I am lucky to live in a time which it is possible for me to spend so much time apart and still be in connection with people. I am also lucky that I was born in a geological area which enables me to be as such.

At the same time, I am sad to leave this place. It is a nice area, with good food and a very long history. As Nikos reminded me the other day, it is the birthplace of Democracy, which I hold to a fairly high regard. It is also the birthplace of modern medicine, and a significant portion of many early sciences. The wealth of the nation as a whole and as parts is clear to me, and I wish to return.

I've learned part of the language while I am here, and it has made me more cognizant of communication with other cultures. People in Athens seemed put off when I tried to talk in Greek, but in Kos they were at the worst amused at my attempts. The language may not be as romantic to learn as French or Italian, and may not be as useful in the long term as Latin, but it would be interesting to speak nonetheless. I am tempted to start working on some of the i18n Greek QA issues in Debian -- It would help me learn the language more, and also keep me aware of the issues in software when you start working with other countries' characters. I have been lucky to be using a MacBook this entire time with decent Greek support, allowing me to type at least some greek while I was here. It helped me understand even more of the language - I now know mostly the numbers 1-9 at least.

The trip has highlighted for me something which is highly lacking in airports: power. Every airport that I have been to has prominently displayed the signs of wireless access, and has people using their laptops in almost every direction you can look (I am one of those), but they are sorely without power points for people who are using those laptops. This may be a designed flaw in order to save energy, but I don't really believe that 50 or even 5,000 laptops at 65-150W will put a significant dent in the power grid at somewhere as large as an airport, where LCD screens are running 24/7 with ads and bright lights illuminate every corner for all to be seen. I won't even start talking about the airplanes themselves.

All in all, the trip has been a happy one, and I would very much repeat it. I will make different decisions on lodging at some places and probably plan a shorter trip. I however wholly endorse Greece as a vacation spot, and even would consider moving here if offered a position (that would depend highly on Diana as well, obviously). I find it entirely favorable. I hope that this small series on the site has caused you, the reader some pleasure at least, even if it was at my expense.

Now begins the long trip home, on three planes and with 14.5 hours in the airplane and 18 hours in total. It'll be only 10 hours counting local time, unfortunately -- the jet lag will, I'm sure, be horrible.

Comments: (0) Trackbacks: (0)

Comments

1. No comments

Trackbacks

1. No Trackbacks

So with the GPLv3 and LGPLv3 finally out (hurray!) it’s time to actually pay attention to the practical consequences of an upgrade to the world’s premier copyleft licenses. One of the things I hadn’t noticed, though I should have, is that the LGPLv3 is not compatible with the GPLv2 – that is, GPLv2 programs that used LGPLv2 libraries cannot make use of new versions of those libraries released under the LGPLv3 without also updating their license to GPLv3 (or “GPLv2 or above”, or LGPLv2 or LGPLv3 or something similar). Fortunately one of the major LGPLv2 libraries out there – glibc – is reportedly going to remain available under the LGPL v2.1 but for other libraries, it could become a problem, and it’s something GPLv2 authors will need to pay attention to. Also fortunately the opposite problem – GPLv3 apps using LGPLv2 libraries isn’t a problem, because the LGPLv2 explicitly allows redistribution and modification under the terms of GPLv2 or later, so you can take an LGPLv2 library and incorporate it in a GPLv3 application and just release the whole thing under the GPLv3. Unfortunately, the LGPLv3 doesn’t have a similar clause, so when the GPLv4 comes out, if it ever does, we could easily find the GPLv4 apps can’t use LGPLv3 libraries, and GPLv3 apps can’t use LGPLv4 libraries. Really these are side-effects of a fundamental change: up until now, almost everything we’ve distributed – and certainly all the low level stuff: the kernel, the C library, the compiler, the shell and basic tools – have all been under a GPLv2 compatible license. Some of them have been more flexible than that, due to being under a BSD or MIT license, or being under the LGPL or having a “GPLv2 or later” clause, but we’ve been essentially able to ignore all that and just treat everything as GPLv2. There are a few exceptions that we have to deal with – the occassional app that’s not GPL compatible doesn’t cause much of a problem, because things that depend on it don’t usually create copyleft issues, which just leaves checking that it doesn’t use any GPL-only libs, such as gnutls or libreadline. Openssl is a nuisance, in that it’s a popular, GPL-incompatible library, but it’s also reasonably easy to deal with on a case by case basis. But with the GPLv3, that changes: as GPLv3 apps and LGPLv3 libraries get uploaded, the assumption that more or less everything is GPLv2 compatible goes out the window, and while some things are GPLv2 only (such as the kernel!) we also can’t assume everything is GPLv3 compatible. Fortunately, both the GPLv2 and GPLv3 have what is known as the “system library” exception, which allows you to not bother distributing the source to your C compiler or C library or kernel with your app, because everyone’s assumed to have one of those already. Unfortunately, the GPLv2 version of that exception includes the phrase “unless that component itself accompanies the executable”, so Debian has always cautiously assumed the exception doesn’t apply to us, because everything we distribute in main “accompanies” everything else in main – kernel, C library, C compiler, etc. Equally unfortunately, while the GPLv3 avoids that problem, but ends up with a clause that’s not terribly comprehensible, despite a couple of rewrites during the drafting process. What’s all that mean? The big thing is that it means Debian’s going to have to put a bit more attention into the system library exception, both for GPLv2 and GPLv3 apps, because our current policy of trying to avoid needing it is probably not going to be sustainable. I’m not sure how in depth we’ll need to go for that; we’ve already contacted the FSF licensing folks for an explanation of the GPLv2 clause in relation to the CDDL, but it’s possible the FSF might prefer to be ambiguous in responding so as to keep their options open for defending or enforcing the GPL later, in which case we might need to get legal advice of our own as to what we should do. The second thing is that in reviewing our understanding of the system library exception, we might be in a position to have a Debian GNU/OpenSolaris port, making use of the exception for the CDDL-licensed OpenSolaris kernel and libc, as Nexenta do. That seems to me to be almost certainly possible for GPLv3 (or “GPLv2 or above”) applications, and probably possible for GPLv2 apps as well. If so, that’ll let us expand Debian to supporting four kernels in the near future: Linux, Hurd, FreeBSD and OpenSolaris, which will be pretty fun, in my book. The downside, is that if we find an interpretation of the system library exception that lets us mix and match licenses comfortably, that same interpretation might be able to be used to let propietary software vendors mix and match GPLed software with non-free software, rather than just free software with different copyleft licenses – and thus defeat the entire point of copyleft. What’s worse, is that by following Debian’s lead, they’d have some moral authority in doing that, beyond just having a legal argument as to why it’s okay. And that’s the rub. (My thoughts on the license texts as posted to -legal are here, here, here and here.)

I've been spending some time on the #bittorrent channel on Freenode, to see if I could get any good information from other bittorrent client developers. After some poor initial contact, I did get some good suggestions, as well as some tidbits of information I wasn't aware of. One of the tidbits was that most of the bittorrent client developers frowned on the use of selective downloading for torrents, to the point where some refuse to implement it in their clients (apparently the latest mainline client doesn't even have it, though I haven't checked due to license issues). This concerned me, as the DebTorrent client I'm working on will rely heavily on selective downloading to only download packages to be installed by the user. It's not clear to me that it would be an issue though. I'm sure it will make it more difficult to find a rare package in a large swarm of peers, but that's to be expected, and is nicely solved by using an HTTP download from a backup mirror. This backup downloading could become extreme though, making the bittorrent-like download (peer-to-peer) mostly just an HTTP download (client-server). To see if it will be a problem, I conducted a simulation of different sized swarms to determine the amount of unnecessary HTTP downloading that will occur. First, some assumptions to make the simulation easier:

• peers join and download sequentially and one at a time
• peers never leave
• peers download all the packages for their system at once (i.e. all fresh installs, no upgrades)
• peers are all interested in the same version and architecture of packages
• there are N total peers, and each can make C connections to other peers

With these assumptions, the simulation becomes quite simple. I used the popcon data to assign packages appropriately to the N peers. The peers download their assigned packages from the C previous peers, if possible, or otherwise by HTTP. I ran this multiple times, varying N and C, and calculating each time how much was downloaded through the debtorrent protocol, and how much through HTTP. This graph shows the percentage of the total download that used HTTP from a mirror. The optimal line shows the minimum possible (i.e. all peers are connected), which corresponds to only one copy (the first) of every package being downloaded using HTTP. This verifies my previous thinking, which is that fracturing the download population into multiple small swarms results in lots of inefficiencies, and lots of HTTP downloading. All efforts will need to be taken to keep the downloading populations together. (This is especially difficult for unstable, when a new swarm could be created twice a day due to archive updates.) Swarms of 1,000 peers or more seem to be sufficient to minimize the HTTP downloading. This graph shows the amount of unnecessary HTTP downloading that occurred, which is just the difference of each line from the optimal one in the previous plot. This shows the danger of selective downloading, as all of this unnecessary HTTP downloading occurs because the swarm is too big to find peers that have the desired package (they do exist). Fortunately, this unnecessary downloading seems to approach a maximum as the swarm size increases. For the large swarm sizes needed to make the downloading efficient, and assuming a reasonable number of connections of 100 (bittorrent defaults to maintaining at least 40, and large swarms usually means a large number of connections), we can expect to be using HTTP for about 4% of the download. This number is surprisingly low due to the popularity distribution of packages (only rare ones are hard to find, but they aren't downloaded very often). I think 4% is manageable in our situation, given that there is a backup method to find the rare packages, though it does show the need to have that backup method available. This is probably more of a problem for regular bittorrent clients, in which there is no backup method.

Ion3 supports docked applications and is compatible with the docking protocols used by WindowMaker and KDE. Normally you would use:

dopath("mod_dock")

to load mod_dock, which provides a standard corner dock, toggled using MOD1+D. However, recent versions also support docking applications in the status bar, in "system tray" style:

-- In cfg_ion.lua:
defwinprop  
   class = "foo-window-class",
   statusbar = "foo"
 
-- In cfg_statusbar.lua:
mod_statusbar.create  
   ...
   template="... %systray_foo"
 

GNOME unfortunately uses a different dock protocol. However, the docker program can adapt from this to the WindowMaker protocol. So you can include a GNOME system tray in your status bar by including docker in your X session and this in your Ion3 configuration:

-- In cfg_ion.lua:
defwinprop  
   class = "Docker",
   statusbar = "dock"
 
-- In cfg_statusbar.lua:
mod_statusbar.create  
   ...
   template="... %systray_dock"
 

From Gareth Bowker, Jaded World View: "Tardorrist (n) - would-be terrorist without sufficient intelligence to build and detonate an explosive device."

My colleague Jonas updated his 3dm2 Debian packages for i386 and amd64 to the latest release 9.4.1.2 today. Unfortunately 3ware decided to use a Java based installer for the latest releases, so it was quite a PITA to update the packages.

In Improving simple charts, Dirk Eddelbuettel wrote a nice little summary how to produce “nicer” charts using R. However, in my opinion, the charts he provided as alternate options to the original, purely linear graph are less readable for the average person. If anything, using a logarithmic scale on the original Y axis (number of packages not built after a specific date) would be of use, but the other graphs only hide the original point of the post Dirk replied to (How old are our (Debian’s) packages?). The point was that 94% of all packages had been rebuilt since the release of Sarge or, if you want to put it the other way around, 1264 (6%) were built before Sarge was released. This relationship is hidden from immediate recognition by the reader of the graphs. I admit though, that Dirk’s second graph (Number of packages against age in days with the age being on y logarithmic scale) is also insightful. The other two graphs he presents might make sense in a scientific publication if some point in the “more detailed” areas of the graph needs proving, but for this specific problem, they really don’t make sense to me. Actually, I would question their use even in scientific publications, since they distort the original data a lot. Scientists are pretty much used to logarithmic scales these days, but I couldn’t work out how the scales work on those two graphs. And another note on the second graph (logarithmic age): The way the X axis is labeled (10, 20, 50, 100,..) is also pretty non-intuitive to me. I’m much more used to something like 5, 10, 20, 40, 80, 160 and so on, which gives the same distance between each label and seems a lot more usable to me.. (edit: Fix spelling of Dirk’s surname)

Share This

Eweek has an article on a new 1TB Seagate drive. Most home users don’t have a need for 1TB of storage (the only way I’ve ever filled a 300G drive is by using it for multiple backups) and enterprise customers generally want small fast drives (with 100G drives still being sold for servers). One interesting thing to note about this is the fact that the new drive is described as being able to sustain 105MB/s (although I suspect that is only for the first few zones of the disk - see my ZCAV results page for graphs of the performance of some typical disks). The previous disks that I have seen have topped out at about 90MB/s. However even if the drive could deliver 105MB/s over it’s entire capacity it would take almost 3 hours to perform a backup - increases in IO speeds have not been keeping up with capacity increases. Another interesting thing is the potential for using the same technology in low power 2.5 inch disks. While a 1TB disk isn’t going to be much use to me a 300G 2.5inch disk that uses less power will be very useful - and it might be possible to perform a backup of such a disk in a reasonable amount of time! The latest trend in PCs seems to be small form factor (SFF) and low power machines that don’t have space for two drives. If you want RAID-1 on your home machines for reliability then that isn’t very convenient. But two 2.5 inch disks can fit in less space than a single 3.5 inch disk and therefore all the SATA based SFF machines that are currently being shipped with a single 3.5 inch disk can be upgraded to a pair of 2.5 inch disks - this will be convenient for me when those machines start going cheap at auction next year! Even for servers the trend seems to be towards 2.5 inch disks. I recently bought a 2U HP server that supports up to 8 hot-swap SFF disks, I wonder how the performance of 8*SFF disks would compare to 4 of the bigger and faster disks. The next thing we need is a method of backing up large volumes of data. The 650M data CD came out when a 150M disk was considered big. The 4.7G data DVD started to become popular when 45G disks were considered big. Now almost everyone has 300G disks and 1TB disks are becoming available yet the best commonly available backup method is a double-layer DVD at 9.4G - it seems that the vast majority of home users that make backups are using cheap IDE disks to do so. Fortunately there are some new technology developments that may improve the situation. Call/Recall has developed technology that may permit multiple terabytes of storage on an optical disk. It’s yet to be seen whether their technology lives up to the claims made about it, but we have to hope. The current storage situation is getting unmanagable.

Share This

Nokia's N800 internet tablet is an intriguing device. When I originally got one a few months back I tried to treat it purely as a consumer object, just using the installed apps and things available through the obvious point-and-click channels.As a consequence it served mainly as a portable (and expensive) internet radio, streaming me the cricket commentary from the BBC. And when I upgraded my wireless network I somehow managed to make it WPA2 only, knocking the N800 offline. An offline N800 is an almost thoroughly useless device, so it went into the drawer and I forgot about it.Ultimately you can't leave something that expensive unused, so I dragged it out again, fiddled the wi-fi router into compliance, and decided not to deny my hacker nature this time. The N800 is Linux underneath, so who could resist?
The result is two-edged, really: I'm a lot happier with the device, but on the other hand must conclude that the N800 is still a bit far from being consumer-ready.Must-have softwareSo, what things did I install this time around that made the device happier?Claws Mail is a nice email client that works well with my IMAP accounts, which all use SSL and TLS, have lots of messages and a deep folder hierarchy. I don't really want to write much mail on the N800, but an easy reading interface is a bonus.The FM radio is something I can't believe I missed before. I had no idea this was in there, but plug some headphones into the N800 and they act as the antenna for an FM receiver. Desperately cute and old-world, a bit like when laptops still used to have parallel printer ports.I had previously ignored Maemo Mapper, thinking it was useless without a GPS, but it turns out to work very nicely as a dedicated client for Google Maps, as well as several other mapping sources. Must-do geeky bitsTry as I might to like the touch screen, the first thing I had to do with the device was find a way of not using the stylus to do sysadmin type tasks on it.So, first stop is to get a terminal up, figure out how to use the root account, install a SSH client and server, and get my pubkey onto the N800. Now I could shell into it and use a decent keyboard.On my local network I hate maintaining DNS if I don't have to, so the next thing I wanted was Zeroconf support in the shape of avahi. One of the quickest ways to get this going is to install the Canola media application, which uses Zeroconf to find shared music.With these basics in place, the N800 supports APT package repositories familiar to Debian and Ubuntu users, so the device becomes a lot less weird and much more manageable. I felt the same pleasant familiarity as I did with the NSLU2. Things to look forward toThe N800's video camera is neat, but nobody I know uses Google Talk for conferencing. Fortunately it seems that Skype for the N800 is just around the corner. Initially, video support is unlikely, but I imagine that if Skype on the N800 proves popular, it won't be far off.The N800 is something I don't mind having kicking around the kitchen or nursery, so staying in touch with my family while I'm travelling will become a lot more fun.
Also, I'd like to get NFS running on the N800, but that requires the installation of a new kernel, which I've not quite yet had the time to do. Once that's done, all my media, photos and storage will be handily available. 

I’ve just watched the video of Stewart Smith’s LCA talk Eat My Data about writing applications to store data reliably and not lose it. The reason I watched it was not to learn about how to correctly write such programs, but so that I could recommend it to other people. Recently I have had problems with a system (that I won’t name) which used fwrite() to write data to disk and then used fflush() to commit it! Below is a section from the fflush(3) man page:

NOTES

       Note that fflush() only flushes the user space buffers provided by  the

       C  library.   To  ensure that the data is physically stored on disk the

       kernel buffers must be flushed too, e.g. with sync(2) or fsync(2).

Does no-one read the man pages for library calls that they use? Then recently I discovered (after losing some data) that both dpkg and rpm do not call fsync() after writing package files to disk. The vast majority of Linux systems use either dpkg or rpm to manage their packages. All those systems are vulnerable to data loss if the power fails, a cluster STONITH event occurs, or any other unexpected reboot happens shortly after a package is installed. This means that you can use the distribution defined interface for installing a package, be told that the package was successfully installed, have a crash or power failure, and then find that only some parts of the package were installed. So far I have agreement from Jeff Johnson that RPM 5 will use fsync(), no agreement from Debian people that this would be a good idea, and I have not yet reported it as a bug in SUSE and Red Hat (I’d rather get it fixed upstream first). During his talk Stewart says sarcastically “everyone uses the same filesystem because it’s the one true way“. Unfortunately I’m getting this reaction from many people when reporting data consistency issues that arise on XFS. The fact that Ext3 by default will delay writes by up to 5 seconds for performance (which can be changed by a mount option) and that XFS will default to delaying up to 30 seconds means that some race conditions will be more likely to occur on XFS than in the default configuration of Ext3. This doesn’t mean that they won’t occur on Ext3, and certainly doesn’t mean that you can rely on such programs working on Ext3. Ext3 does however have the data=ordered mount option (which seems to be the default configuration on Debian and on Red Hat systems), this means that meta-data is committed to disk after the data blocks that it referrs to. This means that an operation of writing to a temporary file and then renaming it should give the desired result. Of course it’s bad luck for dpkg and rpm users who use Ext3 but decided to use data=writeback as they get better performance but significantly less reliability. Also we have to consider the range of filesystems that may be used. Debian supports Linux and HURD kernels as main projects and there are less supported sub-projects for the NetBSD, FreeBSD, and OpenBSD kernels as well as Solaris. Each of these kernels has different implementations of the filesystems that are in common and some have native filesystems that are not supported on Linux at all. It is not reasonable to assume that all of these filesystems have the same caching algorithms as Ext3 or that they are unlike XFS. The RPM tool is mainly known for being used on Red Hat distributions (Fedora and RHEL) and on SuSE - these distributions include support for Ext2/3, ReiserFS, and XFS as root filesystems. RPM is also used on BSD Unix and on other platforms that have different filesystems and different caching algorithms. One objection that was made to using fsync() was the fact that cheap and nasty hard drives have write-back caches that are volatile (their contents dissappear on power loss). As with such drives reliable operation will be impossible so why not just give up! Pity about the people with good hard drives that don’t do such foolishness, maybe they are expected to lose data as an expression of solidarity with people who buy the cheap and nasty hardware. Package installation would be expected to be slower if all files are sync’d. One method of mitigating this is to write a large number of files (EG up to a maximum of 900) and then call fsync() on each of them in a loop. After the last file has been written the first file may have been entirely committed to disk, and calling fsync() on one file may result in other files being synchronised too. Another issue is that the only time package installation speed really matters is during an initial OS install. It should not be difficult to provide an option to not call fsync() for use during the OS install (where any error would result in aborting the install anyway).

Share This