Search Results: "ag"

9 July 2007

Gustavo R. Montesino: Summer of Code mid-term report

I. Introduction

Today, the mid-term evaluations of the Google Summer of Code starts. It means it's a good for a comprehensive summary of what I've done so far.

For those who don't remember, My GSoC project is the Bug Triaging and Forwarding Tool, whose aim is to help triaging of Debian bug reports. On my proposal, I had included the following timeline:

During the proposals evaluations and the community bounding period, it was pointed to me that there's already a lot of good tools to handle bug reports, so I've changed this a bit and already started to work on BTS-Bugzilla interaction, which seems to be the more novel part of the
tool.

The development has been fragmented in three modules: btsutils, bzutils and bug-triage.

II. btsutils

The btstutils is a python module to interact with the Debian Bug Track System. Currently, it can query the bts by bug number, package, source package, maintainer, submitter and usertag. For each bug, it returns bug number, package, summary, status, severity, submitter and tags. Support for returning usertags is under way.

Ths btsutils can use html or soap to access the bug data, and has also a backend-agnostic module wwith tries the best method and, if that fails, tries the other.

Steffano Zacchiroli has offered me to include the btsutils on the debian-bundle, a bundle of python modules for dealing with Debian infrastrcture. I'm very excited by this idea, and believe that if all or at least some of the Debian tools which interact with BTS end using a standardized module for access, this might end up being the major contribution of my GSoC project for the Debian community.

III. bzutils

The bzutils is similar to the bzutils, but is to interact with bugzillas instead. Currently it can do queries based on GNOME's bugzilla boogle and the more generic boolean charts. The later works (at least) on GNOME, KDE and standard bugzilla (i.e., landfill). For each bug, it returns bug id,
product, component, status, resolution, reporter, assignee, summary, priority and severity.

IV. bug-triage

The bug-triage is the Bug triaging and forwarding tool. It's being developed using python, gtk and glade.

Currently, it can query the BTS using any of the btsutils' supported fields (even though the usertags aren't really accessible through the interface yet). It can also launch one of the returned bug reports in a web browser.

Integration with bugzillas is under development. It already has an interface to establish links between Debian packages and upstream bugzilla's address, product and component. The next steps will be to add an upstream button in the toolbar/menu and make it show the bugs selected through these info, allowing the user to mark the bug as forwarded to one of them.

V. The future

After that, I'll work on these features (in no particular order): filling a new upstream bug using the Debian bug data as a starting point, more advanced filtering options to filter the upstream bugs (such as filtering by words in the bug summary, and maybe through backtraces if available) and more ways to manipulate the BTS through the interface.

To be sincere, it seems this project won't be as advanced as I would like on the end of the GSoC. There are some new, unexpected (and good ^_^) things going on my life, which are taking some of the time I expected to dedicate to GSoC, which means I'll likely keep the current pace until the end of the project (I expected to increase it during July). Even so, I believe the
project is going on a good path, and I hope it'll be very useful for the Debian community.

Martín Ferrari: DebConf, travelling, pictures, SoC... Oh my!

I'm still lacking the obligatory post-DebConf7 post... These weeks had been intense... So, first of all, thanks to all the DC-team for another wonderful conference. Next year I'll feel how it's done one of the two things that everybody likes but nobody wants to know how it's done. Hint: not sausages. This is the first time I've also attended DebCamp, and it was very useful. Not that I could do any real work, as very few people are able to do that during DebConf, but I managed to do what I planned, that is talk with many people who are working on debian-cd, debian-installer and custom-cdds. I've already uploaded my pictures to a temporary place, whitout captions, until I decide how I will replace the stinko called gallery. And there you can see that I was one of the few that actually wore a kilt during the event —not a proper Debian kilt, but hey... After DebConf, and after a year of hard work, I took a seven days vacation in the UK. So I was able to know Edinburgh (I prefer to hang out with Debian people that doing tourism during the conference, so I only knew the way to the supermarket). I also went to the Arthur's Seat HIll, which was very beautiful, and was only a 1-hour walk from the hostel to the top. Then I did a small visit to Stirling and concluded that castles in Scotland are not that interesting (but go to the Smith Museum!). Then, I went to London for a couple of days (dang, that was expensive!!), visited Marx's tomb, rided near the car bombs, got to know all the nice stuff that the british stole from around the world in the British Museum (it's really great, and free), and went to Cambridge to spend a day (and punting, I even did the punting myself!). Finally I spent my last day walking around that beatiful city, in the Soho and South Bank areas. I don't want to forget to say thanks to Nattie, Kai and Ben for hosting me, you guys rock!! This last week has been very productive. I was working GSoC project, trying to recover from lateness, and staying until late. It was worth it, I have finished settling down the architecture and overall design, and wrote the main program which will coordinate all the tests to be run. It includes from the begining dependency-based scheduling and paralellism, so I hope it will scale well in SMP computers. I also wrote a draft specification for the plugins and description of the running plan in the wiki page. I've also asked for the creation of a Alioth's project, so I can start publishing the code, and get feedback from the people that will use it.

Martín Ferrari: DebConf, travelling, pictures, SoC... Oh my!

I'm still lacking the obligatory post-DebConf7 post... These weeks had been intense... So, first of all, thanks to all the DC-team for another wonderful conference. Next year I'll feel how it's done one of the two things that everybody likes but nobody wants to know how it's done. Hint: not sausages. This is the first time I've also attended DebCamp, and it was very useful. Not that I could do any real work, as very few people are able to do that during DebConf, but I managed to do what I planned, that is talk with many people who are working on debian-cd, debian-installer and custom-cdds. I've already uploaded my pictures to a temporary place, whitout captions, until I decide how I will replace the stinko called gallery. And there you can see that I was one of the few that actually wore a kilt during the event —not a proper Debian kilt, but hey... After DebConf, and after a year of hard work, I took a seven days vacation in the UK. So I was able to know Edinburgh (I prefer to hang out with Debian people that doing tourism during the conference, so I only knew the way to the supermarket). I also went to the Arthur's Seat HIll, which was very beautiful, and was only a 1-hour walk from the hostel to the top. Then I did a small visit to Stirling and concluded that castles in Scotland are not that interesting (but go to the Smith Museum!). Then, I went to London for a couple of days (dang, that was expensive!!), visited Marx's tomb, rided near the car bombs, got to know all the nice stuff that the british stole from around the world in the British Museum (it's really great, and free), and went to Cambridge to spend a day (and punting, I even did the punting myself!). Finally I spent my last day walking around that beatiful city, in the Soho and South Bank areas. I don't want to forget to say thanks to Nattie, Kai and Ben for hosting me, you guys rock!! This last week has been very productive. I was working GSoC project, trying to recover from lateness, and staying until late. It was worth it, I have finished settling down the architecture and overall design, and wrote the main program which will coordinate all the tests to be run. It includes from the begining dependency-based scheduling and paralellism, so I hope it will scale well in SMP computers. I also wrote a draft specification for the plugins and description of the running plan in the wiki page. I've also asked for the creation of a Alioth's project, so I can start publishing the code, and get feedback from the people that will use it. PS: do MIND THE GAP, please. :-)

Martín Ferrari: DebConf, travelling, pictures, SoC... Oh my!

I'm still lacking the obligatory post-DebConf7 post... These weeks had been intense... So, first of all, thanks to all the DC-team for another wonderful conference. Next year I'll feel how it's done one of the two things that everybody likes but nobody wants to know how it's done. Hint: not sausages. This is the first time I've also attended DebCamp, and it was very useful. Not that I could do any real work, as very few people are able to do that during DebConf, but I managed to do what I planned, that is talk with many people who are working on debian-cd, debian-installer and custom-cdds. I've already uploaded my pictures to a temporary place, whitout captions, until I decide how I will replace the stinko called gallery. And there you can see that I was one of the few that actually wore a kilt during the event —not a proper Debian kilt, but hey... After DebConf, and after a year of hard work, I took a seven days vacation in the UK. So I was able to know Edinburgh (I prefer to hang out with Debian people that doing tourism during the conference, so I only knew the way to the supermarket). I also went to the Arthur's Seat HIll, which was very beautiful, and was only a 1-hour walk from the hostel to the top. Then I did a small visit to Stirling and concluded that castles in Scotland are not that interesting (but go to the Smith Museum!). Then, I went to London for a couple of days (dang, that was expensive!!), visited Marx's tomb, rided near the car bombs, got to know all the nice stuff that the british stole from around the world in the British Museum (it's really great, and free), and went to Cambridge to spend a day (and punting, I even did the punting myself!). Finally I spent my last day walking around that beatiful city, in the Soho and South Bank areas. I don't want to forget to say thanks to Nattie, Kai and Ben for hosting me, you guys rock!! This last week has been very productive. I was working GSoC project, trying to recover from lateness, and staying until late. It was worth it, I have finished settling down the architecture and overall design, and wrote the main program which will coordinate all the tests to be run. It includes from the begining dependency-based scheduling and paralellism, so I hope it will scale well in SMP computers. I also wrote a draft specification for the plugins and description of the running plan in the wiki page. I've also asked for the creation of a Alioth's project, so I can start publishing the code, and get feedback from the people that will use it. PS: do MIND THE GAP, please. :-)

Matthew Palmer: Hell is other people('s code)

I've been working (part-time) on an existing Rails application for the last month or so, getting comfortable with it, modifying some sections to support needed functionality, and beginning an all-out, carpet-bombing "shock and awe" assault on it's test suite. However, a significant portion of the system is (or rather was, since I've already gutted a lot of it) a chunk of rather messy and unpleasant auto-generated code. I've always been a bit iffy about code generation tools, ever since my first encounters with them whilst working on some $DEITY-awful access application years ago. (First encounters with anything Microsoft usually have that effect on me, though). Since then, I've never found any reason to change my mind, and dealing with this particular chunk of auto-generated code has pushed me over the edge into "wild-eyed extremist" territory. Please note that when I say "code generator", I'm talking here about programs or wizards or whatever that create a block of code that you're then expected to tweak to suit your conditions. If it's just a mechanical transformation of an input format to an output format, where a human is never expected to maintain the output by hand, that's not code generation in my book -- that's just a compiler being used for it's intended purpose. I like compilers. I'm fairly sure you do too. Code generators, though? EVIL. Consider what you get out of your generator: A big blob of code in your application that you then have to understand and maintain. It's like you gave a commit bit to some random person who dropped a 5000 line change into your tree and then ran away into the night, cackling. If there's bugs in the generated code (and remember, there's always bugs in software), when they get fixed upstream you're up the creek unless you haven't modified the code at all (and thus can regenerate), or you're willing to apply the necessary patches to your modified version, often by hand. Hand-patching is error-prone, and if you've modified the code at the same place that has to be patched, things can get unpleasant. In the case of the generator I'm dealing with at the moment, I'm not even blessed with it being decent code. It's ugly code, in every possible sense of the word. There's inconsistent formatting, poor commenting, it uses some totally whack coding constructs[1], the hard-coded text strings are ungrammatical, it's got very few (and poorly covering) test cases, the methods are huge and unwieldy, and it's got security bugs (not so hot for something that's supposed to be guarding your application). In short, it looks like some quick-and-dirty internal hack that escaped one night and is now wreaking havoc on the world. There's probably some really good code generators out there. I've been poking around a bit, and there's some written for Rails by people who I know produce good code. However, no matter how good the author of the code generator is, the very best you can hope for is solid, clean code that is the equivalent of some portion of your application having been coded by some member of your team who you (and nobody else) has ever met, who didn't follow your team's coding conventions, and who has never looked at any other part of your application to make sure that everything hung together in a consistent manner. Which you now have to maintain. I've read a lot of books, articles, blog posts, and mailing list messages, and participated in no shortage of "why software sucks" discussions with other programmers, but I've never heard anyone advocate that the way to clean code, minimal bugs, and happy programmers was to drop a big chunk of unknown code right in the middle of things. Before you start charging up the e-mail-delivered tasers, ready to fry my delicate braincells for the twin crimes of Xenocodophobia and a vicious case of Not Invented Here syndrome, though, please understand that I'm not against code reuse in the right place. That right place is a library, component, or other abstract, documented, fairly static, and presumably-replaceable chunk of code. By using a library, you get a relatively simple interface to somebody else's complexity. This is a good thing, for all the reasons that everyone should already be able to recite. Items such as a comprehensible interface, minimal learning curve, and so on. Things you don't get with generated code because, pretty much by definition, the entire chunk of code is you "interface". There are some times, though, when what you want to do can't be done in a library. Either the thing has too many tentacles to be able to easily encapsulate behind an abstracted interface, or you need to modify it's behaviour in too many places to be able to get away with a few simple callbacks to get the job done. Hence, it might be tempting to fire up the code generator to get yourself out of a hole quickly. Don't do it though. Don't succumb to the temptation of saving yourself a little bit of time by firing up a code generator. What you're doing there is just making pain for yourself, by inheriting a big chunk of code that you don't fully understand, probably doesn't solve your problem particularly well, and which you (or whichever poor slob gets the job after you) will have to maintain into perpetuity -- most likely by gradually rewriting large pieces of it so that it fits your application better. The output of a code generator is likely to be the least maintainable code in your tree -- either by virtue of the fact that it's crap code, or at the very least because you just don't understand it. The general principle of good codebase hygiene is that you get rid of unmaintainable code quickly, so it doesn't infect everything else (a la the "broken window theory"). With that in mind, why would you ever consider deliberately introducing a huge chunk of unmaintainable code into your application? The closest I can come to a defence of code generators is something like Rails' scaffolding, where the controller and views for a set of simple CRUD operations on a table are auto-generated. The key word there, though, is scaffolding. It's very name suggests that it's temporary, to be replaced with something more robust and useful as soon as is reasonable. The default scaffolding is ugly as sin, besides, which encourages you to replace it, and -- as of Rails 1.1 (I think), there's a 'scaffold' method which creates the scaffolding output at runtime, so you never, ever need to generate scaffolding code any more. Anyone planning on producing any significant piece of their application through the use of code generators is, in my book, committing a crime against software decency.
1. My favourite whack code construct thus far has been this gem:
 case somevar
   when 17
     do_something_here()
 end
Because, you know, if statements are for people who know how to program, or something.

Evan Prodromou: 19 Messidor CCXV

Busy weekend, and I haven't taken the time to blog over the last few days as I should have. On Thursday and then Friday, Maj, Amita June and Brian and I went down to the Montreal Jazz Festival, which took place last week. It was a great time -- we went and saw some free acts, including some Dixieland, some Latin jazz, and the awesome Klez Dispensers playing klesmer. We also went to the huge playground and crafts centre for children on-site. It had slides, a big moonwalk bouncy-room, tunnels, a sandbox, and a giant piano that kids can step on to play music. They also had cool drawing and crafts projects -- a neat, active addition. The Jazz Fest is a great place to bring little kids. Amita June loved dancing to the music and playing in the playground. tags:

Quebec City Yesterday and today we spent in Quebec City. Brian hadn't been before, and the old, walled city is on the list of UNESCO World Heritage Sites, so we dragged him up there. It was also the Festival d' t de Quebec while we were there, which was pretty fun. We had no idea the festival was happening, so it was a pleasant surprise. Hell, I have a hard time keeping the festivals in our own city straight. We got rooms at the really nice Hotel Ch teau Laurier just outside the walls near the festival site. It was really nice -- modern, sleek, and pretty affordable (we got a big double-queen room for $150). Then we walked around the Upper Old Town in on-again-off-again rain, and watched some of the street performers. There was a troupe of stilt-walkers dressed like stone angels and gargoyles -- an awesome performance that scared the bejeesus out of Amita June. But as they left she said, "Bye bye, monsters," so I guess they weren't all bad. We had dinner at the cozy Moine chanson, a really nice restaurant just outside the walls of the Upper Old City. Quebec's restaurant scene in this are is, well, pretty dreadful -- lots of mass-production restaurants that serve tourists mechanical food from conveyor belts. The Moine was an accidental find, and it was really a godsend -- pleasant, interesting pan-Mediterranean food. I had grilled sardines (memories of Portugal); Brian had a thoughtful paella and Maj had a roasted vegetables with polenta plate. They also had a choice selection of organic wines. I have preconceptions about organics, I admit, but we had a very tasty and complex organic Douro from Portugal last night that set me back in my seat. We walked home through the music festival just as Nickelback was coming on the main stage. Brian wandered a bit, Amita June and Maj and I went to bed. Unfortunately, that wasn't the last time we were up: AJ woke up vomiting in the middle of the night. She only barfed once, and we're not sure what caused it: whether she overate at the restaurant (she packed away a lot of food), or if she got some sausage or chicken in the couple of cupfuls of Brian's paella she ate. Either way, she was a sad toddler and we were freaked-out parents. Having your child get sick while you're traveling is really scary. You start thinking about helicopter rescue. But she was fine after one big purge, and we all slept well until this morning. Maj got up to change Amita's diaper, and we realized we didn't have any. Not in the hotel room, not in the car. I went out from the hotel in increasingly wide spirals looking for a convenience store, but those that had diapers weren't open on a Sunday morning, and those that were open didn't carry diapers. After a couple of hours (!), I came back to the hotel in defeat. Fortunately, Maj managed to find a diaper in the emergency stash in the trunk of the car. We picked up Brian and drove to another part of town, where we got a tasty brunch on the terrasse and found some diapers at the Jean Coutu drugstore. Brian kept going from Quebec City -- he's heading all the way around the Gasp Peninsula to New Brunswick, Nova Scotia and Prince Edward Island, and coming back to Montreal in about a week. Maj and I drove home on the South Shore -- Amita fell asleep as we hit the freeway and woke up when we pulled into the driveway -- and had a nice afternoon at home. tags:

Seafood windfall On Friday, our friend Sebastian called us with a problem: his sister, who lives in the les-de-la-Madeleine, had sent him too much fresh seafood. It had come out of the sea and been driven to Montreal, and there was too much for him and his family to eat. The fish was less than 24 hours out of the water -- could we help him out? So we ended up with two huge bags of succulent lobster meat and three more of scallops. Oh, man, were they good. On Saturday morning before we left for Quebec City I made up two giant lobster omelets for the four of us to share. They had just huge chunks of lobster in them, and it was the most decadent meal I've had in a while. It was so much, we all lay around on the couch moaning for about an hour. Tonight, I cooked up the scallops. They were gigantic, tender, and fresh, so I pan-seared them in butter and olive oil with salt & pepper and prepped garlic and oregano. I know, that seems too much for such a delicate meat, but it worked out great. I put them on a bed of whole-wheat linguine, arugula, and red peppers. They were really, really good. I don't think I'm going to have seafood this good again for a while. Mmmmm. Thanks, Sebastian! tags:

News in brief from the land of the Free Two Freedom news items: tags:

Matthew Palmer: Britney Spears explains Garbage Collection

... in song, no less! And for the few people in the world who haven't seen it before, Courtney Love explains the basics of BGP.

David Moreno Garza: Back

Yeah, we are almost two weeks back now, but until now I can feel we are done with lots of pending issues.

To summarize, I left DebConf on Saturday morning, arrived NYC on the afternoon where Raquel was waiting for me. By Sunday, we had the very nice chance to meet with abrotman, rcox and stew. We stayed at stew and rcox’s place until Tuesday, when we got back. We also had the chance to meet with people that we have been working for remotely. We were walking around in NYC, and suddenly, the 5th avenue became multicolor. It was the gay pride parade. You can take a look at the pics on the gallery, lots of gay people. Our visit to NYC was pretty gayish actually, since we spent our first night in Chelsea, presenced an electronic music festival or similar for lesbians, and the next day, first row on the parade :-)

And that’s it, since we came back until like yesterday, life was a mess; lots of work pending to deliver and many other issues. Finally, we are right now settled up again, enjoying a cold rainy nice weather in Mexico City with very cheap beer (compared with UK pints!) and loved people :-) Thanks to all who made such a wonderful DebConf, even that, in my case, I arrived late and left early, it was just great (thanks also to MJ Ray, specially; and also to all people who was worried about my arrival, you know who you are, thanks). Thanks also to Mike and Rachel who hosted us, they are a very nice couple. And also, thanks to Alex who joined us in NYC, coming from Philadelphia, spent a nice day with us, took us to Chinatown and Little Italy and gave me a Phillies baseball cap (just as exchange of the worldwide famous umbrella hat).

8 July 2007

Mirco Bauer: Hello World, how small can you get?

Today I was checking how much diskspace is need when a small CLI (.NET) application like gfax is installed. The last 2 years I am optimizing the Mono packages in Debian to reduce dependency chains. Dependency chains can cause a small application to become very big (at least it looks that way) when they are installed, as modern Linux distributions automaticly install all required dependencies. So first I worked against GUI dependencies for non-GUI applications, those are annoying for servers, you don't want to install a X server or other GUI libraries when you only serve webpages using ASP.NET.
Splitting CLI 1.0/1.1 vs 2.0 also seemed to be a very good candidate to split apart, as applications usually are target at 1.0/1.1 or 2.0.
Then I split Base-Class-Libraries that have dependencies on native libraries (via P/Invoke), as those packages must depend on those native libraries, like DB drivers.
Last but not least I split debug symbols in a different package, as those are not needed for normal production usage, and embedded systems don't have much diskspace. The mono-dbg package is 28.2MB in size when installed, that's alot.
The Mono source package in Debian is now building 67 binary packages, but thanks to APT nobody has to care for that, every part is installed when needed. So today I was checking the results of it, how much will a small application download and install?
I am testing with a clean minimal Debian system, that way all dependency chains will show up very easily.
Installing GFax + required libraries today takes 43MB to download and 138MB on the harddisk. GFax is a small GNOME application, it uses libraries like: Mono, GTK#, GNOME#, Evolution#, GConf#.
So someone might say: "of course, it is based on that bloaty Mono virtual machine!", well let's check a different application.
What about a GTK# only application (no GNOME) like graphmonkey + required libraries? That will download 19.5MB and uses 56.4MB on the harddisk. That's alot less, isn't it? graphmonkey only uses Mono and GTK#.
So it's not Mono taking all no diskspace? Well after all my packaging optimizations, it's not Mono smile.png
So how big is a install of a minimal Mono runtime + all required libraries then?
Minimal as in enough to run famous the "Hello World".
It's 2.4MB to download and 7MB on the harddisk, yes you read it correctly, the super monster bloaty virtual machine just needs 7MB. Mono only has glib as dependency which will be replaced by the eglib library some day. After I had this great result, I wondered how big other runtimes are.... like.... Java and Python? The results are interesting...
A minimal Python install takes 3.9MB to download and 13.4MB on the harddisk. Yes, that's almost double the size of Mono, I was surprised too.
So and Java? Here it becomes a bit unfair as SUN's license is not allowing to ship parts of a Java runtime. So the Java package in a distribution is one package. 34.5MB to download and 95.2MB on the harddisk. Yes, that's alot, but thats the smallest install you can get when you want to run anything on it. So, if you want to run Hello World on a virtual machine: use Mono! tongue.png just kidding...
Small runtimes are important when it comes to acceptance and adaption of new technologies, as nobody wants to kill all his computer resources for it. And I often hear the replies on Linux IRC channels, when people ask which software they can use to do $task: "No I don't want o install 100MB just for $task" and they decide to try some other software with less dependencies and less resource usage.
Also for the embedded area, resources are very limited, every MB still counts there. For the completeness of my report, here the naked numbers of other available Java runtimes: sablevm 22.0MB (63.2MB), kaffe 52.2MB (118MB), jamvm 28.3MB (65.4MB), cocoa 28.8MB (66.0MB).

Jose Carlos Garcia Sogo: Tagging entries

I am using PyBlosxom as my blogging software. It is quite simple and thus I love it. I think it is a bit overkill to use MySQL for writing some few entries (as wordpress does). As it is simple, it uses file text entries, with a directory based hierachy, so your entries are only going to one of those categories you have defined. Of course this is little flexible; you cannot file a entry in two oro more different categories. This leads you to have quite generic categories, in which almost everything can fit. I have looking today for a tagging system for PyBlosxom and I have found a folksonomy tagging plugin which serves my needs fine. Welcome tagging world to my blog!

Miriam Ruiz: Hex-a-hop, a puzzle game in an hexagonal world

Sometimes wandering through the netsphere you suddenly find small pearls you inmediately fall in love with. This is the case of Hex-a-hop, a small little cute puzzle game in which you move a girl through an hexagonal map, trying to break all the green tiles. The game has no time limit of any kind, and it does not involve any kind of real-time issues either. The visual appearance of the game is really great, a lot of care has been taken in the details, the graphics are really sweet, the water moves slowly reflecting the world, it really feels nice to play. For the moment, I’ve put a package available for Debian SID, until it goes into Debian’s repositories, in case someone wants to try it. Tom Beaumont, the author of the game, has really done a nice work :)

Bastian Venthur: Reportbug-NG finally in testing!

Waiting ten days for rng to slip into testing, I had enough time to stuff the next version with new features and small bugfixes. The most interesting ones are: In totally unrelated news, I finally managed to master Rubenstein’s Revenge.

Matt Brown: POSIX/NFSv4 ACL Inheritance Problems

I (as root) have a directory hierarchy that I want a particular group to always have write access to. The files and folders inside the hierarchy are owned and manipulated by a wide variety of diffrent users. Essentially I want to delegate ‘root’ access for a portion of the filesystem to a particular group. My first attempt at implementing this was to use the standard POSIX ACLs that are available for almost every filesystem Linux supports. I recursively set an ACL on the top-level directory to give the group write access to all files and directories that currently exist and then I recursively set a default ACL to give the group write access on all the directories. This default ACL should be inherited by any new files that are created ensuring that the group keeps write access to everything. Problem solved? Unfortunately not. The intricacies of complying with POSIX means that ACLs are implemented as an ACL plus a mask. To gain access to a particular file or directory the user or group must match an appropriate ACL granting the access and the mask for that file or directory must also allow the requested permission to be granted. When you add an ACL to a file or directory, the ‘group’ bits of the standard Unix permissions magically switch from controlling group access to controlling the mask portion of the ACL, effectively providing an upper bound on the permissions that an ACL entry can grant. This prevents legacy POSIX applications that do not understand ACLs from unintentionally granting excessive permissions - arguably a good thing. Unfortunately this also makes it very hard to preserve the ACL granting write access to the ‘root’ group which I legitimately intended to have in place on this portion of the filesystem. Newly created files under the hierarchy generally inherit the ACL as intended, as most applications attempt to create files with as many permissions as possible, leaving it up to the umask to remove undesired permissions. However any file that is copied into the hierarchy without the ‘group’ write bit set, or any file that has the ‘group’ write bit removed via chmod will actually remove the write bit from the ACL mask invalidating the ACL and leaving me back at square one! After a bit of Googling I thought that NFSv4 ACLs might be the answer to this problem, as they are marketed as “very similar to Windows ACLs” and I’m sure that I vaugely recall Windows being able to properly inherit ACLs from parent directories. Unfortunately after downloading the NFSv4 ACL patches and trying all the various mount options I cannot find any combination that will offer the functionality I need. The implementation conforms to POSIX, so it still has a mask parameter and the same problems as the standard POSIX ACLs. The only benefit from using NFSv4 ACLs that I can see is that you have more permissions to grant. So once again, I’m back to square one. I’m hoping that there is some fundamental point that I’m missing as this seems like a very common use-case that I would have thought would be well supported. If a command-line example is clearer to you look at:
http://www.mattb.net.nz/blog/dump/acl-inheritance-problems.txt My current solution is to run a cronjob every X minutes to recursively ‘chmod -R g+w /dir’, however that’s far from optimal as it exposes all sorts of race conditions and just seems ugly! Any suggestions or solutions will be gratefully received.

Steve Kemp: BC

If you use xen-tools and you've ever wanted to be able to setup arbitrary partitions within your new guests you can do so now.
skx@vain:~$ sudo xen-create-image  --boot --hostname=pt.my.flat --partitions=sample-server
...
...
General Information
--------------------
Hostname       :  pt.my.flat
Distribution   :  etch
Partitions     :  swap            2G    (swap)
                  /               1G    (ext3)
                  /var            4G    (xfs)
                  /opt            1.5G  (xfs)
                  /usr            4G    (xfs)
                  /tmp            0.5G  (xfs)
                  /home           1G    (xfs)
                  /var/tmp        1.5G  (xfs)
...
How cool is that? Many thanks to Sascha Kettler for the patch.

7 July 2007

Sune Vuorela: paranoia++ ?

Hi! Earlier I blogged about being too (little) paranoid. One of the comments was my unencrypted /boot on my harddrive. So I did something to it. Daniel Baumann told me some stuff at debconf about how to make bootable cd’s with grub - so I had to try to move my /boot to a cdrom instead. It works. A quick and dirty howto:
mkdir -p builddir/foo
cp -r /boot builddir/foo
cd builddir
genisoimage -no-emul-boot -boot-load-size 4 -boot-info-table -r -b boot/grub/stage2_eltorito -o boot.iso foo
and now you are not yet up and running. It looks like it, but you have to adapt boot/grub/menu.lst to make it boot from cd. First issue: boot fails with a “Error 29: Disk Write Error” - I wondered a bit and asked for help around and wondered a bit more why grub wanted to write to my harddrives in order to boot … finally solved it with help from the Super Grub Disk Page - it was the savedefault point of all my kernels. Removing that made it boot. So time to remove /boot and trust my cd. Which was a wrong decision. All my kernels had root=(hd0,0) so it kind of didn’t work. Changing this to root=(cd) did it - and removing (hd0,0) from my splashimage was also a bit nessesary. So after this, rebuild the cd again with the genisoimage command mentioned above and now I am up and running completely. And I of course have secretly marked the cd so evil people can’t replace it without me noticing it. So what’s next step in my tinfoil covered world ? SELinux? something else? I guess you know how to make comments. Feel free. /Sune

Rob Taylor: Moving..

This week I’m going to be moving house to the lovely Hope Valley in the Peak District. Its an exciting move and I’m looking forward to going bouldering and climbing, but there’s a terrible side-effect - I’m missing Lug Radio Live. I’m insanely jealous of everyone who can go - I know Jono will show you all a wonderful time :) I’ll also have to miss the warm up days of GUADEC but I’ll be there from the core days till the end. Can’t wait to see everyone there!

Romain Francoise: Shared links for 2007-07-07

Mike Hommey: Finding where a tarball came from with git

I got the idea reading this question on the webkit-dev list, and from my recollection of the git documentation. Well, for the webkit case, the basic script I put up would be of no help because the tarballs don’t contain all the files (plus, they use subversion, not git, so it would also require a long importing process). Anyways… Considering how SHA-1 hashes of objects are created with git, it is actually pretty easy to generate a SHA-1 hash from a random (non-git) tree, and then, the corresponding commit. First, you have git-hash-object that helps you creating a hash for a particular object (though it’s also trivial to do with sha1sum). For regular files, git-hash-object -t blob $filename is enough. For symbolic links, you have to read the link destination, and give it without a trailing character (be it the NULL character or a carriage return) to git-hash-object -t blob --stdin. For directories, you have to generate a tree “structure” by yourself and pass it to git-hash-object -t tree --stdin. I haven’t bothered looking at other file types. The tree structure can be guessed by either looking at mktree.c or at the output of git-cat-file tree $sha1 where $sha1 is the SHA-1 hash for a tree object. It contains 3 informations for each node in the tree : the file mode, with the same format as what stat() returns, except for some reason, permissions are 000 for directories and symbolic links ; the file name ; and the SHA-1 hash. These informations are written with the following format : file mode in octal ascii and no padding zero (”%o”) followed by a space character, then the filename followed by a NULL character, and the binary form of the SHA-1 hash. Nodes are sorted in a not-so-quite lexical order (take a look at base_name_compare in read-cache.c) and are not separated by any special character: the mode of a file just follows the SHA-1 hash of its predecessor. With all this new knowledge, you should be able to write some code that would return the SHA-1 from an arbitrary directory. Okay, since you must be at least as lazy as I am, you can take the script I wrote. Now, let’s take a look at a real life case : what commit is the latest nightly snapshot for the linux kernel from ? First, download the latest snapshot patch and its baseline, and extract the whole. Then, run my git-hash-tree.pl script with the directory containing the extracted kernel as an argument. It will return, after a while, the SHA-1 hash for the whole tree. During this long process, you also have plenty of time to git clone linus’s tree. Once you’re all done, you can search for the commit corresponding to the tree hash (let’s call it $hash) with the following command :

git-rev-list --all while read h; do git-cat-file commit $h grep -q "^tree $hash" && echo $h && break; done
If you just followed these steps, you should just have spent a great moment having no result at all. There are actually 2 things that prevent this method to properly work with the linux kernel nightlies : Note this is a naive method, because I haven’t dedicated much time going through git documentation and code to find better ways, if there are any. Also note it’s pretty much worthless to do this with the kernel nightly snapshots, since a file containing the SHA-1 hash of the corresponding commit can be found alongside the patch. I guess a similar method could be used with mercurial, though I could not find a documentation detailing what are the hashes calculated from (I’ve not searched a lot, I must say, but for git, it was just before my eyes).

Daniel Stone: google search shortcuts

I'd always had the ludicrously useful Google Search Keys extension installed, but now Google's Beta Search Keys complement this incredibly well (complete with vi mode!). Can't imagine life without the extension, let alone without the new Google one ...

(Via Adam Rosi-Kessel.)

Meike Reichle: More Pancakes


a plate with a half eaten pancake
Following the invitation on the SocialContractTenYears wiki site. We also decided to join the distributed pancake party. So on the evening of July 5th we joined the many other pancake eaters. Pancakes where prepared and much enjoyed with a whole lot of toppings. A nice party! PS: In fact, following the composition of this blog post pancakes where had again. Now, that's the Good Life!

Next.

Previous.