Search Results: "zed"

20 October 2009

Benjamin Mako Hill: Updating the Ubuntu Code of Conduct

The Ubuntu Code of Conduct is one of the most surprisingly successful projects I've ever had the privilege of working on. On my first day working for the company that would become Canonical, I talked with Mark Shuttleworth about some ideas for community governance. Partially in reaction to some harsh behavior in other free software projects we'd worked on, Mark and I agreed that some sort of explicit standard for behavior in Ubuntu would be a good thing. Over lunch of what was my literally first day working on Ubuntu, I wrote a draft of code of conduct that was essentially the version that Ubuntu has used until today. Shuttleworth made a series of modification to my draft but I don't think either of us took it too seriously. We figured it would be easy to update it later. Over time, that code has become a central piece of the Ubuntu community. Every new Ubuntu member cryptographically signs the code. When conversation in any Ubuntu forums, channels, or lists becomes disrespectful, users almost instinctively remind each other of the code. Through this process, the code has become a sort of constitution of our community and a widely enforced standard. People treat the code as a reflection of what "ubuntu" --- both the concept and our project --- stands for. Over time, the original code has spawned a Leadership Code of Conduct (which I also worked to draft), and has been modified and employed by scores of free software projects and by many projects that have nothing to do with free software at all. This is all wonderful, but a side effect has been that updating the code has become a more a difficult process that we originally imagined. Despite it success, the code remains a text written in an afternoon in Mark's flat. At times, this fact shows. For example, the code contains some off-hand humor that now seems a little akward and the text was a bit too developer centric at points. And there was a lot that, quite simply, we would have done better if we had realized that the code would be so important. So this summer, Daniel Holbach and I spent another afternoon in Berlin discussing and crafting a new version of the code along with a detailed rationale document that describes all the things we'd changed and why. We believe that what we've created is fully in the spirit of the original code. We've made efforts to minimize the delta in terms of text as possible. Daniel and I realize that changing the code out from under our community is a dangerous game, and we've make exceptional efforts to make sure that the new code doesn't say anything substantively different than the old code --- but that it does say it better. So I'm thrilled that, after being posted since early June and after incorporating a series of revisions with members of the Ubuntu Community Council, the new draft was approved at a council meeting earlier today. Of course, we are continuing to think about how we might improve the text going forward. One important goal we've thrown around, for example, is the creation of a code that is no longer Ubuntu specific and that can be employed by a wide range of different groups and different free and open source software projects.

19 October 2009

Jaldhar Vyas: Sal Mubarak 2066

Best wishes to everyone in Debianland for a happy and prosperous Gujarati New Year (Vikram Samvat 2066, called Plava) This year the local mandirs continued their recent trend of coordinating their Diwali/new year plans and there was a big mela planned. Unfortunately the weather didn't really cooperate but the kids got to manhandle some explosives and after all isn't that the reason for the season? Presentwise, I got a DIY 4-shelf bookcase from Sears, the Catalyst Book plus a much-needed woolen hat. I've decided that I am going to quit my job. It's a soul-sucking dead end but in these tough economic times it pays bills so I've agonized about this for a long time. This was supposed to be the year we moved out of Jersey City but it has yet to happen. And it never will unless I take some sort of decisive action. So I'm going to do it. Wish me luck!

15 October 2009

David Moreno: Debian Workshop 1 at New York City

So Debian-NYC, mainly the DebConf10 local team, organized a workshop for people interested on Debian packaging. The nice people of drop.io offered their handy location in Brooklyn. So this is the first attempt to bring new comers to Debian aiming DebConf10 next summer. Micah and dkg are the ones leading the workshop so far. Pictures!

14 October 2009

Christian Perrier: Virtualization (take 2)

Thanks to everybody who went through the pain of my blog (which doesn't allow comments because I don't want to bother with blog spam) and still cooked up a private reply to me. It's very much appreciated. So, Patrick, Dan, Dirk and No l, you just won a free beer for your wise advice.....except the two marathoners among you guys who shouldn't drink beer (though it would help you slowing down enough to be able to run along with me). In short, all this can be summarized to "Christian, you're going the wrong way with Xen, use KVM". One of the advices was using VirtualBox OSE, which is certainly a wise advice as well...but not what I want to experiment (I already use VB for the few D-I work I'm still doing and that would not be a new experiment for me). I'll go your proposed way and will let you know. Still, I'd love being able to figure out Xen stuff as this is what we're more and more using at ONERA.

11 October 2009

David Pashley: mod_proxy or mod_jk

This entry was originally posted in slightly different form to Server Fault There are several ways to run Tomcat applications. You can either run tomcat direcly on port 80, or you can put a webserver in front of tomcat and proxy connections to it. I would highly recommend using Apache as a front end. The main reason for this suggestion is that Apache is more flexible than tomcat. Apache has many modules that would require you to code support yourself in Tomcat. For example, while Tomcat can do gzip compression, it's a single switch; enabled or disabled. Sadly you can not compress CSS or javascript for Internet Explorer 6. This is easy to support in Apache, but impossible to do in Tomcat. Things like caching are also easier to do in Apache. Having decided to use Apache to front Tomcat, you need to decide how to connect them. There are several choices: mod_proxy ( more accurately, mod_proxy_http in Apache 2.2, but I'll refer to this as mod_proxy), mod_jk and mod_jk2. Mod_jk2 is not under active development and should not be used. This leaves us with mod_proxy or mod_jk. Both methods forward requests from apache to tomcat. mod_proxy uses the HTTP that we all know an love. mod_jk uses a binary protocol AJP. The main advantages of mod_jk are: A slight disadvantage is that AJP is based on fixed sized chunks, and can break with long headers, particularly request URLs with long list of parameters, but you should rarely be in a position of having 8K of URL parameters. (It would suggest you were doing it wrong. :) ) It used to be the case that mod_jk provided basic load balancing between two tomcats, which mod_proxy couldn't do, but with the new mod_proxy_balancer in Apache 2.2, this is no longer a reason to choose between them. The position is slightly complicated by the existence of mod_proxy_ajp. Between them, mod_jk is the more mature of the two, but mod_proxy_ajp works in the same framework as the other mod_proxy modules. I have not yet used mod_proxy_ajp, but would consider doing so in the future, as mod_proxy_ajp is part of Apche and mod_jk involves additional configuration outside of Apache. Given a choice, I would prefer a AJP based connector, mostly due to my second stated advantage, more than the performance aspect. Of course, if your application vendor doesn't support anything other than mod_proxy_http, that does tie your hands somewhat. You could use an alternative webserver like lighttpd, which does have an AJP module. Sadly, my prefered lightweight HTTP server, nginx, does not support AJP and is unlike ever to do so, due to the design of its proxying system.
Read Comments (3)

10 October 2009

Jaldhar Vyas: Barack Obama is a Keynesian

pointless award I haven't written about politics since the election for several reasons. First and foremost, Obama won fair and square. While I might wish the other guy had won, I can't whine and pout or threaten to move to Canada like some people. The vox populi chose and that choice has to be respected. Secondly, one of the worst aspects of the current American political scene is the paranoid obsession with personalities in which the president is not just an implementor of policies good or bad, but personally responsible for the entire state of the cosmos. It started with Nixon, started getting alarming with Clinton and reached batshit levels of insanity with George W. Bush. But just because the left does it it, doesn't mean we have to follow suit. Although I figured Obamas policies would end up a trainwreck, I wanted to see what he actually did before calling him on it. The Nobel comittee isn't hoping and changing with me. In a decison that is bizarre even by the lax standards of a prize that long since jumped the shark, the President is to receive the Nobel Peace prize for...not being George Bush? It's difficult to see what substantive reasons they could have for their choice. But apparently one of these days he is going to make extraordinary efforts to strengthen international diplomacy and cooperation between peoples...democracy and human rights are to be strengthened. Well let's see now: Perhaps I am being a little unfair in not taking into consideration all the great things the president is going to do real soon now. It's just that I'm worried that this might be an attempt to sway him in the task of winning the war in Afghanistan (and more likely than not carrying it into Iran and Pakistan.) I take solace in the knowledge that heretofore Obama has shown a refreshing inclination to throw people under the bus when they are no longer convenient. If he can do that to the guy who says he wrote his book than some silly Norwegians won't be a problem.

8 October 2009

Barry Hawkins: More Expensive and Complicated Equals Better: Carseats and Software

So I finally got around to checking out the TED site; I ve quickly become a fan. One of the first talks I watched was Steven Levitt s child carseats talk. Both the talk and the feedback comments on the TED site reminded me of things I see in software development. Here s the video if you haven t seen it.
<object height="326" width="334"><param name="movie" value="http://video.ted.com/assets/player/swf/EmbedPlayer.swf"><param name="allowFullScreen" value="true"><param name="wmode" value="transparent"><param name="bgColor" value="#ffffff"><param name="flashvars" value="vu=http://video.ted.com/talks/dynamic/StevenLevitt_2005G-medium.flv&amp;su=http://images.ted.com/images/ted/tedindex/embed-posters/StevenLevitt-2005G.embed_thumbnail.jpg&amp;vw=320&amp;vh=240&amp;ap=0&amp;ti=30&amp;introDuration=16500&amp;adDuration=4000&amp;postAdDuration=2000&amp;adKeys=talk=steven_levitt_on_child_carseats;year=2005;theme=unconventional_explanations;theme=presentation_innovation;theme=how_the_mind_works;theme=not_business_as_usual;event=TEDGlobal+2005;&amp;preAdTag=tconf.ted/embed;tile=1;sz=512x288;"><embed allowfullscreen="true" bgcolor="#ffffff" flashvars="vu=http://video.ted.com/talks/dynamic/StevenLevitt_2005G-medium.flv&amp;su=http://images.ted.com/images/ted/tedindex/embed-posters/StevenLevitt-2005G.embed_thumbnail.jpg&amp;vw=320&amp;vh=240&amp;ap=0&amp;ti=30&amp;introDuration=16500&amp;adDuration=4000&amp;postAdDuration=2000&amp;adKeys=talk=steven_levitt_on_child_carseats;year=2005;theme=unconventional_explanations;theme=presentation_innovation;theme=how_the_mind_works;theme=not_business_as_usual;event=TEDGlobal+2005;" height="326" pluginspace="http://www.macromedia.com/go/getflashplayer" src="http://video.ted.com/assets/player/swf/EmbedPlayer.swf" type="application/x-shockwave-flash" width="334" wmode="transparent"></embed></object>
Steven Levitt shows in his talk how 30 years of data on car crash fatalities imply that carseats do not outperform regular seatbelts for children ages 2 and up. Anyone who has a child or grandchildren will probably bristle at hearing that; as the father of a pre-schooler, it certainly gave me pause. We spent no small amount of time and decent chunk of money in selecting carseats for our child, thinking we had done our best to ensure our child s safety. For that matter, it would be illegal for us not to have done so. To see a decent-sized data sample suggesting my child would be better off in a seatbelt at her age is rather unsettling. By the end of the talk, I took away these observations: So, software professionals, does any of that sound familiar? It reminds me of numerous initiatives over the years that have led us inexorably to the software productivity morass we have slogged through for years now. I suppose the most heinous case study in my own experience would be J2EE and the insistence that it was the solution that any reasonable business application would choose for a platform. (Before you .Net folks jump on that one, DCOM and later the .Net enterprise stack was much the same.) And who hasn t read a benchmark or white paper with seemingly incontrovertible data depicted in highly-polished graphics insisting that Product Y is the one solution to address them all? I noticed that there were comments below the video on its page at the TED site, largely because the negative verbiage of the topmost comment jumped out at me. I took the time to read through them all (there were 36 at the time of this writing), and to my amusement, I saw parallels between them and how people react to questioning and examining our existing practices and means of developing software. See if any of these strike a chord: In recent years I ve been greatly encouraged by the willingness of companies to question whether or not the heavyweight frameworks and technology stacks are what they should be using. Helping companies slough off high-ceremony processes in favor of right-sized process that focuses on delivering the right software in a timely manner has been some of the more rewarding work I ve done the past several years. I think we have an encouraging number of people in the industry who are challenging the more expensive and complicated always equals better mantra. For the brave individuals willing to put these questions to the community at large, I hope you find some comfort in knowing that the resistance and rejection you will encounter is a thing to be expected; you are not alone in that respect. Here s hoping we continue to be open to self-examination, no matter what emotional responses it might provoke or what fear of the future it may stir up within us.

Joey Hess: ikiwiki dependency types

Like make(1), ikiwiki knows about dependencies between the files that it is building, and will rebuild a file if its dependencies change. I've always known that it was possible to greatly reduce the amount of work ikiwiki does, if it gets smarter handling of different types of dependencies. It would be as if make knew that changes to comments inside header files did not change the resulting binary, and avoided rebuilding it for such a change -- but did rebuild the doxygen documentation. Except that, in ikiwiki, this can be done sanely, since it figures out the dependencies by examing the wiki's files, not by using a Makefile. Also, there were some nasty bugs with certian complex dependencies not being handled right. Fixing one of them, the transitive dependency problem, would require ikiwiki doing significantly more work. So fixing all this has been on my todo list for a while. Though when I mentioned "transitive dependencies" at the ikiwiki BOF @ DebConf, eyes glazed over. :) And I expect I've lost most of my readers already here too.. Which is a pity, because I've just gotten to the fun stuff. So, a few (how many?!) days ago I started my current insane hackfest, and began adding dependency types into ikiwiki. The first special type was a presence dependency -- a dependency that only cares if a page is created, or deleted, but does not care about changes to the page. Next type was a links dependency -- a dependency that only cares if the page changes what pages it links to. That was great, except it often didn't work. For features like gathering "posts/*" together into a blog, or displaying a tag cloud of "tags/*", ikiwiki uses a PageSpec. This is a small, domain specific language, that can specify a set of pages with lots of control. For example, "(index or backlink(index)) and created_after(oldpage)". The problem is that when using a PageSpec like the one above in a presence dependency, it doesn't only matter if pages are added/removed. Certian changes to "oldpage" or "index" can also change what it matches. So now, I had this PageSpec DSL, that I needed to somehow analyise to find these influences. While thankfully not turing-complete, parsing it to extract the influences would still be annoying. Except: Operator overloading to the rescue! The solution was this perl:
package IkiWiki::FailReason;
use overload (
        '""'    => sub   $_[0][0]  ,
        '0+'    => sub   0  ,
        '!'     => sub   bless $_[0], 'IkiWiki::SuccessReason' ,
        '&'     => sub   $_[0]->merge_influences($_[1]); $_[0]  ,
        ' '     => sub   $_[1]->merge_influences($_[0]); $_[1]  ,
        fallback => 1,
);
package IkiWiki::SuccessReason;
use overload (
        '""'    => sub   $_[0][0]  ,
        '0+'    => sub   1  ,
        '!'     => sub   bless $_[0], 'IkiWiki::FailReason' ,
        '&'     => sub   $_[1]->merge_influences($_[0]); $_[1]  ,
        ' '     => sub   $_[0]->merge_influences($_[1]); $_[0]  ,
        fallback => 1,
);
Ikiwiki already dealt with pagespecs by translating them to perl (which is then compiled (yes, perl can), and happily memoized). A pagespec is translated to something like this:
(match_glob("index")   match_backlink("index")) & match_created_after("oldpage")
The match_* functions return IkiWiki::SuccessReason or IkiWiki::FailReason objects. These objects are special in a lot of ways; they act like boolean values most of the time, but if viewed as a string, you get a user-understandable message saying why the pagespec didn't match. And when ANDed or ORed together, they build up a list of influences, that can be accessed at the end to get all the influences for the whole pagespec. (Why am I using what's supposed to be bitwise operations & and , rather than "and" and "or"? Because perl doesn't seem to consider the latter to be operators. Odd.) Anyway, this stuff should be available in an ikiwiki release just as soon as I've finished making it smoking fast.

5 October 2009

Christian Perrier: 42195 K lschmeter done!

Here we are back from our german week-end... The major event of this week-end was running the K ln Cologne marathon. Why Cologne? Mostly because my marathon Debian friends Ralf Treinen and No l K the ran it last year and enjoyed it. So, when we ran the Paris marathon together back in April, we decided to run either Berlin or Cologne later on. I finally settled for Cologne because this is closer from Paris and the date was more convenient. That was great also for No l, who lives close to K ln, so of course he planned that race in his (busy) schedule. Unfortunately, Ralf got tendinite problems since last Spring and could not join us. We really hope this will be temporary and we will enjoy running together in the future. So, I spent most of the week-end both visiting Cologne with Elizabeth...and running these 42,195 km. That was great, really. Even if there were much more Germans than in Debian, they were really as friendly and welcoming as Debian Germans are...:-). The race happens in Cologne center city and is well arranged for followers to be able to see runners several times, as the track comes back in the city center several times....with a memorable finish through the cathedral place, then central pedestrian streets and the bridge over the Rhein river (well, climbling it at km.42 is a small PITA, but still..). Animation nearly all along the streets with many K ln inhabitants and tourists encouraging the runners and shouting and applausing and playing percussions, and.... My own race was really as I planned to do it. I targeted 4h exactly (Paris was 4h10) and thus tried to keep a pace of 5'41"/km....which I nearly respected. I had a small alert around km13-15 when my legs where kind of heavy....and I was already imagining a terrible second part for the race. Finally, seeing Elizabeth encouraging me at km.19 in the middle of a big crowd, then passing through Friesen Platz in a great noise and with huge encouragements, just made it. I then completed the first half in 1h59'20", which was just fine. I slightly slowed down between km. 24 and 28, mostly because I was trusting my GPS watch to keep the pace...while it is indeed a bit inaccurate (its kilometers are about 950m long). So, when I realized that I was about 1-2 mins late at km. 30, I decided to accelerate a little and thus target 5'30"/km on the watch...which I could sustain up to km.38. I was fearing cramps and a very hard finish, just as it happened to me in Paris...but finally could finish in a quite good shape and keeping my pace. My final time is officially 4h00'47". So, I missed my target by very few seconds but I am really very happy with the result and the way I finished. Recovering in the hotel's sauna was amazing, after this, and I could then remember all the great moments of that race by viewing Elizabeth's pictures (soon online on my gallery, promise made). And, finally, today, we had another day off to drive back home and continue recovering....and register for Paris Marathon on April 11th 2010..:-)... Hopefully, No l (who did an amazing 3h15) will convince self that he should come with his running friends. So, dear readers, if you reached that part of my blog...I'm afraid you will again read about marathon stories from me.... As everybody was telling me: once you start, you never want to stop..:)

3 October 2009

Josselin Mouette: Sexism . Again.

I can t say I m surprised, but I m still appalled by reading, again, a completely out-of-reality polemic about a leader of the Free software community being called sexist (among other nice names), after an innocent and random word used in a talk. Oh noes! Serving in the navy will hurt! It s often that you hear jokes or references picturing French as arrogant, Italians as womanizers, geeks as bearded and smelling under the arms, sailors as gay, et c tera. It s called a clich , and it helps people understand each other because it s part of the collective imagination. If you insert a reference to an Italian in a speech about womanizers, it helps picturing the clich of a guy in a black Armani suit, wearing sunglasses and using gomina. It doesn t mean you think Italians are like this, and if it ever convinces someone that they are so, it means that someone really had trouble using the thing she calls brain to begin with. The same goes for a talk about people not being familiar to computers, that is using women as a clich . When you look for a picture in your mind of someone who has trouble with using computers, what do you find? A granny fighting with her keyboard which keys are not in alphabetical order, or a busty blonde following with her eyes each movement of the mouse cursor. It s only natural to use these pictures as a ground for communication, and it doesn t mean you think bad of grannies or busty blondes. And if, because of that or of anything else, some people end up thinking that women can t use a computer, or that they are not equal to men in this regard, these people are the sexist ones. Not the guy who drew that picture in a magazine you read 10 years ago and forgot since, nor the one using it as a reference. My intolerance is worth more than yours And instead of actually focusing on educating or slapping those people, some of us are conducting a witch hunt, in which anyone implying the very existence of different categories of people, called men and women, regardless of his merits or opinions, is called a sexist and has stones thrown to him on several public fora. This is not acceptable. Do not let your speech be dictated by intolerance. If you apply the same reasoning to anyone that could be possibly hurt, in a completely literal and simplistic analysis of the speech, you can stop right now using any kind of metaphor, idiomatic expression or proverb. All that remains is a poor, sterilized speech. And when the speech is poor, the ideas get poorer as well. Imagine what would a complete stranger reading those blogs understand. She would surely picture hackers as a completely misogynistic community in which people only talk about women (whether it is for sexist jokes or for rhetorical discussions about them) and never talk with them. Which would be utterly wrong, since hackers (and geeks in general) are on the contrary open-minded and friendly to feminism, compared to the rest of the population of course, except for the OMG OMG it s a woman! We need to be nice!!! kind. And this is actually the reasoning that hides behind all this insane RMS and Mark Shuttleworth are sexist trolls. The picture they are giving of women are that of fragile little things that need to be protected against any attack of all these evil misogynist bearded geeks. Pretty ironical for people who claim fighting for gender equality, isn t it? Questioning the unquestionable Frankly, RMS and Mark are among the people I trust the less in the community, but I don t need to paint them as misogynists for that. Look at what they say instead of how they say it: one is an intolerant man who used to be a visionary but turned his principles into dogma; the other has seen the light and behaves as if he was the savior of Free software. I d prefer if we questioned their vision and leadership based on their ideas and actions, not on distorted ways to interpret what they say based on a few people s neuroses. Or, to say it shorter: dudes, get a life.

1 October 2009

Daniel Kahn Gillmor: sexist behavior in the free software community

So not even 3 months out from RMS's sexist Gran Canaria virgins remarks, we have another powerful leader in the Free Software Community making sexist remarks in a talk to developers (this time, it's Mark Shuttleworth). It's a shame that these two people have said stupid things that hurt their causes and their communities by perpetuating an unfriendly environment for women. And it's a bigger shame that neither leader appears to care enough about their community to issue a sincere public apology for their screwup (if i'm wrong about this, please point me to the apology i've looked). These guys are in a situation which is nowhere near as hard as writing good software or managing complex technical projects: if you make a stupid mistake, own up to it, apologize, and try not to make similar mistakes in the future. Perhaps worst of all, are the remarkable number of unreasonably fucked-up comments on the blog posts discussing these unfortunate events. If you're in the habit of defending remarks like those made by RMS and Shuttleworth on the 'net, please take a minute and ask yourself a few questions: Please read the Geek Feminism wiki and blog. Even if you don't agree with everything on those sites (hey, it's a wiki! and a blog! you don't have to agree with everything!), people are at least trying to address the problem of sexism in our community there. Engage constructively and don't hide or ignore problems! UPDATE: I've asked the site admin to remove some trollish comments. If you want to use the comments on this blog to announce something like "i'd like for free software to be a boys-only club", that's fine, i'd like to know who you are so i can adjust my perception of you accordingly. Anything advocating or excusing rape and/or murder is way over the line and is not welcome here. i'm sure you can find some other place to troll. Tags: community, conduct, feminism, free software, sexism

Ingo Juergensmann: Debian Unstable - nomen est omen

Oh well... in the last days my machine crashed during the nightly backup. When I restarted it the next morning I realized two problems: the LVM is not recognized anymore and my X crashes on startup.
With the help of Tino Keitel (scorpi) I fixed the first issue by installing an older version of the lvm2 package (namely: lvm2_2.02.39-7_i386.deb) and libdevmapper1.02.1 (namely: libdevmapper1.02.1_1.02.37-1_i386.deb). Tino reported this in Bug #546817.

But there's still the Xorg problem left. Xorg.0.log reports:

CODE:
(II) RADEON(0): RADEONRestoreMemMapRegisters() :
(II) RADEON(0): MC_FB_LOCATION : 0xefffe800 0xefffe800
(II) RADEON(0): MC_AGP_LOCATION : 0xf87ff800
restore common
restore crtc1
restore pll1
finished PLL1
set RMX
set FP1
enable FP1
disable primary dac
disable TV
(II) RADEON(0): RandR 1.2 enabled, ignore the following RandR disabled message.

Backtrace:
0: X(xorg_backtrace+0x3b) [0x81314bb]
1: X(xf86SigHandler+0x51) [0x80c57d1]
2: [0xb7f32400]
3: X(xf86DiDGAInit+0x2f) [0x80f08cf]
4: X(xf86CrtcScreenInit+0x110) [0x80ed9e0]
5: /usr/lib/xorg/modules/drivers//radeon_drv.so [0xb7a2a114]
6: X(AddScreen+0x19d) [0x80712cd]
7: X(InitOutput+0x206) [0x80adfa6]
8: X(main+0x1db) [0x80719bb]
9: /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5) [0xb7bb37a5]
10: X [0x8071051]

Fatal server error:
Caught signal 11. Server aborting


Maybe this is Bug #546586? I've already detached the second display, but it still crashes. Pointers and solutions appreciated!

UPDATE: thanks for the points! X is now working again!

30 September 2009

Joey Hess: fractal directory structures

I was thinking about how I often end up in a directory path where the names of one or more components repeat. It seemed there should be a name for when this happens.
/home/joey/src/ikiwiki/doc/ikiwiki/
/home/joey/src/pdmenu/src/
/home/joey/src/joeywiki/joey/
/lib/modules/2.6.26-2-686/kernel/lib/
/proc/1/task/1/
Then I realized there was a name already: Fractal. While directory structures are supposedly hierarchical, different levels in the hierarchy are self-similar, and so sometimes the same name is duplicated, with a subtly, or vastly different meaning, at different levels in the same directory path. I had wondered if we were doing something wrong, that this happens so often. It is occasionally confusing. But aside from using a deep, nominally hierarchical directory structure, and displaying a slice through it (as if talking about electron orbitals in a rock orbiting sol), I don't think we're doing anything wrong. Here is a oneliner that will find all such directories inside a specified start directory. You will probably find some amusing ones you've never noticed.
perl -MFile::Find -le 'find(sub   if (-d $_)   my %bits; foreach $bit (split "/", $File::Find::name)   if (++$bits lc($bit)  > 1)   print $File::Find::name; return         , shift   ".")'

27 September 2009

John Goerzen: How To Record High-Definition MythTV Files to DVD or Blu-Ray

I ve long had a problem. Back on January 20, I took the day off work to watch the inauguration of Barack Obama. I saved the HD video recordings MythTV made of the day (off the ATSC broadcast), intending to eventually save them somehow. I hadn t quite figured out how until recently, so there they sat: 9 hours of video taking up about 60GB of space on my disk. MythTV includes a program called mytharchive that will helpfully transcode your files and burn a DVD from them. But it helpfully will transcode the beautiful 1920 1080i down to DVD resolution of on a good day 720 480. I couldn t have that. Background My playback devices might be a PC, the MythTV, or the PlayStation 3. Of these, I figured the PS3 was going to be hardest to accommodate. ATSC (HD) broadcasts in the United States are an MPEG Transport Stream (TS). Things are a bit complicated, because there may be errors in the TS due to reception problems, the resolution and aspect ratio may change multiple times (for instance, down to SD for certain commercials). And, I learned that some ATSC broadcasts are actually 1920 1088 because the vertical resolution has to be a multiple of a certain number, and those bottom 8 pixels shouldn t be displayed. Adding to the complexity, one file was 7 hours of video and about 50GB itself. I was going to have to do quite some splitting to get it onto 4.7GB DVD+Rs. I also didn t want to re-encode any video, both for quality and for time reasons. Attempts So, I set out to try and figure out how to do this. My first approach was the sledgehammer one: split(1). split takes a large file and splits it on byte or line boundaries. It has no knowledge of MPEG files, so it may split them in the middle of frames. I figured that, worst case, I could always use cat to reassemble the file later. Surprisingly, both mplayer and xine could play back any of these files, but the PS3 would only play back the first part. I remembered this as an option if all else failed. Next, I tried avidemux. Quite the capable program and I thought I could use it to cut my file into DVD-sized bits. But I couldn t get it to let me copy the valid MPEG TS into another MPEG TS it kept complaining of incompatible formats, but wouldn t tell me in what way they were incompatible. I could get it to transcode to MPEG4, and produce a result that worked on the PS3, but that wasn t really what I was after. Then, I tried mpgsplit. Didn t recognize the MPEG TS as a valid file, and even when I used a different tool to convert to MPEG PS, acted all suspicious as if it bought the MPEG from a shady character on a grungy street corner. dvbcut I eventually wound up using dvbcut to split up the ATSC (DVB) recordings. It understood the files natively and did exactly what I wanted. Well, *almost* exactly what I wanted. It has no command-line interface and didn t have a way to split by filesize, but I calculated that about 35 minutes of the NBC broadcast and 56 minutes of the PBS broadcast could fit on a single DVD+R. It worked very, very nicely. The resulting files tested out well on both the PS3 and the Linux box. So after that, I wrote up an index.txt file to add to each disc and a little shell scripting later and I had a directory for each disc. I started burning them with growisofs. Discs 1 and 2 burned well, but then I got an error like this:
File disc06/VIDEO/0930-inaug-ksnw-06.mpg is larger than 4GiB-1.
-allow-limited-size was not specified. There is no way do represent this file size. Aborting.
Eeeeepp. So apparently the ISO 9660 filesystem can t represent files bigger than 4GB. My files on disc 1 had represented multiple different programs, and stayed under that limit; and disc 2 s file was surprisingly just a few KB short. But the rest of them weren t. I didn t want to have to go back and re-split the data to be under 4GB. I also didn t want to waste 700MB per disc, or to have to make someone change video files every 15 minutes. So I decided to investigate UDF, the filesystem behind Blu-Ray discs. mkisofs couldn t make a pure UDF, only a hybrid 9660/UDF disc that risked compatibility issues with big files. There is a mkudffs, but it doesn t take a directory of its own. So I wrote a script to do it. Note that this script may fail with dotfiles or files with spaces in them:
#!/bin/bash
set -e
if [ ! -d "$1" -o -e "$2" -o -z "$2" ]; then
   echo "Syntax: $0 srcdir destimage [volid]"
   echo "destimage must not exist"
   exit 5
fi
if [ " id -u " != "0" ]; then
   echo "This program must run as root."
fi
EXTRAARGS=""
if [ ! -z "$3" ]; then
   EXTRAARGS="--vid=$3"
fi
# Get capacities at http://en.wikipedia.org/wiki/DVD+R as of 9/27/2009
SECSIZE=2048
# I'm going to set it a few lower, than capacity 2295104 just in case.
# Must be at least one lower than the actual size for dd to do its thing.
# SECTORS=2295103
SECTORS=2295000
echo "Allocating image..."
dd if=/dev/zero "of=$2" bs=2048 "seek=$SECTORS" count=1
echo "Creating filesystem..."
mkudffs --blocksize=2048 $EXTRAARGS "$2"
mkdir "$2.mnt"
echo "Populating..."
mount -o rw,loop -t udf "$2" "$2.mnt"
cp -rvi "$1/"* "$2.mnt/"
echo "Unounting..."
umount "$2.mnt"
rmdir "$2.mnt"
echo "Done."
That was loosely based on a script I found on the Arch Linux site. But I didn t like the original script, because it tried to do too much, wasted tons of time writing out 4.7GB of NULLs when it could have created a sparse file in an instant, and was interactive. So there you have it. HD broadcast to playable on PS3, losslessly. Note that this info will also work equally well if you have a Bluray drive.

John Goerzen: Jacob Is Three

Jacob s third birthday was Saturday, and I thought I ought to give a bit of an update on him. I ll start with his exactness. Jacob has become very particular about certain things. This morning, I was getting him dressed for church. We had this conversation:
Me: OK Jacob, it s time to get your pull-ups on. Jacob: No, Dad! Not pull-UPS. I don t have free (3) of them! I have one. It a pull-UP. Me: OK. Let s put it on Now, let s put on your pants. Jacob: Daaaad! Not pants! They SHORTS!
So yes, Jacob is trying to correct me about plurals. Also, when I sing Home On The Range to him at bedtime, and get to the line that says, and the skies are not cloudy all day, Jacob will look outside, and point out, the skies ARE cloudy today! (Even if they re not.) But the big thing is guitars. Terah wrote about how this happened. But suffice it to say that he s been almost all-guitar, all the time. At dinner time, he ll hold a spoon across his chest and strum the lots of pretend strings over the big end. Outside, he ll find leaves from irisis, and excitedly yell, I have LOTS of guitars! Pretty soon it wasn t just him that needed to play guitars. He d hand an iris leaf or a spoon to me and say, shall you play it too? So I d start strumming my leaf, and he d smile happily for a second. Then, No, not like that! This way! He meant I was supposed to hold it in a certain position, but it was not always clear exactly what he wanted. Then at supper Friday night, we had some family here, and Jacob said, Shall we ALL play guitars? And he d keep repeating it until we all did it. And he spent about 5 solid minutes getting us each to hold our spoons in the proper musical position. Then he was happy. So you can imagine the excitement when he got a guitar from his grandparents on Saturday. It s a child-sized one, but a real one, with metal strings. He proudly went around sitting on many different chairs to try out his guitar on each one. He played it in the car on the way home. He strummed it at home. He skipped dessert so he could play. And he cried every time he had to put it down: at dinner time, at bedtime last night, when it was time to go to church today. The next thing he wanted was for everyone to sing while he played the guitar. That was a bit difficult, since, after all, it is a 3-year-old playing a guitar he doesn t know how to play. He seemed to really enjoy it though. I have some great photos of him playing his guitars I ll be uploading hopefully this week.

22 September 2009

Bernd Zeimetz: cupt rocks

Although I'm a really happy cupt user for several weeks now, I have to admit that I didn't read the manpage until a few minutes ago. But after being annoyed by apt-get source again, I thought that chances are good that cupt will help me. And so it was! Now there is a new alias in my zsh config:
alias tget='cupt source --tar-only'
Very useful if you work in teams which keep only the debian directory in their vcs and you need to upload a new revision of a package. Another thing that really rocks is the cupt shell. Give it a try! And don't forget to read cupt(1).

20 September 2009

Martin Meredith: Ubuntu Global Jam Birmingham (UK)

The Birmingham Bug Jammers at the Last event

The Birmingham Bug Jammers at the Last event

The Ubuntu Global Bug Jam is just around the corner, and once again, the kind folks at the Linux Emporium have kindly offered the use of their offices to the Ubuntu UK team. Last time, we focused on bugs, and this time it s only slightly different. Instead of it being a Bug Jam it s just going to be a Jam . There will be people on hand to talk to about Bug Triaging, packaging, translating (and I hope we ll get the whole of Ubuntu translated into English by the end of the Jam), and anything else you can think of. So, if you re interested in helping make Ubuntu a better distro, and live in (or can travel to) the Birmingham Area in the UK, then you re welcome to come along. The Jam is on the 2nd-4th October, and we ll be around on all three days. If you need more info, please feel free to email me on <mez AT ubuntu DOT com>. Last year we had a great time, with refreshments (including beer) provided. Juggling penguins, and one person sleeping underneath the conference room table overnight. Hopefully we ll also have the live Video Streams up and running again for anyone who can t make it (or other Ubuntu Jams) to join in with! You can also find some blog posts about our last bug jam

Gunnar Wolf: Toast to Turing

I was pointed to this Toast to Turing, by Matt Harvey. Very much worth sharing.
Here s a toast to Alan Turing
born in harsher, darker times
who thought outside the container
and loved outside the lines
and so the code-breaker was broken
and we re sorry
yes now the s-word has been spoken
the official conscience woken
very carefully scripted but at least it s not encrypted
and the story does suggest
a part 2 to the Turing Test:
1. can machines behave like humans?
2. can we?
What, don't you know who Alan Turing was? Read a bit on him then, one of the core seminal minds for Computer Science. And a scientist vilified for being different from what is regarded as normal. [update] And answering to some people's doubts: Why this toast? Because the UK Government, in the voice of the Prime Minister Gordon Brown, after over 50 years of leading Alan Turing to commit suicide due to criminally accusing him for gross indecency for being a homosexual and forcing him into a deep body-altering hormonal therapy to cure him, has finally posthumously apologized. Brown said, So on behalf of the British government, and all those who live freely thanks to Alan's work I am very proud to say: we're sorry, you deserved so much better.

19 September 2009

Christoph Haas: Tired of Nagios and Cacti? Try Zabbix.

One of my professional duties in my past ten years was monitoring systems. Even my diploma thesis was dedicated to distributed monitoring (altough my professor sucked badly). Apart from a few custom-programmed scripts to analyze special situations (e.g. proxy clusters) I used tools that fellow administrators will find familiar: Nagios and Cacti. And another less famous text-configuration-based monitoring tool called Cricket. That worked well somehow but Cricket was hard to learn for my coworkers and Cacti seems unreliable and fundamentally broken in terms of SNMP checking. Besides why do I have to set up availability checking in Nagios and set up checking of the same parameters in another software to draw graphs? Just recently I came across an open-source software I hadn t heard of before: Zabbix. And although it has a few rough edges it seems way more professional than other common tools (the commercial tools I saw are even worse than the open-source variants). I tried it and after a lot of reading and trying it looks like it has a good potential to replace Nagios and Cacti. So I thought I d sum up my personal experiences with all of these tools. Nagios. Their makers claim that it s the industry-standard in IT infrastructure monitoring . Honestly it s a great tool but considering how many years it has been existing it barely evolved. During my diploma thesis I wrote an alternative software that I called MrNetwork that dealt with flaws that Nagios hasn t even fixed today. Still Nagios is a tool I have used for many years and it is very reliable.Nagios service detail screen Advantages: Annoyances: Cricket. As Nagios does not support plotting graphs of the monitored values I was in need of another piece of software. Basically Cricket is a frontend to RRD (which stores data in a rotating/round-robin file that keeps data of the last X minutes/hours/days). It has a textual configuration that takes a lot of getting used to. It s main principle is inheritance of settings they call it configuration tree . Which means you have a master DEFAULTS file that contains general settings like how to query SNMP. In a subdirectory you define a certain class of devices that you want to monitor e.g. routers (the DEFAULTS are inherited to this level). Within the routers directory you can just define a list of routers you want to monitor. All settings are inherited from above (parent directories). It s more a geek tool for shell lovers. Cricket screenshotAdvantages: Annoyances: Cacti. Another frontend to RRD and a pretty sophisticated one. Nearly everything is configured through its web interface. And the result is beautiful. It s not entirely reliable though and SNMP support (at least in version 0.8.7b) is a big fail. I like Cacti for its user interface is much better than Cactis but it s less reliable and flexible. Cacti screenshotAdvantages: Disadvantages: Zenoss. People pointed me to Zenoss which is supposed to offer the same functionality as other monitoring systems but is much more integrated. So this short list is more a quick one-day-experimental expression than a thorough analysis. But in the end much of the fuss is just marketing. Advantages: Annoyances: Zabbix. This is the tool I have been evaluating for the past two weeks and currently I m moving all checks away from Nagios and Cacti and put it into Zabbix. So the following list is more a first impression instead of a year-long experience. (I m using Zabbix 1.6.5 here. Debian Lenny s 1.4 version lacked some interesting features like proper SNMP handling so I backported the newer version to Lenny.) Zabbix seems very close to the perfect monitoring system I had always dreamt of. I would have designed it differently in some aspects though. Zabbix screenshotAdvantages: Annoyances: See also: Ben Rockwood s blog Further similar software I didn test thoroughly: Hyperic and Opsview (I ll surely update this blog post while I learn more about Zabbix.) All of the above tools are great. I m not meaning to say that Zenoss is total crap for example. The differences are subtle. And whether a piece of software suits your needs really depends on your expectations. I love that all this software is available as open-source. And a totally unscientific but fun analysis of the community is counting the number of active people in the respective channels on the Freenode IRC network: Either Nagios has the largest fanbase or perhaps that means that the majority of people needs help with it. :)
http://en.wikipedia.org/wiki/Network_monitoring

18 September 2009

David Paleino: RC bug squashing

Following zack's suggestions, I've been looking at RC bugs in my sparetime. In two days (well, two evenings really), I've been able to cope with a good number of bugs: I must admit that some of them (didn't count them, might also be the greatest part of them) already had patches, and I just prepared the NMU. For others, there were other FTBFS's other than the ones filed, and those were fixed too. It would be extra cool if some DD uploaded the remaining 3 NMUs! I also had time to release and package syslog-summary 1.14, but, not being DMUA yet, I'd need a sponsor for that too (but I'll probably ask my AM, since this upload sets DMUA). Happy RC-bug-squashing! Your friendly neighborhood hanska Smile

Next.

Previous.