Search Results: "nate"

6 September 2007

Andrew Stribblehill: Mountain View Fuzz

Got back to the appartment last night to find police on High School Way, interviewing somebody innocuous-looking. When I went to the door to the appartment complex, it turned out that there were upwards of a dozen police, with six patrol cars, on Castro St. Most were standing around doing nothing but one of them appeared to have stopped a car and was interviewing the driver. I approached a chap at the edge of a crowd of onlookers and asked what was going on, and he declined to comment. At about that moment, the helicopter flew past for the second time. Just as Pete and I were about to go, somebody cycled along the and stopped to cross Casto, halting in the middle of a gathering of police officers. One told him, "It’s illegal to cycle on the sidewalk, sir." His reply was a work of genius: "Oh, really?" It was all very bizarre, though not at all menacing or confrontational. No guns were drawn and nobody was incapacitated by tazer or baton. Just another data point. Then we went up to the appartment proper and watched Hot Fuzz. Excellent film, especially given that the room was still being illuminated by the occasional strobe of blue lights.

5 September 2007

Kevin Rosenberg: Distributing Repositories

I've been using source control management (SCM) systems going back to the the very limited RCS back in the early 1990's. Over the years, I've migrated my SCM to take advantage of improved functionality. Recently, I migrated from subversion to git. While I keep my home directory in SCM (initially CVS, then SVN, now GIT) and that directory is replicated across Linux, Windows, and Mac OS X systems, a big reason for changing from svn to git is to give people more access to my open-source projects. Periodically, I'd have my CVS and SVN repositories available on a public web server. But, that typically required some effort of mine to mirror a public copy from my private repository. Using a distributed SCM like git, there is no additional effort needed to make a public copy of my respository. But, a bigger win over just making a public respository available is allow users to have their own local respository. While I appreciate the patches (bug fixes and improvments) users email to me, there are some cases where I feel the patch has limited value and don't incorporate it into my upstream respository. With a distributed SCM, users can maintain their own local patches while being able to merge changes that I make to the project's canonical repository. Though, I do hope users to continue to send patches that would be useful to a project's user base. As an added benefit, git has much improvement mechanisms for users to do so. For my private use, the distributed repository system is not a big win. However, git has much improved branching over subversion which will help when I want to make a local branch for experimental changes. To convert from my single private respository to a useful public git repository, I've separated my open-source projects into discrete git repositories. The histories of the git repositories on some projects go back to 2002. Unfortunately, I was not able to migrate my tags from subversion to git because git-svnimport got confused about the odd tagging/commits from the cvs2svn tool I used when I migrated from CVS to SVN. If checking out a particular old version is important, one can refer to the dates in the project's debian/changelog file to find the commit matching that release. At some point, I may write an automatic tool to retag the version numbers on the projects. The public git repositories can be browsed at http://git.b9.com/gitweb. I'm modifing my Lisp web site creation tool so it will automatically add the git repository to each project's website. For now, the typical URI to clone a git repository would in the form of git://git.b9.com/<project>.git. I expect users of my projects will find the new, much deeper access to my open-source projects empowering. The set of commands to clone the current set of open-source repositories are:
git clone git://git.b9.com/cl-base64.git
git clone git://git.b9.com/cl-modlisp.git
git clone git://git.b9.com/cl-photo.git
git clone git://git.b9.com/cl-readline.git
git clone git://git.b9.com/cl-rss.git
git clone git://git.b9.com/clsql.git
git clone git://git.b9.com/cluck.git
git clone git://git.b9.com/ctsim.git
git clone git://git.b9.com/getopt.git
git clone git://git.b9.com/hyperobject.git
git clone git://git.b9.com/irc-logger.git
git clone git://git.b9.com/kmrcl.git
git clone git://git.b9.com/lml.git
git clone git://git.b9.com/lml2.git
git clone git://git.b9.com/md5.git
git clone git://git.b9.com/pipes.git
git clone git://git.b9.com/postoffice.git
git clone git://git.b9.com/ptester.git
git clone git://git.b9.com/pubmed.git
git clone git://git.b9.com/puri.git
git clone git://git.b9.com/reversi.git
git clone git://git.b9.com/rlc.git
git clone git://git.b9.com/rt.git
git clone git://git.b9.com/uffi.git
git clone git://git.b9.com/umlisp.git
git clone git://git.b9.com/umlisp-orf.git
git clone git://git.b9.com/vcs-tree.git
git clone git://git.b9.com/wdq2wav.git
git clone git://git.b9.com/wol.git
git clone git://git.b9.com/xlunit.git
git clone git://git.b9.com/xmlutils.git
git clone git://git.b9.com/xptest.git

Kevin Rosenberg: Distributing Repositories

I've been using source control management (SCM) systems going back to the the very limited RCS back in the early 1990's. Over the years, I've migrated my SCM to take advantage of improved functionality. Recently, I migrated from subversion to git. While I keep my home directory in SCM (initially CVS, then SVN, now GIT) and that directory is replicated across Linux, Windows, and Mac OS X systems, a big reason for changing from svn to git is to give people more access to my open-source projects. Periodically, I'd have my CVS and SVN repositories available on a public web server. But, that typically required some effort of mine to mirror a public copy from my private repository. Using a distributed SCM like git, there is no additional effort needed to make a public copy of my respository. But, a bigger win over just making a public respository available is allow users to have their own local respository. While I appreciate the patches (bug fixes and improvments) users email to me, there are some cases where I feel the patch has limited value and don't incorporate it into my upstream respository. With a distributed SCM, users can maintain their own local patches while being able to merge changes that I make to the project's canonical repository. Though, I do hope users to continue to send patches that would be useful to a project's user base. As an added benefit, git has much improvement mechanisms for users to do so. For my private use, the distributed repository system is not a big win. However, git has much improved branching over subversion which will help when I want to make a local branch for experimental changes. To convert from my single private respository to a useful public git repository, I've separated my open-source projects into discrete git repositories. The histories of the git repositories on some projects go back to 2002. Unfortunately, I was not able to migrate my tags from subversion to git because git-svnimport got confused about the odd tagging/commits from the cvs2svn tool I used when I migrated from CVS to SVN. If checking out a particular old version is important, one can refer to the dates in the project's debian/changelog file to find the commit matching that release. At some point, I may write an automatic tool to retag the version numbers on the projects. The public git repositories can be browsed at http://git.b9.com/gitweb. I'm modifing my Lisp web site creation tool so it will automatically add the git repository to each project's website. For now, the typical URI to clone a git repository would in the form of git://git.b9.com/<project>.git. I expect users of my projects will find the new, much deeper access to my open-source projects empowering. The set of commands to clone the current set of open-source repositories are:
git clone git://git.b9.com/cl-base64.git
git clone git://git.b9.com/cl-modlisp.git
git clone git://git.b9.com/cl-photo.git
git clone git://git.b9.com/cl-readline.git
git clone git://git.b9.com/cl-rss.git
git clone git://git.b9.com/clsql.git
git clone git://git.b9.com/cluck.git
git clone git://git.b9.com/ctsim.git
git clone git://git.b9.com/getopt.git
git clone git://git.b9.com/hyperobject.git
git clone git://git.b9.com/irc-logger.git
git clone git://git.b9.com/kmrcl.git
git clone git://git.b9.com/lml.git
git clone git://git.b9.com/lml2.git
git clone git://git.b9.com/md5.git
git clone git://git.b9.com/pipes.git
git clone git://git.b9.com/postoffice.git
git clone git://git.b9.com/ptester.git
git clone git://git.b9.com/pubmed.git
git clone git://git.b9.com/puri.git
git clone git://git.b9.com/reversi.git
git clone git://git.b9.com/rlc.git
git clone git://git.b9.com/rt.git
git clone git://git.b9.com/uffi.git
git clone git://git.b9.com/umlisp.git
git clone git://git.b9.com/umlisp-orf.git
git clone git://git.b9.com/vcs-tree.git
git clone git://git.b9.com/wdq2wav.git
git clone git://git.b9.com/wol.git
git clone git://git.b9.com/xlunit.git
git clone git://git.b9.com/xmlutils.git
git clone git://git.b9.com/xptest.git

3 September 2007

MJ Ray: Current Issues / Ride Report: Kewstoke-Congresbury

In preparation for Wednesday's cycle forum meeting at Backwell station (6.30pm), I went for a ride to Congresbury.

I followed roughly the route out along Ebdon Lane and Hewish bridges shown on this javascript map but on the way there, I went south from Puxton and turned off Nye Road onto the Strawberry Line (NCN route 26) to test it out.

The main surface seems to be very loose gravel on top of a very hard subsurface.  It's uneven in places, but I think I still averaged about 10mph. At each end of the route, the connecting paths are mud tracks.  The left turn just before Congresbury station is unsignposted. It's not a bad route, but a long diversion to go to Congresbury. The whole ride out took about an hour in total, going at a fairly gentle pace.

Coming back, I rode along the bridleway that is pretty direct from Congresbury to Puxton. Unfortunately, somebody has tried to fill in some uneven sections with rubble since I rode it last year. The route is now "get off and walk" in several places and uncomfortable otherwise.

I think I'll be riding along the A370 from Wick Road to Congresbury on Wednesday evening. If only there was a track across Tutshill sluices...

In other cycling news, the police have an undercover bike which they'll leave lying around, see if it gets nicked, then track the thieves back to their lair.

Felipe Augusto van de Wiel: 3 Sep 2007

To Blog, or Not To Blog...

Amazon: in the hands of a few
Thinking about writing something for a while, about my Debian work or about my daily stuff, sometimes you lack the time or the motivation to do so, I should kept this more up-to-date so people can follow a little bit what's going on with me... specially considering that I try to keep this diary neutral about non-technical subjects, but hey! it is somewhat hard to be neutral in every single situation. Anyway, I found that I should help to spread the word from a Brazilian colleague, Sulamita, she posted about democracy (or the lack of it) today. Two videos, about the Amazon Forest area and some of the crazy things we find in Brazil. You can watch the Brazilian Portuguese version or the English one.

A lot of people are not aware of some of the political problems that Brazil faces in the last 10 years, it is not like it doesn't exist before, but it is getting worst and worst and, for a while, I've been wondering...
    A lot of friends want to leave Brazil as soon as they can, I love Brazil but I can understand their motivation.

  • Where do you find the desire to stay when you look around and you can't find fairness and equality? And if you look to the houses or places that should protect people, you realize that there is no protection?
  • How do you keep the passion for a country with huge levels of corruption in public government, in police forces, in your law makers? And day after day, corruption schemas, robbery and law abuse are noticed on the television?
  • What to do when you realize that the perception of the disloyal advantage is shared among your colleagues and that the feeling of failure dominates a great part of the population that could actually do something to change?
  • Can anybody, or group of individuals, feeling alone, work to change that? How? Specially when you look around and although you know there are people willing to help they need to take care of their lives (money, food, family)?
Or should I ask: Where's the love? (Black Eyed Peas Lyrics).

And before somebody thinks that the entire Brazil is lost, that we are under a Civil War and that monkeys and other animals are walking on our Jungle streets in the middle of the cities, no it is not... yes, we need help; yes people need to join forces; yes, we need to find ourselves and fix it; but it is still a great country to live, and yes, I love it!
Thanks: beraldo and sysdebug. And I'm happy to discover that advogato upgraded my status, I'm now a Journeyer.

2 September 2007

Russell Coker: Optus Password changeme

I have just given my parents a new computer, and part of the upgrade process lost their email passwords (which were stored in KDE preferences - this seems to happen every time KDE is upgraded). The only password that is not under my control is the password for their Optus account. So I had to get the password changed, I got my mother to phone Optus but they decided that my father had to make the request (not that I would have had any difficulty impersonating him, any of the details that they asked for which I might not have known could have been provided by my mother). So my father requested that my mother be listed as someone who is authorised to make changes to the account. It turned out that changing a password for a mailbox is a difficult operation and I needed to talk to the phone-support guy. He needed to get permission from my father for this (he didn’t seem to realise that he had just granted my mother full access so that she could authorise such things on future calls). After my father had given permission for the second time I got the password changed, the new password was “changeme“. The call-centre guy advised me to change it as “it’s a very common password“! An ISP with any clue in their call centre would get the password changed to a semi-random string. Then there was the process of changing the password. The web site didn’t work at all with Konqueror (the call-centre guy told me that only IE is supported). I used Iceweasel (Firefox) and it allowed me to change the password. The Optus web site is one of the worst I’ve ever seen, and redirecting to the MS site live.com as part of the process was only part of it. Another bad thing that Optus does is punish customers who do things that they don’t like. My original Optus contract specified unlimited uploads, the first time they cut me off for actually uploading a moderate amount of data (running Bittorrent) they disconnected me and claimed that it was a DOS attack. On a later occasion they disconnected me on a Wednesday because I had uploaded some files on the weekend. They alleged that the upload was continuing on the Sunday (although I had a very clear recollection of ending it on the Saturday), but we all agreed that the upload had ceased before the Monday. But apparently I needed to be disconnected on the Wednesday to teach me a lesson. The lesson of course is that Optus sucks really badly. They do have reasonable prices and at the time their cable network offered download speeds significantly higher than any ADSL plans could match, which is why I have continued using them (and subscribed again after buying a new house). Recently ADSL2+ has started to become common and prices have been falling. For my use and for most people who I know ADSL is now either cheaper than cable or close enough that it’s worth paying the extra to escape Optus. My parents are on an Optus plan that gives them 100MB per month of data transfer (which counts both download and upload) before they are limited to modem speed. This is enough for them, but I expect that when their Optus contract runs out there will be an ADSL plan that’s not much more expensive and they can cease using Optus too. I just wish there was a real choice of providers for the base phone service. Currently there is only Telstra and Optus, both of which are expensive and suck. Fortunately Telstra has an insane CEO who is determined to make sure that everyone in Australia learns that Telstra is a nasty monopoly that needs to be broken up or severely constrained.

Stefano Zacchiroli: clean distclean

On cleaning and distcleaning ... ... and CDBS and "FTBFS if built twice in a row" Yesterday I closed two bugs of mine (424331 and 424392) of the "FTBFS if built twice in a row" class. Shame on me, they were open since too long. Both packages use autotools and, on the Debian side, CDBS. For both packages the fix was as simple as adding:
DEB_MAKE_CLEAN_TARGET = clean

to debian/rules. The rationale for that is rather obvious (now that I've thought about it of course). CDBS per default uses "distclean" as the upstream target which is invoked when "debian/rules clean" is executed. Unfortunately, "distcheck" (used by upstream to create the distribution tarball) does not check that "distclean" leave the distribution tarball as it was before the build. So nothing grants a priory that stuff removed by "distclean" is rebuildable with just the build-deps. A lot of packages come with generated sources (for example by XSLT or by some template toolkits) included in the source package, if that stuff is removed by mistake during distclean you'll see yet another instance of this kind of bugs. I bet a lot of other "FTBFS if built twice in a row" bugs out there have been fixed this way or can still be fixed with the addition of the line above. Update: changed this post, the first writing was made thinking at "maintainer-clean" instead of "distclean"

31 August 2007

Michal &#268;iha&#345;: World clock applet

Being far away from your home adds you need to know what time is there and whether you can safely call people not waking them up in the middle of night. So I looked for some Gnome applet that would show me time in another time zone. I expected this to be easy, but unfortunately I didn't find anything in Debian. Few minutes of Googling revealed something, but both applets I found look quite unmaintained. After a bit of testing I decided to give World Clock Applet 0.0.0 a try. It seems to work so far and installation only needed few tweaks. As I hate to use something not in Debian packages, I packaged this beast. For now it is available only in my repository, as I'm pretty sure it would not pass any license checks :-).

30 August 2007

Pablo S. Torralba: Shell date and time tricks

One of the things I hate the most when programming is dealing with time and date functions and all the special cases which exists. Even more, I have the feeling I'm walking a path many times walked before, so it becomes one of the most unpleasant tasks for me.

Fortunately, it's true, so many others walked before that path so here are some quick tricks for shell programming using the wonderful date UN*X program:
  1. Converting epoch:
    1. From epoch to anything else: date -d @$epoch_value +FORMAT (where FORMAT is of course as described on date(1), and '@' makes the actual undocummented trick).
    2. From anything else to epoch: date +%s
  2. Calculating times
    1. One day forward: date -d "1 day"
    2. One day backwards: date -d "1 day ago"
    3. Just imagine "1 month", "3 months ago" and the like. Not only google is so friendly with human language ;-)
  3. More format conversion: '-d' option accepts several other formats as input, even with calculations:
    1. date -d "1977-08-19 30 years", yeah! my 30th birthday was on sunday. Thanks date, and it was (as epoch): date -d "1977-08-19 30 years" +%s... 1187474400 :-D
    2. Funny ls:

      ls -l while read perms links user group size d t name
      do
      echo $perms $links $user $group $size $( date -d "$d $t 1 day" ) $name
      done
      You can, of course, change the way date is shown. That's because 'YYYY-MM-DD hh:mm' is also a valid input format for date as it is 'YYYY/MM/DD'.
So '@', '1 day' and 'date' saved me the day.

Update: This features and more are indeed described in the coreutils info manual. Thanks mp for pointing it out.

Enrico Zini: procmailrc-trap

Procmailrc, TRAP and LASTFOLDER I use procmail and lbdb. lbdb is invoked at the beginning of procmailrc harvesting From: addresses from all incoming mail, so that they can later be queried with mutt. It's really neat. However, if it's put at the beginning of procmailrc, it harvests From addresses from spam, too. This causes the ldbd databse to become very big, slow to query, and in some unlucky cases, to return lots of bad results. Problem is, I run the spam filter AFTER I delivered to the mailing lists. The rationale is that mailing lists already do their own spam filtering to some useable level. At least those I'm subscribed to. So, my current procmailrc goes like:
feed a copy through lbdb
filter list 1
filter list 2
...
filter list N for N very big
filter out spam
deliver to inbox
Now, is there a way I could invoke lbdb for every mail except what is caught by the spam filter? A way, I mean, that doesn't involve hooking into each and every of the N list rules an lbdb invocation? Here is the way. In .procmailrc, add:
TRAP=~/bin/scan-nonspam
Then write ~/bin/scan-nonspam:
#!/bin/bash
if [ $ LASTFOLDER:0:4  != 'spam' ]
then
    /usr/bin/lbdb-fetchaddr
fi
exit 0
For reference, this is the documentation of TRAP:
When procmail terminates of its own accord and not because it received a signal, it will execute the contents of this variable. A copy of the mail can be read from stdin. Any output produced by this command will be appended to $LOGFILE. Possible uses for TRAP are: removal of temporary files, logging customised abstracts, etc. See also EXITCODE and LOGABSTRACT.
And this is the documentation of LASTFOLDER:
This variable is assigned to by procmail whenever it is delivering to a folder or program. It always contains the name of the last file (or program) procmail delivered to. If the last delivery was to several directory folders together then $LASTFOLDER will contain the hardlinked filenames as a space separated list.

Theodore Ts'o: On the benevolent dictator model

Recently, Josh Berkus blogged about The Myth of the Benevolent Dictator. In it he complained about people who try to posit that a benevolent dictator was "an unalloyed asset" to their project, and that there were many successful and valid forms project governance other than the singleton project leader. Furthermore, he argues that the best way is to form a community, and then ends by suggesting that people who say "dictatorship good" is really saying "democracy bad", and perhaps the motives of the person using that argument should be questioned.

Given that I was one of the first people (and probably the first person) to use that phrase to describe Linus's governance model, obviously I feet the need to respond to his thoughts on the matter.

First of all, the term "benevolent dictator" comes from an old saying, "the best form of government is the benevolent dictator --- there's only one problem, finding the benevolent dictator"[1]. And in my presentations where I explained the Linux project management model, I was always careful to say that there were other models that were used by other projects, and that it was interesting to study why certain management models worked for some projects but not others.

So by definition, if a project is successful with a single project leader ultimately responsible for making decisions to settle conflicts, then that leader is a "benevolent dictator"; if they aren't successful, then that would be an example of a dictator who wasn't able to be "benevolent", eh? :-)

In all seriousness, though, I agree with Josh when he says that the key part of an open source project being successful is gathering together a healthy community. But let's unpack that a little. In order for a community to be healthy, there needs to be general consensus on the values and cultural norms for that community. So for example, in the case of Debian, which Josh holds up as a successful community, not everyone has the ability to vote; you have to pass a fairly stringent process which tests your technical skills and your willingness to subscribe to Debian's core values as embedded to the Debian Social Contract. People who do not pass the New Membership process do not have the right to vote in Debian, even if they are the most rabid, fanatical users of Debian.

This brings up a key point about governance analogies; they all don't map cleanly into the real world for two closely related and extremely important reasons: (1) No one has a fundamental human right to be a member in some OSS project, and (2) No one is forced to stay in a particular OSS project. These are really two sides of the same coin, and what it means is that, as in Debian, "democracy" does not mean that everyone is entitled to political franchise, or the right to vote, and in a project which has a "benevolent dictator", the powers of the dictator is fundamentally limited by the simple fact that if he is too unreasonable, people are always free to leave or fork the project. These differences are extremely important when trying to reason about "democracy" or "dictatorship" as being good or bad as applied to open source projects.

Democracy is rarely the best way for an open source project to make decisions. For technical issues, you want the right answer, and majority vote isn't necessarily way to resolve a technical issue. As far as policy decisions, if the decision can not be made by consensus, then some percentage of the community will be unhappy with how the decision, and that can cause a huge amount of arguments and dissension. If the community has very clearly expressed values, then the number of times where such votes need to be made will be minimal.

For example, Debian has an explicit declaration of its values and mission in its Debian Social Contract; people who don't agree to the Social Contract aren't allowed to become Debian Developers and thus are not allowed to vote. But, one of the more controversial disputes that have threatened to tear apart Debian is the question of the non-free packages, and that's fundamentally because of an ambiguity in the Social Contract. On the one hand, Debian has declared that it is fundamentally 100% about Free Software, to the extent that it has adopted Richard Stallman's rather petulant request that the Linux operating system be called GNU/Linux (which admittedly is better than Lignux.) On the other hand, the Social Contract also explicitly specifies the existent of the non-free section in the distribution. Over time, this ambiguity have been the source of a lot of highly passionate debate and controversial votes that have satisfied neither the FSF fanatics nor the more Linux-like moderate position tolerating the use of proprietary as well as open source software. I submit that if Debian had either explicitly stated an FSF-centric position, and only accepted members that supported point of view, or explicitly subscribed to a position that users should be able to use whatever software they feel meets their needs, as Ubuntu has, it would avoided many arguments that have threatened to tear apart Debian.

More recently the fact that the Social Contract has been silent about whether or not Debian Developers should be paid or purely be 100% volunteer. This was never an issue in the Linux Kernel community, but for some reason it has been a highly divisive and disruptive issue inside Debian. Democratic votes have not really resolved the conflicts, and in fact some might claim that these votes have if anything hardened various Debian Developers' positions.

Going back to Josh's final point, "Next time you hear the words 'benevolent dictator' you probably want to think about the motives of the person saying it. Because the corollary to 'dictator good' is 'democracy bad', and is that a place you really want to go?", I am quite proud to stand by my convictions that for Open Source Projects, pure democracy is a bad idea. If a projects wants to be technically successful, it needs to be a meritocracy, and chose the best technical decisions; Linus does not decide by abiding with majority vote whether or not a patch should be accepted, and that's as it should be.

Even Debian's "raw democracy" doesn't let any random luser get to vote on who should be the Debian Project Leader. Decisions need to be made by people who are competent to make a valid choice. Even in real life, one could argue that the choice made by Americans, over of which 70% believed that Saddam Hussein was "very likely" or "somewhat likely" to be personally involved with the 9/11 attacks even though there has been absolutely no evidence to support this "gut feeling", made a rather disastrous choice in voting to re-elect George Bush to the presidency. Democracy without sufficient education, literacy, and intellectual curiosity to learn about what is happening in the world can result in truly disastrous choices being made. So certainly, democracy is not a panacea. In the real world, it may be the best choice of many bad choices, given how much people have suffered under military dictatorships. However, in the Open Source world, things are different because people are always free to leave one project and affiliate with another --- not an alternative most people living in dictatorships have historically had. Given these key differences, I think it is clear that for open source projects, democracy is indeed often a bad idea, and that if you if a project has a single person with a strong technical vision and good interpersonal skills, the benevolent dictator model can indeed be a very viable and successful.

Does that mean that other forms of governance can't work? Of course not! The rotating dictatorship used by Perl[2], the core-team-with-strongarm leader used by OpenBSD, the corporate dominated scheme of MySQL, the self-selecting core team model of Samba, all are successful models. I just have to strongly disagree with Josh's claim the single strong leader approach is somehow inferior to "democracy" as if democracy was somehow a magic bullet.


[1] For am early example of this quote, nine years ago, please see this message, where I'm pretty sure I authored the colon prefixed quote, and I know I was using the term "benevolent dictator" much earlier than 1998.

[2] In the early days of Perl, it also used the benevolent dictator model. Larry Wall is a great exemplar of how someone who has a strong vision of what he wanted to accomplish, and was humble and easy to work with, both on-line and in person, can make all the difference in the world for an up-and-coming open source project.

29 August 2007

John Goerzen: Desktop Linux: Gnome

I had been intending to write an entire series of posts about our corporate switch to Linux on the Desktop. To date, I wrote only one introducing the project and our reasons for switching from Windows. That was back in April.

Today I'd like to start talking about it all some more.

We have standardized on Gnome for our desktops. Given the Windows background of our user base, it was pretty much a given that we would have to use either Gnome or KDE. Something like fvwm or a non-integrated environment just wouldn't be a good option.

We evaluated both Gnome and KDE. The very "clean" appearance of Gnome was a nice thing for us. KDE seemed to be to "chatty", talked about entering in audiocd:/ when it shouldn't have needed to, and generally violated the KISS and principle of least surprise too often. That said, I continue to run KDE for my personal desktop because Gnome just doesn't have the flexibility that KDE does. It is too bad that Gnome has gone on this remove functionality kick, and KDE hasn't gotten the KISS religion yet.

Anyway, Gnome worked well for the most part. We have set some defaults in gconf for things like panel icons. We also set a few mandatory defaults. I fixed a couple of bugs in the vfs system related to nfs4 support, which manifested themselves as icons for files newly saved to the desktop never showing up.

We wanted to present a customized menu to people based on what their job function is. That is, we are using a single system image, so all apps will be installed on all machines. But we didn't want people to have to see a ton of software that they don't use. That was easily enough accomplished for custom apps by creating desktop files with mode 0640 and setting the group to the set of people that should see the program on their menu. We removed a few stock programs (such as the terminal) from the menu as well, using dpkg-statoverride. That was also quite easily done. However, I will say that the entire Gnome XDG menu thing is woefully under-documented.

We use Firefox for the standard web browser. It is integrated well enough with Gnome and we have no problems there, aside from sites that are IE-only. We solve that with a Windows terminal server, which I'll discuss later.

Our network printing was already based on Cups. The individual machines are set up as Cups clients only, which works fine. We did find, however, that gnome-cups-manager automatically installs a tray monitor for cups. This monitor puts little printer icons on the tray when printers are in use. Unfortunately, it figures out which printers are in use by polling the server, and it is turned on by default out of the box, with no good way to disable it short of dpkg-statoverriding it to 0000. You can imagine that hundreds of users times dozens of printers times numerous polls per minute created quite the load on the server. This was a really braindead design and the people that wrote it should have known better. It is also quite useless to have icons coming on for all the printers on the network, which on some networks could be thousands, and not even on the same continent as the user.

Printing is generally a bit iffy in Gnome. They seem to be transitioning between about 3 different printing toolkits, all of which have different print dialog boxes with different supported features and different ways of selecting printers. One chief annoyance is that the print box in evince (the document/PDF viewer) does not let people access printer-specific features such as hole punching and stapling. So we installed gtklp and xpdf for people. The people that print heavy PDFs are huge fans of gtklp these days; it's a nicer solution than we had in Windows. Nobody really likes evince. We also have had some trouble with evince generating PostScript output that some printers can't grok. It sounds like all this should be much better in newer versions of Gnome, which if true, would be welcome news.

The Gnome screenshot tool makes it easy to save off a screenshot to a file, or to drag it into an email, but it is difficult to print it (you have to save it first). That was a common complaint around here, so I wrote a little wrapper around xwd and gtklp for printing screenshots. People really like that because gtklp gives them lots of options about orientation and size of the image if they want it, or a simple "Print" button to click if they don't care. We set a gconf default to bind this to Ctrl-PrintScr and it works well. KDE's screenshot tool is much more capable, and if we were using KDE, we wouldn't have had any problem with screenshots.

The bottom line on Gnome is that we, and are users, are happy with it after we've made these customizations. But we have had to do more customization that we should have. I still think that Gnome has been better for our users than KDE, but I do wonder how long we'll be able to survive with our "no KDE libraries" policy, as people want ksnapshot, kolour, etc.

28 August 2007

Kai Hendry: MBP versus X40

Thinkpad X40 vs MBP I’ve spent a bit of the bank holiday weekend by probably violating the warranty of a new 15” Macbook Pro (MBP) from work. I also was at the Debian bbq 2007 in Cambridge and met some friendly faces. :) More MBP & X40 porn So now I can boot into MacOSX and run Ubuntu or boot straight into Ubuntu from rEFIt. There are a couple of problems. There are some things I quite like about the MBP compared to the X40: Currently I remain not totally convinced about the Apple MBP. I’ll keep my Debian X40 my personal machine and the MBP for work. Now I can compile Webkit on Apple OSX (Debian can too) and see what’s happening in Ubuntu.

27 August 2007

Romain Francoise: More fun with md5sums: collisions

During my experiments with file duplication in the Debian archive, it occurred to me that having a list of files with identical MD5 hashes was a good starting point for finding MD5 collisions in the archive. If any of those files were different but had the same hash as computed by MD5, I'd have a collision.

Unfortunately, checking if the files differ involves extracting the data tarball of each affected package and computing another hash for the files (I used SHA-1), which takes a while. 3h32m and 47GB of extracted files later, I now have the results and there is no MD5 collision in the Debian archive. The chances were slim if not null, but at least now I'm sure.

(In fact I did find one occurrence, but it turned out that the file's MD5 hash in DEBIAN/md5sums was incorrect, for some reason.)

Russell Coker: Writing a CV for a Contract Position

There is a lot of career advice related to writing CVs in books and floating around the net. Unfortunately almost all of it is tailored to permanent positions. Some advice that I have seen includes researching the company you are applying for and emphasising your experience in related fields. This probably makes some sense if you have targetted a major company to be your employer for the rest of your life. But if you are after contract work (which incidentally tends to pay better than permanent positions) then it’s not going to work. About half the time when applying for contract work you won’t know the name of the company that is receiving your CV until after they have decided whether to grant you an interview. Contractors therefore need to write their CVs to have broad appeal (emphasising common technologies such as C programming and skills such as debugging), and to appeal to a recruiting agent. Some people recommend writing your CV in MS-Word to support agencies and potential employers who are not competent enough to open other file formats (such as PDF and HTML). I strongly disagree with this advice, most contract positions are advertised by multiple recruiting agencies and it would be best for you to be represented by the most competent agency. Therefore having an agency staffed by incompetent people not represent you because they can’t figure out how to read a HTML or PDF file is a good thing! The same applies for employers, while applying for the best paying contracts will often result in some strange and unpleasant working conditions having a manager who can’t open a PDF or HTML file is likely to result in something that is unusually bad - run away! Recruiting agents will often give specific requirements for a CV. Sometimes these requirements are a little odd (such as demanding that references to C programming be removed from a CV when applying for a Unix sys-admin job), but being merely a little odd is no reason to boycot the agency. To solve this I wrote my CV in M4 macros to generate HTML code. When I wanted to apply for a position I would just type make to compile the M4 source into five or more different CVs. Sometimes a recruiting agent would request two versions when applying for one position, it’s a strange request but when five versions were compiled in one pass it was easy to satisfy it. If I liked PDF I could have written M4 macros that generate TeX and have the make process convert TeX to PS and then PS to PDF. Unfortunately I can’t share any macros because the code was not of particularly high quality (which is a little ironic) and because it’s tightly tied in to my CV. The best piece of advice that I can give to anyone trying this is to get columns sorted out as the first issue. If you want to have some lists of programming languages, lists of OSs, and lists of hardware, etc in your CV then it usually works best to have two columns in the page. If the largest list is removed then the other lists need to be shuffled around to get the column lengths to match. Doing this in HTML required some ugly M4 macros. There probably is a better way of doing this in TeX (but I’m really not good at TeX and can’t give advice about it). James has some interesting advice about writing CVs. I think that his most useful points are:
  1. Use file names based on your name so that agents can find your CV in the directory full of CVs

  2. Include complete contact details. One thing he doesn’t mention is that you should buy a mobile phone for the purpose of getting a job even if you don’t want one for any other reason. A 5% difference in pay for a year’s work would cover the cost of many mobile phones and the difference between a good job and a bad one is likely to be more than 5%. If you are afraid of brain cancer then don’t worry, if you only spend a few minutes a month talking to recruiting agents then you are at no risk.

  3. Don’t start a sentence in the career history section with the word worked - everything in that section will be related to where you worked in the past. It seems obvious in retrospect but I realised that I have made this mistake on my CV.

26 August 2007

Russell Coker: Never IRC as Root

Ben Fowler blogs about the issues related to running IRC as root. Google searches for (irc client exploit) and (irc client “buffer overflow”) give a number of interesting web pages. Many of the exploits require the user to perform an action that’s slightly unusual, but why take a chance? The advice to not run as root while generally sensible (run everything with minimum privileges as much as possible) is IMHO not very useful in recent times (and probably was never very useful). Generally when a user is worried about system compromise they are not worried about attackers having direct hardware access, the ability to corrupt system files, etc. They are worried that the attacker might read their email and access other personal files. Therefore the instruction should be “don’t run IRC as root or as any account that has access to data which is important to you“. It’s not difficult to start an X-term that runs “exec su - ircuser irc” or “ssh -t ircuser@localhost irc“. Note that the -t option is required for ssh to make it allocate a pty even when receiving a command to run. Note also that in the case of su you need the exec option so that if the irc client is compromised and tries to perform a ioctl(0, TIOCSTI… based attack then it won’t succeed. In any of these methods make sure that X access is not granted. Until we get Security Enhanced X working in a viable manner any process that can display an X window on your screen can own you totally. There are of course relatively safe ways of doing X, I have previously documented how to configure the Xephyr X server (replacement for Xnest) to allow a process with a different security context to safely display a graphical window on your desktop. Generally I recommend not using a graphical X client on an untrusted network (IE anything other than an Intranet IRC server). I prefer to do my IRC in an account that’s not even on a machine that I care about and have it run screen so I can disconnect and re-connect from anywhere in the world. When I first got SE Linux in Debian to be useful (when I could boot and run all programs without problem) I logged on to some IRC channels related to Debian with the security context of root:user_r:user_t. I admit that my actions in this regard could possibly be described as trolling, but I wanted to demonstrate what SE Linux can do. Unfortunately of the many people who told me off for logging in to IRC as root, none of them wanted to hear an explanation of why user_r is safe in this regard. I expect that most of them were running their IRC client in the same Unix account that was used for their email etc (and probably most of them had GPG keys accessible from such an account). Sigh, it’s so easy to run IRC as a different user - in fact it’s probably the easiest of all network client programs to run in such a manner. There’s no reason not to.

Russell Coker: Solar Hot Water - Not Expensive

The Australian has a new Environment writer named Matthew Warren who has a history of doing PR work for the coal industry. This has the potential for insightful articles based on knowledge of what the industry is doing or for PR work for the coal industry masquerading as journalism. Unfortunately it seems like the latter is what we are getting. Matthew’s latest effort is an article claiming a $6.5 billion cost to home-owners because of an “effective ban” on electric hot water systems that Labor might impose. Unfortunately he doesn’t clearly state what an effective ban is, but does note that apartments are exempt. The specific claim that is used as the basis for the article is that solar hot-water systems cost $2,800 more than electric systems, and that after the solar hot-water rebate ($1,000) the additional cost would be $650,000,000 per annum, and that a period of 10 years would be required to replace all hot-water systems thus giving a cost of $6,500,000,000. So I presume that he expects that there would be 650,000,000/1,800 hot water systems installed per year which would be about 361,111. According to the Australian Bureau of Statistics there were 6,744,800 households in Australia in 2003. Matthew’s article states that about 40% of homes can have gas hot-water systems fitted that comply with the proposed new regulations, given that and the exemption for apartment buildings let’s assume for the sake of discussion that 50% of homes would require a solar hot-water system. Assuming that each household has a separate hot-water system that means that for these claims to hole we need an average expected lifetime for a hot water system of 6,744,800/2/361,111 = about 9.3 years. If the currently installed hot-water systems are expected to be replaced in 9.3 years then we can expect that hot water systems tend to survive for an average of about 18.6 years. I wonder if that number is correct (some google searching didn’t turn up an answer). One of the disadvantages of old-fashioned media is that they tend not to include calculations or cite sources adequately so we can’t easily verify or disprove their claims, I wonder if this is deliberate… Matthew admits that using a solar hot water system can be expected to save households $300 per annum in electricity expenses, I presume that this is based on current energy prices and that the savings can therefore be expected to increase as energy prices increase (we have a lack of water which is increasing the cost of producing electricity from coal). Let’s assume for the sake of discussion that the typical home-owner has a mortgage, the Commonwealth Bank is currently advertising mortgages with a rate of just over 8%, the way things work is that there are various criteria for getting a discount rate which most borrowers can meet so the result will be slightly below 8%. If a solar hot water system costs $1,800 extra to install and the money comes from a mortgage then every year it will cost the home owner about 8% of $1,800 which is about $144 (a saving of $156 per annum). If the solar hot water system saves the home owner $300 per annum then at any interest rate below 300/1800 (16.66%) they will make money. If there was no government subsidy and the entire $2,800 extra was paid by the home owner then at 8% interest it will cost $224 per annum (a saving of $76 per annum) and the interest rate would need to increase to 300/2800 = 10.7% to make it break even. So for solar hot water to not save the home-owner money we need to have a significant increase in interest rates (which incidentally would bankrupt many home owners), AND to have electricity prices remain the same (which would require even more of our tax money to be spent on supporting the coal industry). Finally Matthew complains that the solar hot-water rebate could cost the government $4 billion over the course of the scheme (the next 10 years). This sounds like a lot of money until you think about the 9 billion dollars a year that the government spends on subsidies for the coal, oil, and gas industries! On the current course the government would spend 90 billion dollars of our tax money subsidising polluting industries that cause climate change, but Matthew opposes spending 4 billion subsidising technology that prevents pollution and reduces climate change.

Cameron Dale: DebTorrent Usage: details of a successful dist-upgrade

I just completed a very successful and long overdue dist-upgrade of my unstable machine, using mainly DebTorrent for downloading the packages, so I thought I'd post some of my thoughts and experiences. The download consisted of 1294 packages to upgrade, totalling 1350 MB and taking 2h12m to download. Here are some of the good and bad things I observed. Good things There was a single other peer with me in the same torrent, and I managed to download 182 MB from him, which is about 13.5% of the total download. This is the first time I have noticed this downloading from peers occurring, as usually there are too few peers, too many torrents and too many possible packages to download for any sharing to occur. This will hopefully change in the future when more people start using DebTorrent, and when unique piece numbers are introduced to make the torrents last longer. However, it does show how the use of the backup HTTP downloader can seamlessly integrate with downloading from peers to provide a good user experience, even for early adopters. The CPU time used was only 10m46s, which translates to an average CPU usage of 8%, which is very reasonable. The average download speed was 174 KB/s, which is 58% of my maximum download speed. Though this may seem like a bad thing, my goal all along has been to make sure that the download time would not be more than twice as long as using HTTP. Of course, using DebTorrent may never be as fast as a straight HTTP download from a well-provisioned server, but that it not the point. The idea is to reduce the bandwidth needs of hosting a debian archive. But in the future, when there are many peers in a single torrent the download speed may be even faster than using HTTP, especially for peers with very high download rates that could not be matched by a single server. Things that need improvement The completion percentage reported by APT during the download was fairly inaccurate. Here are some sample readings I noted, compared with the actual completion percentage from the DebTorrent status page:
APT Reports Actual Completion
10% 40%
20% 70%
30% 85%
40% 90%
50% 93%
60% 96%
70% 97.5%
80% 98%
90% 99%
The discrepancy occurs because larger packages are broken up into pieces, so they can be partially downloaded without APT knowing about it (since only fully downloaded packages are passed to APT). Clearly this situation is far from ideal, and can lead to the user feeling that the download is progressing very slowly, or not at all. There are some plans to add status updates to the communication between APT and the DebTorrent client, but they require changes to the APT code to support them, so it may take some time to implement. Another problem is the memory usage I saw during the download, which was approximately 213 MB. This is obviously unnecessarily large, though the metainfo that DebTorrent needs to be aware of is quite large (stored in a text file it is about 3 MB). There does seem to be a memory allocation bug in Python 2.4 which causes increased memory usage, so moving to python 2.5 might help. However, my preliminary tests show this only saves you about 20% for DebTorrent. I think I will have to delve deeper into which parts are using all this memory, and unfortunately python doesn't seem to have a good memory profiler to help with this. I will be looking at both PySizer and Guppy/Heapy to start, but if anyone knows of a better solution, please let me know.

25 August 2007

Gustavo R. Montesino: SoC Final Report

The Google Summer of Code coding period has ended earlier this week, so it seems to be a good time to stop and look what was done (and what wasn't). Let's start with an objective analysis, following the list of planned features:

Advanced query of bugs on the Debian BTS

In a 1-10 scale, I would give myself something like 6 or 7 here. bug-triage can query bugs by number, source package, package, maintainer and submitter. These parameters can be combined to make an query (reports must match all parameters) and more than one query may be added, subtracted and and'ed. A clearly missing feature is querying by usertag, support for it has been added on btsutils already and is just a question of adding a new input box on the query builder dialog (it should work if you enter the query manually in the bug-triage query line AFAICT).

Show all the information related to a bug report

bug-triage has a "goto" button which opens the bug in the user's preferred browser. Missing is the support to open the bug in a MUA, which I have promissed to Lo c.

Send additional information to a bug report

bug-triage has a "followup" button which opens the user's MUA with the right address. Maybe setting the subject would be nice...

Manipulate the tags of a bug report

I haven't had time to implement this at all. I've started it, the bugs have a context-sensitive menu which have each of the available tags as checkboxes on the right current status, but clicking on them does nothing currently.

Search for similar bugs in upstream developer BTS

This one would value something like 3 or 4 in a 1-10 scale. It searches upstream bugzillas by product and component, but doesn't do any cool trick like searching words in the bug subjects. The plus is that it does support KDE bugzilla, even though I haven't been brave enough to promisse it before SoC :P

Forward a bug report to the upstream developer BTS

There isn't really a good interface to report bugs programatically on GNOME bugzilla, which was disappointing. <rant>There is the bug-buddy xmlrpc(?) interface, but the developers say they will actively break anything using it, which I find very free-software-unlike.</rant>. The current bug-triage solution opens a browser with some of the fields filled. It would be nice to fill the full bug description, but its limited by the maximum size of an URL, so it fills only an introductory text (something like This bug was filled on Debian BTS #XXX) currently. Let's hope bugzilla 3 get on the GNOME world soon.

Conclusions

Unfortunately, this project hasn't evolved as much as I would like in the second half of the GSoC. As I said elsewhere, a lot of stuff has happened, some good, some bad, but on the end everything I hadn't been able to dedicate myself to anything I should. Even so, I think the project may be considered sucessful.

Maybe the more interesting product of this project, at least currently, is still on python-btsutils. As I said in the middle-term report, it'll be integrated in the debian_bundle soon, and might be powering important stuff like the devscripts in the future.

The bug-triage application has a lot to evolve yet, but is also an interesting tool IMHO. There are a lot of hurdles in the way to get different BTSes integrated, but the effort will for sure pay in the future. I plan to keep developing it (evon though keeping the GSoC pace is impossible) and am anxious to see the results ;)

Finally, as I said before, one of the main things on GSoC for me was the potential to learn cool stuff. The best choice I've made during all of this was to use Python. Even though it's still a new language for me, and i still have to search embarassingly simple things in the tutorial/reference, it was a lot more productive than if I was developing in C. Learning to use git was also a great. This goal was the more fulfilled one, I've really learnt a lot of things which will surely be useful in the future. BTOH, I still can't get used to glade :/

Michal &#268;iha&#345;: First Japanese experiences

Thursday morning I arrived Chubu airport near Nagoya and at this time started my first visit to Japan. It will take quite long time (3-6 months), so I should have enough time to do as much sightseeing as I want. But there was not much time for sightseeing so far. Besides work and having rest after 24 hours of travelling, we only went today to see a bit of city. No real sightseeing, it was rather finding out some things. The most important was to find ATM accepting our credit cards. I didn't expect it will be that difficult, but it turned out that most ATMs accept only Japanese Visa and MasterCard cards and not ours. Fortunately post office has ATMs accepting foreign cards, so we finally have enough cache to survive next week ;-). The next step was to buy something we can start to learn Japanese a bit. It would be great to be able to read at least some signs and menu in restaurant. We bought few books, but later in restaurant it turned out to be almost useless. Fortunately the cook can a bit English, so we managed to order something we wanted. But this is the largest problem here - our poor Japanese knowledge as there was not time to prepare for this back at home...

Next.

Previous.