Search Results: "tar"

3 September 2009

Uwe Hermann: Open-Source software for LightScribe CDs and DVDs?

Dear Lazyweb, is there any Free Software utility for putting those nice logos/images on LightScribe CDs or DVDs? I'm considering buying such a CD/DVD burner, but only if there's open-source software to control the device. I guess burning CDs/DVDs should be no problem using wodim/cdrecord, but adding images as labels on the CD/DVD is likely not possible without special software. I know about the LightScribe Linux software (which is binary-only and not open-source, it seems) and LaCie LightScribe Labeler for Linux (which only creates labels but does not write them to the CD/DVD, and is not open-source either, I think). Anyone aware of a Free Software utility for this? Or at least a public specification/datasheet which could be used for creating one? I might even be willing to join development if datasheets are available.

Andrew McMillan: Storing Secrets

Something that has been annoying me recently with my bank has been that their website tells me that they will never ask for my password over the phone. And then their call centre asks me for my password. Over the phone. Of course the call centre doesn't mean my website password - they mean the special 'ultra-secure 5ekr1t code phrase', but they don't have a good, universally understood word to use for that. Hopefully they'll work one out, but they appear to have got the message anyway. This got me to thinking about how these phrases are used, and how insecure they are in reality. After all when I store a website password I go to significant lengths to ensure that the same password is not represented by the same string of characters in my database. How vulnerable are our secrets in the databases of organisations we do business with?
<!--break--> Simple Password Storage Surprisingly often people do store passwords in databases in plain text, so that should their website get hacked someone would quite possibly be able to download the whole password database. Please feel free to name and shame these organisations in the comments below. My own pet hate in this regard is the 'Mailman' mailing list software: by default on 'mailman day' - the first day of each month - it sends me my password. In plain text. Of course many developers recognise this flaw, and work around it by using a one-way hash to obscure the password. Usually they choose md5 for their hashing algorithm though, and they often fail to use a 'salt' to randomise the plaintext prior to hashing. This means that even though a password might seem obscure like 'Supercalifragilisticexpialidocious!', and no doubt it will hash to something that seems obscure like 'a7290d426b6a1764af6fd7fba5db214e', but you can often find it straighforwardly by looking it up through one of the friendly reverse hash lookup websites. There's even a Digest::MD5::Reverse perl module on CPAN to interface to a bunch of these in a more automated way! Oh dear. One way to go beyond this is using a 1-way hashing algorithm, with a random salt included into the plaintext before the hashing, so that if (god forbid) two users had 'password' for their password I might see two rows in my database like:
davical=# select username, password from usr;
  username                       password                          
-------------+------------------------------------------------
 user1          SSHA qCctCH5dirYCf29uxJiE68LvmLRDdnBkbldiWlE=
 user2          SSHA y8yOzjoh9fSkVwLaXGoVtWdiIYxmU2FCb2dOZXc=
(2 rows)
When the user wants to log in I apply the same transformation to their incoming password (appending the same salt) and compare against my stored hash. If they match then it must be the same password they used previously. Storing passwords in this way secures them from casual, or even reasonably determined access, although naturally they can still be logged at the beginning and end of the communication - or even in the middle, if we didn't encrypt that bit! The PHP function I use to salt and hash the password is as follows:
/**
* Make a salted SHA1 string, given a string and (possibly) a salt.  PHP5 only (although it
* could be made to work on PHP4 (@see http://www.openldap.org/faq/data/cache/347.html). The
* algorithm used here is compatible with OpenLDAP so passwords generated through this
* function should be able to be migrated to OpenLDAP.
*
* If no salt is supplied we will generate a random one.
*
* @param string $instr The string to be salted and SHA1'd
* @param string $salt Some salt to sprinkle into the string to be SHA1'd so we don't
*                     get the same PW always hashing to the same value.
* @return string  SSHA  followed by a base64 encoded SHA1 of the salted string.
*/
function session_salted_sha1( $instr, $salt = "" )  
  if ( $salt == "" )  
    $salt = substr( base64_encode(sha1(rand(100000,9999999),true))), 2, 9);
   
  return ( sprintf(" SSHA %s", base64_encode(sha1($instr.$salt, true) . $salt)));
 
What about Secret Code Phrases? The problem with these secret code phrases, apart from all of the forgetability and guessability problems that have repeatedly been identified elsewhere, is that they are much less likely to be stored in a one-way hash. Are you going to ask your call-centre staff to type in the customer's secret code phrase? Didn't think so. And if you did it's going to add pronouncability issues to the whole mix. So this means that those organisations who have our secret code phrases in their database will, in all likelihood, have them stored directly as plain text, displaying them to the random call-centre staffer along with all of our other account details, and especially making them vulnerable to accidental disclosure. Disclosure of a sort that doesn't necessarily involve knowing they have been disclosed. Proliferation of Use These things provide the appearance of security - 'Security Theatre' as Bruce Schneier terms it - and because of that they're taken up in a kind of a cargo cult of security: "if the banks do it that way it must be a really good form of security". This makes the problem much worse, because now I have to remember a secret code phrases not only for banks, but for ISPs, phone companies, online auction websites, and so on. How many mother's maiden names, favourite teachers and friend's phone numbers do I have? I'm sure I'm at well up whatever curve it is that measures the number of passwords a person has, because five years ago I had so many I started to store them all in an encrypted database - protected by a yet another password, of course. Now in order to get my story straight I have to store my 'secret code phrases' in there too. If I didn't store my secret code phrases in that database, I'd obviously be re-using them everywhere, from a very small set - perhaps the same one that everyone around me overhears, every time I have to ring my bank to authorise another payment from my account. Because the proliferation of use is not just the breadth of wannabe thespians hoping to climb on the stage of this latest play, but the way they want to use it all the time, too. In fact the only conversation I've had recently with my bank where they didn't want it was when they rang me. Obviously I was the only person who could answer a phone in my house, right? It isn't just security theatre: I think we can see that this analogy belongs much deeper into the sub-genre of 'Security Farce'. Is there a solution? I don't have any easy answers - other than the ones to my security questions, of course - but some improvements are possible. Other banks have quizzed me about stuff like recent expenditure or credit card limits from time to time, but I've usually passed those tests by reading my last bank statement - or failed them by not having it to hand! I don't really think that the answers can lie in that direction because the information is only quite loosely tied to my identity. For some parts of the call-centre handling of secret code phrases there are changes that could make them more secure, but in the fairly short term these organisations need to find a different way to perform these out of band identity checks. For the actual storage of the code phrases it would be a marginal improvement if the database did not contain the actual phrase. Perhaps it could be encrypted with some application-known key, so that it can be unencrypted when it needs to be displayed, but never stored in the clear. Of course there's still the problem with that key... Verification of the secret code phrase could be done by someone not involved in the transaction, so that the call could be temporarily passed into a 'verification stream' where a different person performed the verification step without the context of the account details or enquiry. Though this sort of complexity seems unlikely with call centres seemingly being outsourced to the cheapest supplier. One thing does seem likely to become increasingly true: there is less and less private data in our lives, and every time we share one of these little nuggets with our bank, or our electricity company, or our on-line associates-we-call-friends, is one more chance that it escapes into the hands of the foaming-at-the-mouth-terrorist-cracker-communist-nazi-right-wing-religious-fruitcake hordes. The highest bar for personal verification which any of my banks currently sets for me is a random choice from a set of personally entered questions, with a set of personally entered answers, for which I have to enter two randomly selected characters using my mouse. That's not bad for safe verification, and I'd have to be really impressed with their security if that was stored in the database by a passphrase-protected encryption key. With that bank I don't know what they do over the phone - I assume they've concluded they can trust my logged on persona enough that I can do what I want online, consequently I haven't had to call them and share those secrets with everyone in earshot. Maybe paranoid freaks like me will go back to a chequebook and close down on-line access to their bank accounts entirely when they find themselves having to supply a skin scraping in order to authorise their next $500 payment. "Please insert your finger in the AccuYou(tm) BloodSucker(tm) to proceed with this payment" - well, I guess it might cut down my spending! In any case, biometrics need to be understood before they can be used effectively and appropriately - and remotely over the phone is probably not one of the ways that they can be trusted to work.

John Goerzen: Late Summer

It s that time of the year again. Everything is changing, and maybe for the better. The days are getting shorter. When I left for work on my bicycle yesterday morning, it was still dark outside, and a little nippy. There s nothing quite like riding a bicycle down a deserted country road at night, a cool breeze at your back, and having the sun come up as you ride. And with the cooler weather, we can open our windows at night instead of running the air conditioner. It s also nice to have a pleasant cool breeze flowing through the house, and hear the frogs, crickets, owls, coyotes, and other wildlife at night. Out here, we certainly don t hear sounds of traffic, or loud car radios, though on a really clear night we might hear the rumble and whistle of a train from a few miles away. This is also sunflower season in Kansas. The wild sunflowers, when their smaller-than-most-people-think flowers, grow everywhere. Some ditches turn into a sea of person-height yellow. Sunflowers are on the sides of bridges, around people s mailboxes just about anywhere that isn t mowed or farmed. Then you also pass the sunflower fields, with their larger flowers, even more sea-like. The beans are getting tall in the fields this time of year, and it won t be long before the milo starts to turn its deep, dark reddish brown. But, you know, we re Kansans. We can t really let ourselves enjoy it all that much. Just today, I heard a conversation apparently the Farmer s Almanac is predicting a brutally frigid winter this year. Gotta keep our sense of pessimism about weather alive now

2 September 2009

Gunnar Wolf: Drupal 6 Tour Centroamerica Now in Mexico!

I met my friend Josef Daberning, who did his Austrian Social Service working with Drupal at the Casa de los Tres Mundos NGO, in Granada, Nicaragua, at the Central American Free Software Encounter, last May. He told me that, when going back to Austria, he would spend some days in Mexico, and wanted to give a workshop on Drupal. The course has just started, and will take place today and tomorrow You can follow the live stream at http://www.iiec.unam.mx:18000/drupal.ogg The videos will be uploaded soon as well, I will post them on this same node. This node will be used for whatever is needed to make public for people following the talk. As of right now, you can download his presentation http://gwolf.org/files/gira-drupal.odp and http://gwolf.org/files/gira-drupal.pdf [update](s):
AttachmentSize
Basic introduction to Drupal presentation (ODP format)306.07 KB
Basic introduction to Drupal presentation (PDF format)214.62 KB
Drupal themes presentation1.09 MB

Junichi Uekawa: Increased MacBook memory to 4GB from 1GB.

Increased MacBook memory to 4GB from 1GB. Now I should be able to run eclipse more smoothly, and things will not start swapping out.

Russell Coker: Healthcare and Free Software

The Washington Monthly has an interesting article about healthcare and Free Software [1]. It seems that a free system named VistA from the US Veterans Affairs department (not to be confused with the unpopular OS Vista that Microsoft released a few years ago) is competing against a range of proprietary software for managing patient data. VistA has apparently performed very well, it s cheap and easy to install, the data can be shared with other programs, and it was largely written by doctors and nurses so it s optimised to their needs. It has been proven that VistA has saved many lives through better management of medicines and through permitting statistical analysis of the results of various treatments. It has also allowed medical staff to work more quickly which reduces waiting times and medical expenses. But as you would expect in the US, whenever there is a way of saving lives while also saving money there are companies lobbying for the opportunity to make money while allowing innocent people to die. Of course there are established medical companies who are doing this now and have been doing so for some time. But the latest news is that Microsoft, Intel, Cisco, and Allscripts, are sponsoring the Electronic Health Record Stimulus Tour an aggressive measure to railroad doctors into buying proprietary software now. MS is known for totally ignoring the law when it gives them an opportunity to gain market share, but this is a new low. What will MS do next? Sell drugs to children?

Sami Haahtinen: SELinux and me...

Once in a while I get this urge to use SELinux on some of the servers I manage, but almost always run in to something that puts me back enough to never finish the project. This time I managed to figure out the last few glitches. In the end, SELinux still has a really steep learning curve, so it's not for the impatient ones. Even though enabling SELinux in Debian has become a lot easier since the first time I tried to get things running, it's still just the tip of the iceberg. In Debian it is just a matter of installing the right packages and running a few commands, but that's just where the troubles start. Most of the howtos focus on single user or shared installations where all users are created locally. Also most howtos fail to mention that you need to relabel files in certain cases. One of the most annoying problem I ran in to was changing all non-system users away from the unconfined_u class. This is of course done like this (found here):
semanage -m -s user_u __default__

The problem here is that it changes the existing user as well and you start to get errors like these:
denied    read   for  pid=32258 comm="bash" name=".profile" dev=dm-5 ino=185474 scontext=user_u:user_r:user_t:s0 tcontext=unconfined_u:object_r:unconfined_home_t:s0 tclass=file

The problem here is that the home directory for the user is still labeled for the wrong class. The fix is to relabel the home. Sadly this is something that you just need to know, it's not explained anywhere. At least I haven't found an explanation anywhere. Another good thing to do before you continue is to change the already existing user to the staff class. Staff class has a bit more relaxed security controls and you get to change the security roles (details here).
semanage login -m -s staff_u myuser
fixfiles relabel /home/myuser

This gets you a semi working setup, next problem usually is that some daemons are denied access to parts of your system. For me, this was postfix trying to access my home directory that was mounted over NFS. For such cases, you should persuade the maker of the module package to update the global policy if it's a common use case. Or you need to create a policy package that allows access to the given files. The process itself is documented in the audit2access manual page. In general you should study the audit2why and audit2allow packages. The former tells you if there is an easier way to fix something (like enable a boolean) and latter will create the required policy lines. Only problem is to compile the policy and load it. The only problem here was to find the right tool and the right lines from the manual. In general the SELinux learning curve is way too steep. It's a system that works pretty well once you learn all the tricks and start to completely understand the toolset. The community should continue working on lowering the bar for new users. There has been some major improvements since I first tried SELinux, so it's the right direction.

1 September 2009

Nicolas Valc rcel: Canoniversary!

It has been a whole year since i joined canonical on 1st September 2008, it s unbelievable a year went that fast, i m still getting used to the company and suddenly, a whole year just went away, but as some people say, time flights when you are having so much fun. I can describe this whole year experience in just one word: AMAZING! I m having so much fun, even if i m getting out of time for other things i wanted to do, everything has it s price and working for canonical isn t an easy task, a lot is expected from you all the time, but the company policy is to be friends more than co-workers, even if we don t see each other to much. In this year i ve had the pleasure of visiting some friends all over the world while traveling for work or escaping a little before or after a work trip, i ve also had the chance to meet some people i wouldn t otherwise, like a guy i met in Germany that develops this horrible scripts that learn from my shopping customs and offer me stuff i want to buy (i said that guy that i hate him between jokes :P , c mon you hate him as well) and learned things i wouldn t in a normal local based company, i learned almost the hard way that when you say Asian people you are talking about Indians as well (isn t it Polly?) and also notice how different cultures can be, some things that are completely natural and normal for me can be really disturbing and offensive for others and vice versa. I ve also have the joy of seeing some friends join the company and grow our friendship, as well as i ve seen some good friends leave the company, but the sadness was compensated with the happiness that they are going for something better and that the friendship will continue. It s also always fun how when we meet in person is like we have worked in the same office for ever, people is always keen to have some beers, chat and test Pisco (no, i will never get bored of seeing your faces when you try it!). In summary, canonical is like a big family, you know, how people say that there is competition even inside a same company, well, this is not the case, people is always happy to help you and give you a hand when you need it, and even i thought it wasn t possible to have a weekly meeting with your boss to talk about your personal lives before you start talking about work, it s always nice to know that they care about you, not only about numbers and results! That said i wanted to thank everyone in canonical, and the ones that already left for making this company such an amazing place to work! And special thanks for the two guys that helped me had this incredible opportunity (you know who you are!)

David Pashley: Setting up gitosis on Jaunty

While git is a completely distributed revision control system, sometimes the lack of a central canonical repository can be annoying. For example, you might want to make your repository published publically, so other people can fork your code, or you might want all your developers to push into (or have code pulled into) a central "golden" tree, that you then use for automated building and continuous integration. This entry should explain how to get this all working on Ubuntu 9.04 (Jaunty). Gitosis is a very useful git repository manager, which adds support like ACLs in pre-commits and gitweb and git-daemon management. While it's possible to set all these things up by hand, gitosis does everything for you. It is nicely configured via git; to make configuration changes, you push the config file changes into gitosis repository on the server. Gitosis is available in Jaunty, but unfortunately there is a bug in the version in Jaunty, which means it doesn't work out of the box. Fortunately there is a fixed version in jaunty-proposed that fixes the main problem. This does mean that you need to add the following to your sources.list:
deb http://gb.archive.ubuntu.com/ubuntu/ jaunty-proposed universe
Run apt-get update && apt-get install gitosis. You should install 0.2+20080825-2ubuntu0.1 or later. There is another small bug in the current version too, as a result of git removing the git-$command scripts out of /usr/bin. Edit /usr/share/python-support/gitosis/gitosis/templates/admin/hooks/post-update and replace
git-update-server-info
with
git update-server-info
With these changes in place, we can now set up our gitosis repository. On the server you are going to use to host your central repositories, run:
sudo -H -u gitosis gitosis-init < id_rsa.pub
The id_rsa.pub file is a public ssh key. As I mentioned, gitosis is managed over git, so you need an initial user to clone and then push changes back into the gitosis repo, so make sure this key belongs to a keypair you have available to the remote user you're going to configure gitosis. Now, on your local computer, you can clone the gitosis-admin repo using:
git clone gitosis@gitserver.example.com:gitosis-admin.git
If you look inside the gitosis-admin directory, you should find a file called gitosis.conf and a directory called keydir. The directory is where you can add ssh public keys for your users. The file is the configuration file for gitosis.
[gitosis]
loglevel = INFO
[group gitosis-admin]
writable = gitosis-admin
members = david@david
[group developers]
members = david@david
writable = publicproject privateproject
[group contributors]
members = george@wilber
writable = publicproject
[repo publicproject]
daemon = yes
gitweb = yes
[repo privateproject]
daemon = no
gitweb = no
This sets up two repositories, called publicproject and privateproject. It enables the public project to be available via the git protocol and in gitweb if you have that installed. We also create two groups, developers and contributors. David has access to both projects, but George only has access to change the publicproject. David can also modify the gitosis configuration. The users are the names of ssh keys (the last part of the line in id_dsa.pub or id_rsa.pub). Once you've changed this file, you can run git add gitosis.conf to add it to the commit, git commit -m "update gitosis configuration to commit it to your local repository, and finally git push to push your commits back up into the central repository. Gitosis should now update the configuration on the server to match the config file. One last thing to do is to enable git-daemon, so people can anonymously clone your projects. Create /etc/event.d/git-daemon with the following contents:
start on startup
stop on shutdown
exec /usr/bin/git daemon \
   --user=gitosis --group=gitosis \
   --user-path=public-git \
   --verbose \
   --syslog \
   --reuseaddr \
   --base-path=/srv/gitosis/repositories/ 
respawn
You can now start this using start git-daemon So now, you need to start using your repository. You can either start with an existing project or an empty directory. Start by running git init and then git add $file to add each of the files you want in your project, and finally git commit to commit them to your local repository. The final task is to add a remote repository and push your code into it.
git remote add origin gitosis@gitserver.example.com:privateproject.git
git push origin master:refs/heads/master
In future, you should be able to do git push to push your changes back into the central repository. You can also clone a project using git or ssh, providing you have access, using the following commands. The first is for read-write access over ssh and the second uses the git protocol for read-only access. The git protocol uses TCP port 9418, so make sure that's available externally, if you want the world to be able to clone your repos.
git clone gitosis@gitserver.example.com:publicproject.git
git clone git://gitserver.example.com/publicproject.git
Setting up GitWeb is left as an exercise for the reader (and myself because I am yet to attempt to set that up).
Read Comments (1)

Biella Coleman: Hello Ladies (aka learning from the lady geeks)

3692689082_dc8494bf13 Whenever I attend Debconf, the first day is always a bit disorienting due to the confluence of jet lag with the sea of male programmers swarming at the venue. I am not sure I should be admitting this but one of the first things that pops to mind is the very cheesy one hit wonder whose video is filled with many bikini clad men, Its raining men. So for a few hours, maybe a day, I sometimes feel out of place but this this first wave of discomfort usually gives way to comfort. This is not only because I connect with friends but because Debconf as well as many other hacker events and this sits in marked contrast to professional academic conferences are remarkably accepting of diversity and difference, at least that has been my personal experience. Most notably, people could care less about how you look, disabilities are consistently accommodated, and some things, like gender changing, which would raise serious eyebrows for instance in most academic conferences, is accepted with next to no gossip. I raise this because this is how I personally have experienced the world of Free Software: overwhelming male but overwhelming accepting of difference as well. So even while I think it is key to confront the problems of discrimination in this world, it is not a simple story, for again, it is an environment that is also accommodates difference and quirkiness to some notable degree. The question of gender in F/OSS is complicated one and here I am not going to ruminate on sources of gender bias except to say that I am sure there are some in operation (as there are in most domains) some of which are internal to Free Software and other s external to Free Software. But what I have been struck by, especially over the course of the summer, is the explosion of sites, blogs, and debates that have confronted gender in Free Software. It has been nothing short of astounding and a really positive turn. The most notable example is this AWEEEEEEEEEEESOME pair: a wiki and a blog that confront gender head on. These are notable because the wiki, for example, catalogs all sorts of controversial events, comments on them, leaving a very visible trace of debate, one that is necessary to change the gender make-up and dynamic within Free Software project. Along with these, the FSF is finally hosting a min-summit, which is great (less great is that participation seems invite only but perhaps they have some good reasons for doing so). And today I just learned of this diversity page coming out the Python project. These are perfect examples of the recursive public in action raised in Chris Kelty s work. And I have long been impressed with the dialogue that has followed from some controversial events in the world of Free Software, including those related to gender and this summer seems to be a watershed of sorts and I look forward to their developments over time. Indeed and this may be controversial as well, but I think my academic field of media, law, and culture has something to learn from these gender politics for there are some very real, though probably unintentional forms of discrimination that are not under that much active discussion. The most glaring problem is the underwhelming presence of female scholars during conferences (and as we know, conferences are exceedingly important for one s professional development and career). What I find most striking about this trend is the number of female scholars is significant. That is, when it comes to scholars and this seems different from the world of tech, there is a sizable community of women scholars and activists so when there is a 5% female participation , as for example, with this event, one has to wonder why is it raining men at these conferences? It also seems that while the debate exists, it is not as vibrant as with what is going on right now in F/OSS. Take for example, this recent mailing list post which unapologetically highlighted the lack of female presence in the up and coming Free Culture event at the Berkman Center. While the post generated a handful of thoughtful responses, including this supremely classic, biting, and quite clever response from Georgetown professor Julie Cohen, the debate did not linger on (however short, the posts and discussion were quite fruitful). What to do? Frankly, blame and finger pointing are pretty counterproductive, mostly because the discrimination is quite unintentional and I believe change can be brought about via more constructive paths. More important is we need to make the issue visible, identify some possible sources, and then create projects that can remedy the problem. Along with Elizabeth Stark, who has also been keen to note and change the gender problems, I am currently whipping together a wiki with a list of female scholars, leaders, and technologists as a resource for folks organizing conferences. We already have a base list (with a remarkably long list of folks) and will hopefully in the course of the next month throwing it up on the wiki. I don t think it will ever be as cool as the Geek Feminism Wiki but it will hopefully do some good!

Zak B. Elep: Returning to the fold

Hi again, blog. Its been a long time. Like, almost a year. How was I? Great, I guess. Mostly out of touch with the rest of the worlds I used to visit, being at work, but after leaving work (and searching for a new one now,) that's going to change. Hopefully. :) I've started by updating some stuff for Debian I should have updated long ago. In the process, I've reformatted my laptop to be Windows-free, containing only Debian sid and OpenBSD-current, and its been very pleasant so far. I've also taken up a couple of new old stuff for Debian, fixing them up, and getting comfortable with their insides^Wcode. So much so that I'm even in the process of fixing a 5-year-old wishlist bug, although it still needs testing. I'm looking forward to rejoining the communities I once missed, and maybe joining a new one too.

Adnan Hodzic: Debian Start! (beta)

First of all, thank you for all your comments/feedback on previous Start! entries. What kept me busy for last ~25hrs (literally) was making Start! to be properly rendered in all of the browsers and resolutions except IE of course. In the end, I ended up writing the whole thing from scratch, 709 lines which also passed W3 Validator, pretty sweet Smile Whole thing is a combination of html + css + php + javascript; it ll work fine even if you have JavaScript disabled, but for full experience I recommend you got it enabled. It works/renders properly in pretty much every browser (links, Iceweasel/Firefox, Epiphany, Konqueror, Opera, Chrome) except IE (surprisingly 4.15% of Start! users use IE). Either way, now that we have the rendering problem solved, what you can expect from next version is options dialog, with hopefully interesting options Smile After that, possible multilingual support (if required) and that should round up to the final v1.0 Smile Comments of any kind as always are welcome Grin start.debian.net

31 August 2009

Rapha&#235;l Hertzog: Interdependence in Debian, how to suffer less from it

Listening to Martin Krafft s talk at Debconf (related to his PhD) shed some new light on the idea that I expressed last year I wanted that each maintainer regularly answers a questionnaire so that he has to ask himself whether he does a good enough job with his packages. When thinking of this idea, I only saw the QA side of ensuring good maintenance on all packages, however I believe that the root problem lies further and this project would not be enough: we are interdependent but we are not equipped to deal with this reality. Martin s only merit has been to mention that we are interdependent, but it s worth analyzing a bit. Our organization is centered around individuals acting as package maintainers, and in theory each package maintainer can work on his corner and all goes well. We know that this model doesn t hold any more: transitions to testing require coordination of uploads and timely fixes of RC bugs, keeping up with the work frequently requires several volunteers that have to coordinate, etc. More and more of the work requires a level of availability that a single individual can t offer, yet in our day-to-day work we mainly interact with individuals. Wouldn t it be better if we could immediately know what we can expect from any Debian developer: All this information should be shared by all Debian maintainers (some of it is already available but either not publicly or not in any machine-parseable way) and we should actively use it. Here are some examples of use: for each RC bug report, you could look up if at least one maintainer is available and you could ping him explicitly if needed. When you plan an NMU, you could look up if the maintainer is likely to respond in the next day or not, and possibly adjust the number of days spent in the DELAYED queue. When organizing a large-scale transition, you could extract a list of packages whose maintainers are not available and arrange immediate NMUs. Furthermore there are many cases where the project s usual expectation exceed what the maintainer is ready to do. Documenting what part of the job is done (or not) by the maintainer makes it clear for volunteers whether their help is needed and whether they could/would be a better maintainer for a given package. Designing solutions to all these problems is going to be the scope of the DEP2 that I reserved some time ago. It s likely to be some sort of dedicated web interface. I would welcome supplementary drivers for this DEP, so if you re interested, get in touch with me. Partagez cet article / Share This

David Pashley: Setting up gitosis on Jaunty

While git is a completely distributed revision control system, sometimes the lack of a central canonical repository can be annoying. For example, you might want to make your repository published publically, so other people can fork your code, or you might want all your developers to push into (or have code pulled into) a central "golden" tree, that you then use for automated building and continuous integration. This entry should explain how to get this all working on Ubuntu 9.04 (Jaunty). Gitosis is a very useful git repository manager, which adds support like ACLs in pre-commits and gitweb and git-daemon management. While it's possible to set all these things up by hand, gitosis does everything for you. It is nicely configured via git; to make configuration changes, you push the config file changes into gitosis repository on the server. Gitosis is available in Jaunty, but unfortunately there is a bug in the version in Jaunty, which means it doesn't work out of the box. Fortunately there is a fixed version in jaunty-proposed that fixes the main problem. This does mean that you need to add the following to your sources.list:
deb http://gb.archive.ubuntu.com/ubuntu/ jaunty-proposed universe
Run apt-get update && apt-get install gitosis. You should install 0.2+20080825-2ubuntu0.1 or later. There is another small bug in the current version too, as a result of git removing the git-$command scripts out of /usr/bin. Edit /usr/share/python-support/gitosis/gitosis/templates/admin/hooks/post-update and replace
git-update-server-info
with
git update-server-info
With these changes in place, we can now set up our gitosis repository. On the server you are going to use to host your central repositories, run:
sudo -H -u gitosis gitosis-init < id_rsa.pub
The id_rsa.pub file is a public ssh key. As I mentioned, gitosis is managed over git, so you need an initial user to clone and then push changes back into the gitosis repo, so make sure this key belongs to a keypair you have available to the remote user you're going to configure gitosis. Now, on your local computer, you can clone the gitosis-admin repo using:
git clone gitosis@gitserver.example.com:gitosis-admin.git
If you look inside the gitosis-admin directory, you should find a file called gitosis.conf and a directory called keydir. The directory is where you can add ssh public keys for your users. The file is the configuration file for gitosis.
[gitosis]
loglevel = INFO
[group gitosis-admin]
writable = gitosis-admin
members = david@david
[group developers]
members = david@david
writable = publicproject privateproject
[group contributors]
members = george@wilber
writable = publicproject
[repo publicproject]
daemon = yes
gitweb = yes
[repo privateproject]
daemon = no
gitweb = no
This sets up two repositories, called publicproject and privateproject. It enables the public project to be available via the git protocol and in gitweb if you have that installed. We also create two groups, developers and contributors. David has access to both projects, but George only has access to change the publicproject. David can also modify the gitosis configuration. The users are the names of ssh keys (the last part of the line in id_dsa.pub or id_rsa.pub). Once you've changed this file, you can run git add gitosis.conf to add it to the commit, git commit -m "update gitosis configuration to commit it to your local repository, and finally git push to push your commits back up into the central repository. Gitosis should now update the configuration on the server to match the config file. One last thing to do is to enable git-daemon, so people can anonymously clone your projects. Create /etc/event.d/git-daemon with the following contents:
start on startup
stop on shutdown
exec /usr/bin/git daemon \
   --user=gitosis --group=gitosis \
   --user-path=public-git \
   --verbose \
   --syslog \
   --reuseaddr \
   --base-path=/srv/gitosis/repositories/ 
respawn
You can now start this using start git-daemon So now, you need to start using your repository. You can either start with an existing project or an empty directory. Start by running git init and then git add $file to add each of the files you want in your project, and finally git commit to commit them to your local repository. The final task is to add a remote repository and push your code into it.
git remote add origin gitosis@gitserver.example.com:privateproject.git
git push origin master:refs/heads/master
In future, you should be able to do git push to push your changes back into the central repository. You can also clone a project using git or ssh, providing you have access, using the following commands. The first is for read-write access over ssh and the second uses the git protocol for read-only access. The git protocol uses TCP port 9418, so make sure that's available externally, if you want the world to be able to clone your repos.
git clone gitosis@gitserver.example.com:publicproject.git
git clone git://gitserver.example.com/publicproject.git
Setting up GitWeb is left as an exercise for the reader (and myself because I am yet to attempt to set that up).
Read Comments (0)

Gunnar Wolf: Serverless and mailless

Oops. Yesterday (Sunday, 31/08/09) I far from any computer-like object for most of the day. When I got back home, of course, I promptly opened my laptop to check my mail who knows what destiny might have for me in a 24 hour period? Maybe I won yet another fortune I have to cash in Nigeria? Maybe there is (GASP!) a new RC bug on one of my packages? But no, my mail server didn't feel like answering to my ssh queries. The connection was established, but shut down before even sending the protocolary SSH-2.0-OpenSSH_5.1p1 string. Fearing an overload (after all, the little bugger is just a Mac Mini running in another room in my house), I tried to check (via Web) its Munin status Apache didn't want to listen either. It answered, but got only access denied. Things started worrying me But (silly me) not enough The machine runs headless1, so I just danced the boring raising elephants song2. Allowed for a couple of minutes for everything to settle, and tried to connect. Horror, now even pings didn't work! So I ran to fetch my old, bulky and trusty monitor. Went back to the machine, plugged it in, switched it off and back on. Everything worked fine this time At least appearingly. I opened up mutt and started happily reading mails, while trying to understand on another console what happened at 07:06 that didn't get logged anywhere and had the machine dead for basically all the day. And then, BRRRT-BRRRT-BRRRT, I started hearing the HDD seeking. I was able to send a couple of mails, but decided to let the machine rest and... Will reduce its disk usage to an absolute minimum. Fortunately, I have already the machine meant to replace it A much nicer, beefier iMac G5, waiting to be vacated from its data, task which has suddenly become prioritary. So, in short: If you need to get in touch with me in the next day or two, don't count on my usual @gwolf.org mail, as it is down. I hope to be able to get the data out of the poor little bugger painlessly after it rests a bit. And I hope not to drown in a sea of mails after I get the replacement back online :-/
  1. 1. for those not used to computerspeak: without a connected monitor
  2. 2. For those following at home who don't understand how Raising Elephants Is So Utterly Boring, this is a (silly, stupid but useful) mnemotecnic for as-properly-as-possible restarting hung Linux systems: Hold AltGr (right-Alt) + SysRq (in reduced/notebook keyboards, Fn+SysRq), and type R E I S U B (leaving a couple of seconds between key, ideally waiting for all disk activity to settle between each). That means:
    unRaw
    Take control of keyboard back from X,
    tErminate
    Send SIGTERM to all processes, allowing them to terminate gracefully,
    kIll
    send SIGKILL to all processes, forcing them to terminate immediately,
    Sync
    Flush data to disk),
    Unmount
    Remount all filesystems read-only,
    reBoot.
    Well... Reboot the system

Robert Collins: 31 Aug 2009

Hi Rich! Re hour+long unit tests I agree that you need a comprehensive test suite, and that it should test all the dark and hidden corners of your code base. But time is not free! A long test suite inhibits:
  • cycle time - the fastest you can release a hot fix to a customer
  • developer productivity - you can't forget about a patch till its passed the regression test suite
  • community involvement - if it takes an hour to run the test suite, an opportunistic developer that wanted to tweak something in your code will have walked away long ago Note that these points are orthogonal to whether developers edit-test cycle runs some or all tests, or whether you use a CI tool, or a test-commit tool, or some other workflow. All that said though, I'm extremely interested in *why* any given test suite takes hours: does it need to? What is it doing? Can you decrease the time by 90% and coverage by 2%? I got another response back, which talks about keeping the working set of tests @ about 5 minutes long and splitting the rest off (via declared metadata on each test) into 'run after commit or during CI'. This has merits for reducing the burden on a developer in their test-commit cycle, but as I claim above, I believe there is still an overhead from those other tests that are pending execution at some later time. From a LEAN perspective, the cycle time is very important. Another important thing is handoffs. Each time we hand over something (e.g. a code change that I *think* works because it passed my local tests), there is a cost. Handing over to a machine to do CI is just as expensive as handing to a colleague. Add that contributors sending in patches from the internet may not hang around to find out that their patch *fails* in your CI build, and you can see why I think CI tools are an adjunct to keeping a clean trunk, rather than a key tool. The key tool is to not commit regressions :) Oh, and I certainly accept that test suites should be comprehensive... I just don't accept that more time == more coverage, or that there isn't a trade off between comprehensive and timeliness.
  • Romain Beauxis: Some quick notes on netboot install

    Since I recently got a new laptop, I wanted to install Debian on it, of course :) However, I really didn't feel like buying a new CDR and burning yet another single-use boot CD and did not have a USB key available. Hence, I looked at installing through netboot using the previous one as the boot server and the available documentation. Well it turned out to be much more easy than what I was expecting initially ! The quick notes around this are:
    - First you need to setup iptables for ip forwarding and masquerading, in order to be able to connect the new laptop to the internet:
    • ifconfig eth0 192.168.1.1
    • echo 1 > /proc/sys/net/ipv4/ip_forward
    • iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE

    - Then you need to enable dhcp and bootp. This means simply:
    • apt-get install dhcp3-server
    • Copy configuration example in Section 4.5.2.1 of the documentation
    • Edit with your own values, in particular check the nameserver used by the laptop sharing the connection. Also the tftp server will be your laptop, so set its address to 192.168.1.1 (in our case). Eventually, you need the MAC address of the new laptop (check ifconfig on it for instance)

    - Eventually, you have to enable the tftp server, which only means:
    • apt-get install tftpd-hpa
    • The server serves files located in /srv/tftp (check ps -Af). You have to download the netboot tarball and untar its content in this directory.
    The only strange thing here is that for some reason, the ethernet card would loose its address when the new laptop was using the DHCP to obtain an address. I didn't check it, but setting it at a regular intervals solved the problem: while true; do sleep 0.1; ifconfig eth0 192.168.1.1; echo "done"; done. Yeah, that's hacky, but I wanted a quick solution :) After all of that, you boot the new laptop, select the network boot and... voila ! In conclusion, since it is not that difficult to setup, it would probably be nice to have some sort of standard script to enable PXE boot on a computer. Then we could claim that the new Debian release is committed to sustainable development and allows to install without wasting a CD :-D

    30 August 2009

    Filip Van Raemdonck: 363 days lead time

    I would like to point out that my last article was in no way meant to be critical towards libvirt, rather on the contrary. I think it's an awesome technology the way it provides a common management platform for various virtualization solutions available on Linux and even other platforms.

    For reference, expensive proprietary virtualization solutions don't necessarily fair better.
    I've in fact recently ran into a problem in a Vmware ESX / Virtual Center environment which, while rather different in manifestation, in the end was similar in nature: diverging state change in the environment and in configuration.

    The Vmware problem surfaced after kernel changes on Linux guests, where, after rebooting into the new kernel and running the vmware-config-
    tools script to rebuild the modules, nightly backups with help of the vcbmount utility would fail.
    The only cryptic error message was:

    Creating a quiesced snapshot failed because the (user-supplied) custom pre-freeze script in the virtual machine exited with a non-zero return code.


    It didn't help that there wasn't any "user-supplied pre-freeze script" to begin with.

    Searching for this error message returned lots of hits, several going at least one year back. However, aside from a few references to a documented Solaris guest issue, which was not applicable, none would actually provide any insight into what the problem might actually be. Unt
    il I found following post on the Vmware communities site, which pointed out that
    changes to the uuid.location weren't propagated properly to the vmx definition.

    For the time being, the best way to work around this (in Linux guests) is to add another reboot right after rebuilding the vmtools modules.
    Since you should be rebuilding
    But Vmware should provide a proper fix their stuff, really. I've got this strong feeling that the libvirt bug will be resolved in less time than a year...

    Eugene V. Lyubimkin: debdelta rocks even more

    Starting with cupt 0.5.2, I'm able to use debdelta for upgrading my packages. Usually it saves 30-50% of traffic costs for me, which is already good.

    Several days ago I felt service was in outage because no new deltas were available, but today it saved 96% of traffic! I guess the reason is change of default delta algorithm on Andrea's site with deltas, but hey, whatever, 1MiB instead of 27MiB! Debdelta rocks.

    Robert Collins: 30 Aug 2009

    Made some time to hack... the results: config-manager 0.4 released, re-uploaded to debian (it was removed due to some confusion a while back). This most notably drops the hard dependency on pybaz and adds specific-revision support for bzr. subunit snapshot packaging sorted out to work better with subunit from Ubuntu/Debian. This latest snapshot has nested progress and subunit2gtk included. PQM got a bit of a cleanup:
  • The status region shown during merges is ~ twice as tall now.
  • if the precommit_hook outputs subunit it will be picked up automatically and shown in the status region.
  • all deprecation warnings in python2.6 are cleaned up
  • Pending bugfixes were merged from Tim Cole and Daniel Watkins - thanks guys!
  • Next.

    Previous.