Search Results: "xam"

30 August 2010

Richard Darst: The DebConf registration process

The DebConf registration team has the hard job of coordinating accommodation, food, and payments for hundreds of people.There are two major dates here: the sponsored registration deadline, and the reconfirmation deadline. There is also the date of opening registration, but that's usually more a matter of "how soon can we?" than "when should we?"The sponsored registration deadline (April 15 for DC10) is the deadline for requesting sponsored accommodation, food, or travel. This allows us to figure out how much we'll have to pay for attendees - attendees who register after this point should pay enough to cover their own costs.The reconfirmation deadline (June 10th for DC10) is used as a ping for attendees, particularly the sponsored ones. Attendees must reconfirm they will attend, so that we don't waste money reserving more food/rooms than we will have people using them. This deadline usually depends on the date we must give firm numbers to our accommodations.Historically, registration hasn't ever closed, anyone can come, just if you register too late you won't necessarily get any free stuff (and definitely no free food/accommodation). The deadline for food and accommodation themselves has been based on the logistics of the conference itself - DC9, when we ran out of physical rooms for accommodation, and DC10, when we had to give firm housing numbers to Columbia. Food has tended to be easier as caterers could take updates on shorter notice, usually the day before.DebConf8 began a system where attendees could pay a registration fee if they desired. This was encouraged, for example, for attendees whom could get this fee reimbursed by their company, but was not required nor encouraged of sponsored attendees. DC8-9 had two classes of fees: Professional, covering the actual costs, and Corporate, covering about double of professional and for companies or registrants whom could provide extra financial support for DebConf. DebConf10 added an extra layer of "pay per day" on top of this.It's up to the registration team to pin down attendee numbers as early as possible, as this is usually the biggest cost. DebConf provides food and accommodations to a huge number of people, but the numbers are constantly changing. It's hard to know how many people will drop out at the last minute. It's hard to know how many people will actually pay a corporate or professional fee, as opposed to mistakingly selecting it when they'd rather something else. Our registration system can't support all the little intricacies of how much an attendee will cost us, or pay us. Without knowing how many people will attend, how can we accurately get the right number of meals and right number of rooms? Sponsored people may always choose to not eat some meals with us, possibly saving us money there. How many people with travel sponsorship will actually request their full reimbursement?The registration system, Pentabarf, is used to track attendance numbers, but it isn't completely reliable. Not everyone in it will attend all days, and most people who aren't staying with us don't have accurate dates of attendance. Thus, it can only be used as estimates, not for all decisions.
The difficulties above are challenges for the registration and budgeting teams, but are not too bad - they just require thought, time, and answering many questions from attendees.Tags: debconf

27 August 2010

Colin Watson: Windows applications making GRUB 2 unbootable

If you find that running Windows makes a GRUB 2-based system unbootable (Debian bug, Ubuntu bug), then I'd like to hear from you. This is a bug in which some proprietary Windows-based software overwrites particular sectors in the gap between the master boot record and the first partition, sometimes called the "embedding area". GRUB Legacy and GRUB 2 both normally use this part of the disk to store one of their key components: GRUB Legacy calls this component Stage 1.5, while GRUB 2 calls it the core image (comparison). However, Stage 1.5 is less useful than the core image (for example, the latter provides a rescue shell which can be used to recover from some problems), and is therefore rather smaller: somewhere around 10KB vs. 24KB for the common case of ext[234] on plain block devices. It seems that the Windows-based software writes to a sector which is after the end of Stage 1.5, but before the end of the core image. This is why the problem appears to be new with GRUB 2. At least some occurrences of this are with software which writes a signature to the embedding area which hangs around even after uninstallation (even with one of those tools that tracks everything the installation process did and reverses it, I gather), so that you cannot uninstall and reinstall the application to defeat a trial period. This seems like a fine example of an antifeature, especially given its destructive consequences for free software, and is in general a poor piece of engineering; what happens if multiple such programs want to use the same sector, I wonder? They clearly aren't doing much checking that the sector is unused, not that that's really possible anyway. While I do not normally think that GRUB should go to any great lengths to accommodate proprietary software, this is a case where we need to defend ourselves against the predatory practices of some companies making us look bad: a relatively small number of people do enough detective work to realise that it's the fault of a particular Windows application, but many more simply blame our operating system because it won't start any more. I believe that it may be possible to assemble a collection of signatures of such software, and arrange to avoid the disk sectors they have stolen. Indeed, I have a first draft of the necessary code. This is not a particularly pleasant solution, but it seems to be the most practical way around the problem; I'm hoping that several of the programs at fault are using common "licence manager" code or something like that, so that we can address most of the problems with a relatively small number of signatures. In order to do this, I need to hear from as many people as possible who are affected by this problem. If you suffer from this problem, then please do the following: I hope that this will help me to assemble enough information to fix this bug at least for most people, and of course if you provide this information then I can make sure to fix your particular version of this problem. Thanks in advance!

Richard Darst: DebConf budgeting for a single conference

My last post discussed the year-to-year budgeting of DebConf. This post discusses what makes budgeting difficult within one particular DebConf. Since every DebConf is unique in its arrangements, the particular difficulties always vary year-to-year. For example, DebConf10 was easy in that Software in the Public Interest was in the same country so that we had easy access to bank accounts, but more difficult venue payment logistics. I'll use DebConf10 as an example below.
For DebConf10, we get our venue "for free" thanks to the support of the Columbia Computer Science department. However, we had to pay for guard costs after building hours, and custodial costs. For custodial costs at least, we had to go through many layers of bureaucracy, and people kept insisting it would be more than they quoted. (It ended up being about half, since DebConf attendees were so responsible!)DebConf sponsors many attendees. This means that all their food and accommodation costs are paid by us. We have sponsored registration and reconfirmation deadlines to set a maximum of the number of people we have to sponsor, but people are continually dropping out and it's hard to know just how many people to reserve for. Sponsored people won't always eat every meal with us, and usually won't decide until the day before - so figuring out food amounts is hard, too. Furthermore, we have unsponsored people who will want food and accommodations (paying themselves) which we must somehow plan for.Different venues operate on different payment rules. This year, CU Housing was fairly strict on asking for attendee counts and room assignments two weeks in advance. Then, as time gets closer, they got more flexible, and in the end told us that they will, in fact, only charge based on number of people actually attending - a huge win for us. Similarly with CU Dining, we had to go through great lengths to make sure that we would pay only for meals we ate. At CU, food had to be paid in advance, but accommodation and other costs are paid afterwards.We have to guess how much will be spent on miscellaneous supplies once DebConf starts. We have to then decide how much to spend on extras such as the day trip or conference dinner, and how to sponsor those, if possible.Debian (and thus DebConf) has money held in trust in many currencies in different accounts. Different sponsors will pay to different currencies, depending on what is convenient for them. This means that we need to consider fluctuating currency conversion rates, and decide if and when to transfer money. In general, DebConf hasn't transfer money just to make sure the DebConf-tagged money gets spent first. Some surpluses are best left in other countries to help pay future travel sponsorship since it is expensive to move money out of the country. Some conferences may result in a net movement of money from USA to Europe (more donors in one, more expenses in the other), some the other way around.Travel sponsorship is in the form of reimbursement. These can take many months to have the documentation received from attendees and the reimbursements all sent out, and thus finalizing the budget and surplus for next year.The overall principle we've tried to stick to is plan for a balanced budget in the worst case scenario, and then be happy when it ends up better than that.
At the DebConf&Debian BOF, someone mentioned how budgeting for any (free software) conference is hard. I think that's certainly true. Some of the registration fluctuations could be made easier by being less flexible, but DebConf wants to serve our attendees even if it is a bit of extra work for us. Each DebConf will have its own ways budgeting is difficult, but things have always work out, and usually with a bigger surplus than we were expecting.Tags: debconf

26 August 2010

Sylvain Le Gall: CentOS 5 chroot with schroot

OCaml compiles native executables in static mode. It allows to have a minimal set of dependencies when delivering an executable. It has also disadvantages like the size of the executable and problems arising when considering libraries update -- but this is another topic. There is still one strong dependency that you should not forget when you want to deliver a product for most of the Linux distributions: dependency on the glibc version. Trying to run OASIS compiled with Debian Lenny, on CentOS 5.5:
$ OASIS
.../OASIS: /lib64/libc.so.6: version  GLIBC_2.7' not found (required by .../OASIS)
So when compiling for delivery, one should choose the oldest distribution he targets. In my case, I choose CentOS 5 which comes with glibc v2.5. I usually choose Debian stable at the moment of writing Debian Lenny. But for now, the Debian Lenny's glibc is newer (v2.7) than the one coming from the CentOS 5.5 stable release. CentOS is a Red Hat like Linux distribution. I use a Debian Lenny amd64 host system and I decided to setup a chroot of CentOS 5 i386 and amd64. I also setup schroot to use my CentOS chroot. CentOS 5 amd64 setup First of all we use rinse, which can setup a RPM based distribution in a chroot. The version v1.3 shipped with Debian Lenny has some bugs: it doesn't install nss and other mandatory packages. So I downloaded v1.7 directly from Debian Sid. There is no dependencies problems and the package is arch:all, so it is straightforward to install:
$ wget http://ftp.de.debian.org/debian/pool/main/r/rinse/rinse_1.7-1_all.deb # Replace ftp.de.debian.org by your preferred Debian mirror
$ dpkg -i rinse_1.7-1_all.deb
Then I create the chroot directory and launch rinse:
$ mkdir /srv/chroot/centos5-amd64
$ rinse --arch amd64 --distribution centos-5 --directory /srv/chroot/centos5-amd64 # N.B. you must use --arch, the default is i386
Once installation is complete, you can add an entry for this distribution in /etc/schroot/schroot.conf:
[centos5-amd64]
description=Centos 5 (amd64)
location=/srv/chroot/centos5-amd64
priority=3
users=XXX
groups=
root-groups=root
type=directory
run-setup-scripts=true
run-exec-scripts=true
Replace XXX by your login. If you try to login directly, you will get warnings:
$ schroot -c centos5-i386
I : [chroot centos5-i386-a952de23-7f4b-4bae-a9b9-752ecee4a185] Ex cution de l'interpr teur de commandes initial :  /bin/bash 
-bash: /dev/null: Permission denied
-bash: /dev/null: Permission denied
-bash: /dev/null: Permission denied
-bash: /dev/null: Permission denied
-bash: /dev/null: Permission denied
This is a bit misleading because the real problem is that nothing is created in /dev/. CentOS delegates creating char/block devices to udev. You have two solutions to solve this issue:
$ MAKEDEV random
$ MAKEDEV console
$ MAKEDEV zero
$ MAKEDEV null
$ MAKEDEV stdout
$ MAKEDEV stdin
$ MAKEDEV stderr
$ rsync -av /srv/chroot/lenny-amd64/dev/* /srv/chroot/centos5-amd64/dev/
That's it, you now have a functional chrooted CentOS 5 environment:
$ schroot -c centos5-amd64 cat /etc/redhat-release
I : [chroot centos5-amd64-b9bae264-285b-4d17-a046-13386736cecd] Ex cution de la commande :   cat /etc/redhat-release  
CentOS release 5.5 (Final)
CentOS 5 i386 setup To setup an i386 environment, we follow almost the same scheme, except we need to fix a bug in rinse v1.7: we need to call linux32 before executing chroot. The problem is that the first stage installation of rinse install an i386/686 environment but as soon as you call chroot yum install ..., it will guess that the system is amd64 and will install missing packages. See the Debian bug report and the example patch attached to correct this behavior. WARNING: this patch is just an example, you can apply it for creating CentOS i386 chroot on Lenny amd64 host but you should remove the patch as soon as the installation is complete.
$ mkdir /srv/chroot/centos5-i386/
$ rinse --arch i386 --distribution centos-5 --directory /srv/chroot/centos5-i386 # With /usr/lib/rinse/centos-5/post-install.sh patched 
$ rsync -av /srv/chroot/lenny-i386/dev/* /srv/chroot/centos5-i386/dev/
Add this distribution to /etc/schroot/schroot.conf:
[centos5-i386]
description=Centos 5 (i386)
location=/srv/chroot/centos5-i386
priority=3
users=XXX
groups=
root-groups=root
type=directory
run-setup-scripts=true
run-exec-scripts=true
personality=linux32
You now have a schroot of CentOS 5 i386:
$ schroot -c centos5-i386 cat /etc/redhat-release
I : [chroot centos5-i386-9acafa91-9862-4488-aaef-4ab2a482771e] Ex cution de la commande :  cat /etc/redhat-release 
CentOS release 5.5 (Final)
Happy schroot hacking!

Uwe Hermann: openbiosprog-spi, a DIY Open Hardware and Free Software USB-based SPI BIOS chip flasher using flashrom

openbiosprog-spi device If you're following me on identi.ca you probably already know that I've been designing a small PCB for a USB-based SPI chip programmer named openbiosprog-spi. The main use-case of the device is to help you recover easily from a failed BIOS upgrade (either due to using an incorrect BIOS image, due to power outages during the flashing progress, or whatever). The device only supports SPI chips, as used in recent mainboards (in DIP-8 form factor, or via manual wiring possibly also soldered-in SO-8 variants). It can identify, read, erase, or write the chips. Of course the whole "toolchain" of software tools I used for creating the hardware is open-source, and the hardware itself (schematics and PCB layouts) are freely released under a Creative Commons license (i.e., it's an "Open Hardware" device). The user-space source code is part of flashrom (GPL, version 2), the schematics and PCB layouts are licensed under the CC-BY-SA 3.0 license and were created using the open-source Kicad EDA suite (GPL, version 2). openbiosprog-spi schematics
openbiosprog-spi Kicad PCB layout The schematics, PCB layouts, and other material is available from gitorious:
  $ git clone git://gitorious.org/openbiosprog/openbiosprog-spi.git
You can also download the final Gerber files (ZIP) for viewing them, or sending them to a PCB manufacturer. Some more design notes: Basic usage example of the device on Linux (or other OSes supported by flashrom):
  $ flashrom -p ft2232_spi:type=2232H,port=A -r backup.bin (reads the current chip contents into a file)
openbiosprog-spi PCBs
openbiosprog-spi parts list Over at the main projects page of openbiosprog-spi at http://randomprojects.org/wiki/Openbiosprog-spi I have put up a lot more photos and information such as the bill of materials, the Kicad settings I used for creating the PCBs, the Gerber files and the Excellon drill files and so on. The first few prototype boards I ordered at PCB-POOL.COM (but you can use any other PCB manufacturer of course), the bill of materials (BOM) lists the Mouser and CSD electronics part numbers and prices, but you can also buy the stuff elsewhere, of course (Digikey, Farnell, whatever). I already hand-soldered one or two prototypes and tested the device. Both hardware and software worked fine basically, you just need a small one-liner patch to fix an issue in flashrom, but that should be merged upstream soonish. In order to make it easy for interested users to get the PCBs I'll probably make them available in the BatchPCB Market Place soonish, so you can easily order them from there (you do still need to solder the components though). Note: I'm not making any money off of this, this is a pure hobby project. All in all I have to say that this was a really fun little project, and a useful one too. This was my first hardware project using Kicad (I used gEDA/PCB, also an open-source EDA toolsuite, for another small project) and I must say it worked very nicely. I didn't even have to read any manual really, it was all pretty intuitive. Please consider not using Eagle (or other closed-source PCB software) for your next Open Hardware project, there are at least two viable open-source options (Kicad, gEDA/PCB) which both work just fine.

25 August 2010

Manoj Srivastava: Manoj: Refreshing GNUPG keys sensibly

It has come up on the Planet recently, as well as the gnupg users mailing list: users need to refresh keys that they use to get updated information on revocations and key expiration. And there are plenty of examples of simple additions to ones crontab to set up a key refresh. Of course, with me, things are rarely that simple. Firstly, I have my GNUPGHOME set to a non standard location; and, secondly, I like having my Gnus tell me about signatures on mails to the Debian mailing lists, so I periodically sync debian-keyring.gpg into my GNUPGHOME. I add this as an additional keyring in my gpg.conf file, so that in normal operations gnus has ready access to the keys; but I do not care to refresh all the keys in debian-keyring. I also prefer to trust and update keys in my keyring, so the commands grow a little complex. Also, I want to get keys for any signatures folks have kindly added to my key and uploaded to the key server (not everyone uses caff), so just refresh-keys does not serve. Linebreaks added for readability.
# refresh my keys
# Note how I have to dance around keyring specification
45      4    *  * 4   (/usr/bin/gpg2 --homedir ~/.sec --refresh-keys
    $(/usr/bin/gpg2 --options /dev/null --homedir ~/.sec
      --no-default-keyring --keyring pubring.gpg --with-colons
        --fixed-list-mode --list-keys   egrep '^pub'   cut -f5 -d:  
            sort -u) >/dev/null 2>&1)
# Get keys for new sigs on my keys (get my key by default, in case 
# there are no unknown user IDs [do not want to re-get all keys])
44      4    *  * 5  (/usr/bin/gpg2 --homedir ~/.sec --recv-keys
    0xC5779A1C $(/usr/bin/gpg2  --options  /dev/null --homedir ~/.sec
       --no-default-keyring --keyring pubring.gpg --with-colons
         --fixed-list-mode --list-sigs 0xC5779A1C   egrep '^sig:'  
            grep 'User ID not found'   cut -f5 -d:   sort -u) >/dev/null 2>&1)

Matt Zimmerman: DebConf 10: Last day and retrospective

DebConf continued until Saturday, but Friday the 6th was my last day as I left New York that evening. I m a bit late in getting this summary written up. Making Debian Rule, Again (Margarita Manterola) Marga took a bold look at the challenges facing Debian today. She says that Debian is perceived to be less innovative, out of date, difficult to use, and shrinking as a community. She called out Ubuntu as the elephant in the room , which is taking away from Debian. She insists that she is not opposed to Ubuntu, but that nonetheless Ubuntu is to some extent displacing Debian as a focal point for newcomers (both users and contributors). Marga points out that Debian s work is still meaningful, because many users still prefer Debian, and it is perceived to be of higher quality, as well as being the essential basis for derivatives like Ubuntu. She conducted a survey (about 40 respondents) to ask what Debian s problems are, and grouped them into categories like motivation and communication (tied for the #1 spot), visibility (#3, meaning public awareness and perception of Debian) and so on. She went on to make some suggestions about how to address these problems. On the topic of communication, she proposed changing Debian culture by: This stimulated a lot of discussion, and most of the remaining time was taken up by comments from the audience. The video has been published, and offers a lot of insight into how Debian developers perceive each other and the project. She also made suggestions for the problems of visibility and motivation. These are crucial issues for Debian devotees to be considering, and I applaud Marga for her fortitude in drawing attention to them. This session was one of the highlights of this DebConf, and catalyzed a lot of discussion of vital issues in Debian. Following her talk, there was a further discussion in the hallway which included many of the people who commented during the session, mostly about how to deal with problematic behavior in Debian. Although I agreed with much of what was said, I found it a bit painful to watch, because (ironically) this discussion displayed several of the characteristic people problems that Debian seems to have: These same patterns are easily observed on Debian mailing lists for the past 10+ years. I exhibited them myself when I was active on these lists. This kind of cultural norm, once established, is difficult to intentionally change. It requires a fairly radical approach, which will inevitably mean coping with loss. In the case of a community, this can mean losing volunteer contributors cannot let go of this norm, and that is an emotionally difficult experience. However, it is nonetheless necessary to move forward, and I think that Debian as a community is capable of moving beyond it. Juxtaposition Given my history with both Debian and Ubuntu, I couldn t help but take a comparative view of some of this. These problems are not new to Debian, and indeed they inspired many of the key decisions we made when founding the Ubuntu project in 2004. We particularly wanted to foster a culture which was supportive, encouraging and welcoming to potential contributors, something Debian has struggled with. Ubuntu has been, quite deliberately, an experiment in finding solutions to problems such as these. We ve learned a lot from this experiment, and I ve always hoped that this would help to find solutions for Debian as well. Unfortunately, I don t think Debian has benefited from these Ubuntu experiments as much as we might have hoped. A common example of this is the Ubuntu Code of Conduct. The idea of a project code of conduct predates Ubuntu, of course, but we did help to popularize it within the free software community, and this is now a common (and successful) practice used by many free software projects. The idea of behavioral standards for Debian has been raised in various forms for years now, but never seems to get traction. Hearing people talk about it at DebConf, it sometimes seemed almost as if the idea was dismissed out of hand because it was too closely associated with Ubuntu. I learned from Marga s talk that Enrico Zini drafted a set of Debian Community Guidelines over four years ago in 2006. It is perhaps a bit longand structured, but is basically excellent. Enrico has done a great job of compiling best practices for participating in an open community project. However, his document seems to be purely informational, without any official standing in the Debian project, and Debian community leaders have hesitated to make it something more. Perhaps Ubuntu leaders (myself included) could have done more to nurture these ideas in Debian. At least in my experience, though, I found that my affiliation with Ubuntu almost immediately labeled me an outsider in Debian, even when I was still active as a developer, and this made it very difficult to make such proposals. Perhaps this is because Debian is proud of its independence, and does not want to be unduly influenced by external forces. Perhaps the initial growing pains of the Debian/Ubuntu relationship got in the way. Nonetheless, I think that Debian could be stronger by learning from Ubuntu, just as Ubuntu has learned so much from Debian. Closing thoughts I enjoyed this DebConf very much. This was the first DebConf to be hosted in the US, and there were many familiar faces that I hadn t seen in some time. Columbia University offered an excellent location, and the presentation content was thought-provoking. There seemed to be a positive attitude toward Ubuntu, which was very good to see. Although there is always more work to do, it feels like we re making progress in improving cooperation between Debian and Ubuntu. I was a bit sad to leave, but was fortunate enough to meet up with Debian folk during my subsequent stay in the Boston area as well. It felt good to reconnect with this circle of friends again, and I hope to see you again soon. Looking forward to next year s DebConf in Bosnia

Kumar Appaiah: Browsing with Elinks and using hooks

Even in the modern times of AJAX, Web 2+ and what not, Elinks is one of my favourite browsers, at least for browsing some sites. The reasons are simple: it cuts the unnecessary distractions, and allows you to digest text based information rather quickly. In addition, since it doesn't display any images or flashy elements like Flash, pages load lightning fast even on poor connections, making it an excellent option.Unfortunately, some sites don't really work very well with Elinks, and for those, I do one of two things. For the ones which are really complex, such as secure login sites with too much AJAX, I just resort to one of the "standard" graphical browsers. For some others, I try to work around the problem using Elinks' rich scripting interface available in many programming languages including Perl and Ruby. I chose Lua, since it seemed simple enough to learn. I wanted to share with you some of the simple workarounds I implemented for some sites I visit often.The first quirk is with the Debian BTS, whose CSS isn't properly interpreted by Elinks (see 593804 and 593840 for details). The short story: the BTS renders like this:So, I grabbed the CSS, and made some small changes (described in the above mentioned bug reports), and got a new CSS file. I put it in /home/kumar/Kumar/bugs.css, and added the following to my ~/.elinks/hooks.lua:
-- Use an alternate CSS for the Debian BTS to work around some quirks
function debian_bts_format_html_hook (html)
   return string.gsub(html, "/css/bugs.css", 'file:///home/kumar/Kumar/bugs.css')
end
function pre_format_html_hook (url, html)
   if sstrfind (url, "bugs.debian.org") then
      return blonnet_format_html_hook(html)
   end
   return nil
end
and with it, the BTS renders like this:Much more readable!The second simple example, for which I won't include images, is the Mobile Twitter interface. The mobile Twitter interface is very convenient and functional in Elinks; however, by default, the text font seems to be a light grey, which is not discernible in the bright background. A default user.css file would be a possible solution, but I have not decided whether to go for it just yet. So, for the Twitter site, I add a function like this:
-- Make the status messages black on Mobile Twitter
function mobile_twitter_format_hook(html)
   return string.gsub(html, ".status.- ", '.status  word-wrap: break-word; color : #000; ')
end
And alter the actual hook function to:
function pre_format_html_hook (url, html)
   if sstrfind (url, "bugs.debian.org") then
      return debian_bts_format_html_hook(html)
   elseif sstrfind (url, "mobile.twitter.com") then
      return mobile_twitter_format_hook(html)
   end
   return nil
end
Finally, a more complicated example. I like reading the Indian business newspaper the Hindu Business Line. However, if you try browsing to the site with Elinks, you'll notice that, barring the top hyperlinks in each sectcion, the links on the left are not "linked" at all.In addition, there are some other irritants which are easy to remove, such as the navigation bar on the top, and the text colour being too light for my taste etc. So, I write the following Lua blurb. Why I write this is left for the interested readers to figure out, though it shouldn't be surprising if you open the page. The workaround for the missing links is interesting, though; it is because the website authors have put a hyperlink (<a>) tag before a list item (<li>) tag, and Elinks seems to not like that too much. So, I just swap the two around, and Elinks is happy.
-- Reformat the Business Line website
function blonnet_format_html_hook (html)
   html = string.gsub(html, "<body ", '<body text="#000000"') -- I like the text in black on the white background
   html = string.gsub(html, "<[Ff][Oo][Nn][Tt] class=leftnavi color=brown>.-</[Ff][Oo][Nn][Tt]>", '') -- An ugly hack to remove the navigation bar
   html = string.gsub(html, "<TD align=left vAlign=top width=\"500\"><IMG.-</FONT><BR>", '', 1) -- Remove description
   html = string.gsub(html, "<A class=groupnavi href=\"http://epaper.thehindubusinessline.com/\">.-</FONT><BR>", '', 1) -- Remove other group links
   html = string.gsub(html, "(<[Aa] class=navi.-)<li>(.-</[Aa]>)", "<li>%1%2</li>") -- Fix the erroneous section sidebar
   return html
end
function pre_format_html_hook (url, html)
   if sstrfind (url, "blonnet.com") or sstrfind(url, "thehindubusinessline.com") then
      return blonnet_format_html_hook(html)
   elseif sstrfind (url, "bugs.debian.org") then
      return debian_bts_format_html_hook(html)
   elseif sstrfind (url, "mobile.twitter.com") then
      return mobile_twitter_format_hook(html)
   end
   return nil
end
And here's the old and new rendering:

24 August 2010

Lucas Nussbaum: RVM: seriously?

There s some hype in the Ruby community about RVM (Ruby Version Manager). It s a tool that allows to switch between Ruby versions on the same system (much like what the alternatives system provides for Java on Debian, except that RVM does it either system-wide or per-user). However, when you look at it, RVM looks quite scary. The recommended installation instruction is:
bash < <( curl http://rvm.beginrescueend.com/releases/rvm-install-head ).
That script doesn t use set -e, and actually does a git clone behind the scenes. Without first checking that git is installed. But if you don t have git installed, it s not a problem: there s another script later on the page that downloads and compiles it for you. After installing RVM itself, you need to install rubies (different ruby implementations). Use rvm install ree,1.9.2-head,jruby. That will automatically download and build the various versions in your homedir. It s interesting to note the the compilation messages were probably too scary, and are not displayed. But how does it handle the switch between different versions ? First, you need to add some magic to your .bashrc:
[[ -s "$HOME/.rvm/scripts/rvm" ]] && . "$HOME/.rvm/scripts/rvm"
or, for a system-wide install of rvm:
[[ -s "/usr/local/rvm/scripts/rvm" ]] && . "/usr/local/rvm/scripts/rvm"
And then, you can select your ruby implementation using rvm --default 1.9.2, for example. That works by redefining $PATH:
# echo $PATH
/usr/local/rvm/gems/ruby-1.9.2-p0/bin:/usr/local/rvm/gems/ruby-1.9.2-p0@global/bin:/usr/local/rvm/rubies/ruby-1.9.2-p0/bin:/usr/local/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11
Of course, that s quite fragile: if you are in one of the cases where bash doesn t read .bashrc, or if you happen to be using dash, you lose. Looking at the code of RVM itself, it is also very fragile: it s pure bash, without any error handling. During my test (in a chroot), I often got error messages about missing commands that were apparently ignored. The installer insists on using colors, and spews lots of error messages if executed without a controlling tty. There are also some interesting code snippets like perl -e 'sleep 0.5'. So, where do we go from here? Well, obviously, with my Debian hat, I m not going to advocate a solution that makes everything possible to avoid the distribution s packaging system, and I don t see RVM being packaged in Debian anytime soon.
In Debian, we already provide co-installability of Ruby 1.8 and 1.9.2, and users are free to choose which one to use on a per-script basis, by running them with version-suffixed ruby executables. The ruby executable itself still points to 1.8, as it s clearly too early to make 1.9.2 the default. Many Ruby libraries are provided for both 1.8 and 1.9.2, and we plan to discuss changes in the packaging system to be able to make it easier to provide packages for both versions. There s also recent work on packaging JRuby, but it s quite hard as it requires removing all the non-free or undistributable parts, and packaging them separately.

Guido G nther: Using git svn and git-buildpackage to build packages maintained in Subversion

Many team maintained packages in Debian are maintained in Subversion which makes a lot of sense if people agreed on a common workflow. If you want to use git to work on these packages you can still do so with a bit of configuration without changing anything for the other team members. Let's take system-config-printer as example. git-svn can be used to checkout the svn repository:
git svn clone svn+ssh://svn.debian.org/svn/pkg-gnome/ -T packages/unstable/system-config-printer -t tags/system-config-printer system-config-printer
Since the pkg-gnome team keeps all the packages in one huge SVN repository and splits the directories by release first and then by package the above command uses -T to point git-svn to system-config-printer's trunk and -t to point it to the tags directory. In simpler repositories where trunk, branches and tags are subdirectories of the same base directory using --stdlayout instead is enough. Since in SVN based packaging the upstream sources usually aren't kept in the repository git-buildpackage needs to know where to find them and how to combine things in the build-directory:
cd system-config-printer
cat <<EOF >.git/gbp.conf
[git-buildpackage]
# Build in a separate build-area
export-dir = ../build-area/
# Look for tarballs in ../tarballs
tarball-dir = ../tarballs/
# Overlay the upstream sources on top of it
overlay = True
# Don't use pristine-tar
pristine-tar = False
# Compression type of the tarball
compression = bzip2
Like with svn-buildpackage the orig-tarball is expected to be found in ../tarballs where you should put it now e.g. using the get-orig-source target of debian/rules (yes, git-buildpackage should support this implicitly). You can then work on the package as usual using all the nifty git commands like git-rebase, etc. and build the package (in this case using git-pbuilder):
git-buildpackage --git-pbuilder
or if you don't want to export the current branch's HEAD of your git repository but your working copy (which might contain uncommitted changes) to the build-area instead use:
git-buildpackage --git-pbuilder --git-export=WC
The built packages can then be found in ../build-area. Once the package is ready pushing the changes into the SVN remote repository is as simple as:
git svn dcommit
git svn tag -m"Tagging system-config-printer (1.2.3-0.2)" 1.2.3-0.2
The last command appropriately tags the new version. This saves a bit of typing. In order to fetch changes others have made use:
git svn rebase
Flattr this

Ian Wienand: Split debugging info -- symbols

In a previous post I mentioned split debugging info. One addendum to this is how symbols are handled. Symbols are separate to debugging info (i.e. the stuff about variable names, types, etc you get when -g is turned on), but necessary for a good debugging experience. You have a choice, however, of where you leave the symbol files. You can leave them in your shipping binary/library so that users who don't have the full debugging info available will still get a back-trace that at least has function names. The cost is slightly larger files for everyone, even if the symbols are never used. This appears to be what Redhat does with it's system libraries, for example. The other option is to keep the symbols in the .debug files along-side the debug info. This results in smaller libraries, but really requires you to have the debug info files available to have workable debugging. This appears to be what Debian does. So, how do you go about this? Well, it depends on what tools you're using. For binutils strip, there is some asynchronicity between the --strip-debug and --only-keep-debug options. --strip-debug will keep the symbol table in the binary, and --only-keep-debug will also keep the symbol table.
$ gcc -g -o main main.c
$ readelf --sections ./main   grep symtab
  [36] .symtab           SYMTAB          00000000 000f48 000490 10     37  53  4
$ cp main main.debug
$ strip --only-keep-debug main.debug 
$ readelf --sections ./main.debug   grep symtab
  [36] .symtab           SYMTAB          00000000 000b1c 000490 10     37  53  4
$ strip --strip-debug ./main
$ readelf --sections ./main.debug   grep symtab
  [36] .symtab           SYMTAB          00000000 000b1c 000490 10     37  53  4
Of course, you can then strip (with no arguments) the final binary to get rid of the symbol table; but other than manually pulling out the .symtab section with objcopy I'm not aware of any way to remove it from the debug info file. Constrast with elfutils; more commonly used on Redhat based system I think. eu-strip's --strip-debug does the same thing; leaves the symtab section in the binary. However, it also has a -f option, which puts any removed sections during the strip into a separate file. Therefore, you can create any combination you wish; eu-strip -f results in an empty binary with symbols and debug data in the .debug file, while eu-strip -g -f results in debug data only in the .debug file, and symbol data retained in the binary. The only thing to be careful about is using eu-strip -g -f and then further stripping the binary, and consequently destroying the symbol table, but retaining debug info. This can lead to some strange things in backtraces:
$ gcc -g -o main main.c
$ eu-strip -g -f main.debug main
$ strip ./main
$ gdb ./main
GNU gdb (GDB) 7.1-debian
...
(gdb) break foo
Breakpoint 1 at 0x8048397: file main.c, line 2.
(gdb) r
Starting program: /home/ianw/tmp/symtab/main
Breakpoint 1, foo (i=100) at main.c:2
      2return i + 100;
(gdb) back
#0  foo (i=100) at main.c:2
#1  0x080483b1 in main () at main.c:6
#2  0x423f1c76 in __libc_start_main (main=Could not find the frame base for "__libc_start_main".
) at libc-start.c:228
#3  0x08048301 in ?? () 
Note one difference between strip and eu-strip is that binutils strip will leave the .gnu_debuglink section in, while eu-strip will not:
$ gcc -g -o main main.c
$ eu-strip -g -f main.debug main
$ readelf --sections ./main  grep debuglink
  [29] .gnu_debuglink    PROGBITS        00000000 000bd8 000010 00      0   0  4
$ eu-strip main
$ readelf --sections ./main  grep debuglink
$ gcc -g -o main main.c
$ eu-strip -g -f main.debug main
$ strip main
$ readelf --sections ./main  grep debuglink
  [27] .gnu_debuglink    PROGBITS        00000000 0005d8 000010 00      0   0  4

23 August 2010

Patrick Schoenfeld: Why do I use facebook and co?

Recently I blogged a serious complain about Facebook marketing practices.
I must note, that, while writing this, this is still a standing fact.
And it somehow leads me to the question why I use facebook (and similar portals like Xing or Meinvz at all) at all.

I would say that I'm a person, who values security. Probably I need to, because my job involves security aspects, even if its only to protect data.
Additionally I value privacy. I would never go onto a toilet in a glass case, where everybody could watch me doing my business.
The idea to keep the door to my flat open all the time would trigger all alarm bells inside of me.
But on the other hand I have a blog on which I write about technical topics, and personal topics. Like the death of my grandma or how my neighbours disturb me. And I use Facebook. And MeinVZ.

I'm well aware of how this makes it easier to trace me. How this could be used against me, if I happen to search for a job. Aware of the everlasting memorization of the internet who will keep this.. eventually forever.
And I have friends who have a serious antipathy against this and probably would never use such a portal at all.
So why do I use it?

That I started using that portals were some kind of peer pressure. I'm a person who likes beeing "thereby" and I like beeing present. Many of my friends were already using those portals and so I followed an invitation to use it, too. But I did think about several points:

I came to the conclusion that there are pros and cons for and against using these services. And I found that I can live with the cons due to the convenience they bring to me. And that I could always stop using them.

However: That does not mean that I have to "Eat or die" what they serve.

Oh and on a side note: I found "Facebook, Br ste und die Currywurst" to be an interesting article about the topic. Its a recommended reading, although only for the people who understand German.

Patrick Schoenfeld: Gnomad = Gnome + Xmonad

Since I started using Linux I've used several window managers. I felt used to blackbox and fluxbox and eventually used enlightenment and some others in the past, but nowadays its been a while since I became the user of an Desktop Environment. I'm using GNOME because it provides what I need and I don't need to spend an hour to configure it before it suits my needs. After all I'm lazy.
With respect to the fact that I'm quiet satisfied with GNOME, there is one feature I were always missing.
Because I spend much time with tiling and arranging windows on my desktops I noticed that I could need
a tiling feature. Something which was already a feature back in Windows 3.11.
GNOME/metacity does not have this features and given that a wishlist bug about this is open since almost
7 years
its unlikley that this will ever change. There are separate tools, which I recently learned about that can assist me with this. For example the perl script 'wumwum'. But this seems to be the wrong solution to a real problem. Additional wumwum does not work properly with metacity and so I'd need to another WM anyway, which lead to the point where I started thinking about integrating a true tiling wm into GNOME... once again.

First, I looked into awesome, which is a window manager I used some time ago.
But documentation about configuring it is basically an API documentation, with no obvious entry point.
It seems to be the best to study the whole API just to set some simple settings (e.g. a padding for the GNOME panel and some always-floating applications). I even thought about learning LUA, because it seems like a language which is quick and easy to learn, but honestly if I need to study a programming language and a whole API documentation just to configure a window manager then IMHO there is something conceptionally wrong with that piece of software.
After all I came to Xmonad. This window manager is using Haskell and I fear I need to learn this language as well if I want to configure weird things. But the wanted scenario is well documented and documentation for more common configuration settings exists all over the place so that I don't
really feel inclined to learn more then needed.
Remember? I'm lazy.

Now I'm feeling quiet happy with this combination. It didn't cost me much time to get used to the most basic keyboard shortcuts or setting the whole thing up. GNOME and Xmonad work together like a dream team. I feel more productive now. As an additional plus I reinstalled the vimperator firefox plugin, because with my new desktop environment I more often use the keyboard for ordinary tasks like switching between apps or desks and I felt beeing able to quickly operate firefox with the keyboard, too, would be a plus. Well, it is.

22 August 2010

Gregor Herrmann: RC bugs 2010/27 - 2010/33

long time no RC bug posting. in july I was at the Munich BSP, & during DebCamp/DebConf I had the excuse of being one of the judges of the RCBC (which was the only reason for the freeze, I'm sure :)).

to keep up with the tradition of posting about my other RC bug activities, here comes a short list. as you can see most of the work was done in or around the pkg-perl group; luckily I often only have to upload packages prepared by ansgar, carnil, & others. I hope we as a project can keep up the momentum to fix the remaining RC bugs, & see a release in a not so far future.

19 August 2010

Steve Kemp: You are in a maze of twisty little passages, all alike

Debian package building scripts consist of several inter-related tools, which are documented well with manpages, but not in the code itself. Should you have some free time I urge you to take a peak at the source code of dpkg-buildpackage, debsign, and debuild. They work. They work well, but they're not shining examples of clear code are they? Given that they work, and that they are complex I'm not sure I'm interested in improving things, but I do have a personal desire to run something like:
debuild -sa --output=/tmp/results/
What would that do? Instead of placing the resulting package in the parent directory, which is what we have by default, it would place them in the named directory. However looking over the code I see that there are too many places where "../$file" is hardwired. e.g. Take a look at dpkg-buildpackage and you see:
my $chg = "../$pva.changes";
..
open OUT, '>', $chg or syserr(_g('write changes file'));
If you change that to the simple code below it works - but suddenly debsign fails to find the file:
my $dir = $ENV 'OUTPUT'    "..";
my $chg = $dir . "/" . $pva . ".changes";
Ugh. I guess I can come up with some hack to index files in the parent, run a build process, and move the only new files into place. But that's sleazy at best, and still doesn't solve the problem that we have tools which we rely upon which are .. unfriendly to changes.
I thought I could handle this via:
debuild \
 --post-dpkg-buildpackage-hook="move-changes-contents /tmp/output/ ../%p_%v_ dpkg-architecture -qDEB_HOST_ARCH .changes" \
 --no-tgz-check -sa -us -uc
(Here "move-changes-contents" would read the given .changes file and move both it and the contents listed in it to the specified directory.) Unfortunately %v corresponds to the package version - which isn't the same as the version in the .changes filename. (i.e. epochs are stripped out.)
ObQuote: I am the servant of the power behind the Nothing - The Neverending Story.

Patrick Schoenfeld: Getting the mp3 mess into control

I have some mp3 files in my collection. Some years ago, back in the times, when I was a Windows-User, I used "The Godfather", to keep the chaos under control. This software, although not Open Source, is really good at what it does. And it does almost everything to organize mp3s. From (mass-)tagging to auto-renaming to auto-sorting your mp3s. But this wouldn't be a blog entry by me if this became... a praise to a windows software so..

When I became a daily Linux User I searched for alternatives. With GUI or without a GUI, but I did not really find an application that suited my needs. As far as tagging and renaming was concerned it wasn't that hard. There are really excellent command line tools (lltag, id3, etc.) and rumours have been heard that there are also GUI tools.
But what I still did not find is a simple, yet flexible, tool to sort a huge collection of mp3s into a flexible structure on the filesystem like "The Godfather" was able to do since.. ehh.. a lot of years.

So the day before yesterday I finally decided to write one on my own and came up with an ~ 280 lines perl script (POD-documentation included) which does exactly what I want and is simple.
It does no tagging.
It does no renaming.
All it does is sorting mp3s into a given template-based directory hierarchy based on their ID3 tag.

At this point a warning is due. I'm sharing that script with you under the terms of the GPL. BUT it still needs testing and therefore you probably do not want to use it without its safety measures (e.g. dry-run or copy instead of move) to avoid data loss. And notice that although it has been written with portability in mind, it has only been tested on a Debian GNU/Linux system.

The script is hosted at github. Here (or raw to download it directly).

After I've talked to a co-worker about the script, he told me that arename would do what I want. So you probably don't want to use my script, because arename is probably more tested and much more sophisticated. But on the other hand I had a quick look at the arename manpage and its so utterly feature-loaded, that it cannot avoid a certain complexicity. My tool is simple.
And there is another advantage. If you want arename to handle a certain amount of mp3s you have to use your shell magic to find the files and pipe it to arename. My script finds all mp3s recursively from where you let it start (default is $PWD so be careful) and will happily move it into a hierarchy under a directory you ask it to. For the simple job of sorting mp3s its probably easier to use.

(Oh and if its worth nothing, it might still be of use as a simple programming example, how one could solve this problem in Perl)

Russell Coker: It s Election Time Again

Linux People and Voting Chris Samuel (a member of LUV who s known for his work on high performance computers and the vacation program) has described why he s voting for the Greens [1]. His main reasons are the Greens strong support of human rights and for science-based policy. Paul Dwerryhouse (a member of the Australian Linux community who s currently travelling around the world and who has made contributions to a range of Linux projects including SE Linux) has described his thoughts about the Filter Conroy campaign [2]. He gives a list of some of the high profile awful candidates who could possibly win a seat and therefore deserve a lower position in the preferences than Conroy. SAGE-AU and Voting for the Internet There has been some discussion by members of the System Administrators Guild of Australia (SAGE-AU) [3] about issues related to the election. As you would expect there was no consensus on which party was best. But there was a general agreement that the Greens are the only significant party to strongly support the NBN (National Broadband Network fiber to the home in cities and fast wireless in rural areas) and to also strongly oppose censoring the Internet. SAGE-AU has an official position opposing Internet filtering, and while the organisation hasn t taken a position on the NBN it seems that the majority of members are in favor of it (I am in a small minority that doesn t like the NBN). So it seems that political desires of the SAGE-AU members (and probably most people who care about the Internet in Australia) are best represented by the Greens. Note that SAGE-AU has no official policy on this, the above paragraph is based on discussions I ve had on mailing lists and in private mail with a number of SAGE-AU members. Also note that not all the SAGE-AU members who agree that the Greens advocate their positions on Internet issues plan to vote for them. The Green support for the NBN is based on the importance of the Internet to all aspects of modern life, the social justice benefit of providing decent net access for everyone (particularly people in rural areas) is very important to the Greens. I still oppose the NBN and believe that it would be better to just provide better ADSL in all suburbs, better net access (through whichever technology works best) in rural areas, and fiber to the central business areas. But the NBN isn t really that important to me, human rights and a science based policy are much more important and are the reasons why I ve been supporting and voting for the Greens. No Wasted Votes One thing to note is that the Australian electoral system is designed to avoid wasted votes. There are two ways of considering a vote to be wasted in Australia, one is if you live in an electorate where both the upper and lower house elections have an almost certain result such that no expected swing can change the outcome I doubt that this is possible for any region in Australia given the way the upper house elections work, although a large portion of the lower house seats have a result that is almost certain. The other way of having a wasted vote is to vote for someone who doesn t actually represent you. Lots of people mindlessly vote for a party that seems to represent them, either they identify with unions and vote Labor every time, they regard themselves as conservative and vote Liberal every time, or they live in a rural area and vote National every time. The Labor and Liberal parties don t differ much in policies and members in safe seats typically don t do anything for the people who elected them. If you generally support the policies of one of the major parties then it can be a good tactic to give your first preference to a minor party. For example if you tend towards Labor then vote Greens first and preference Labor over Liberal. The result will be that your vote will count towards Labor in the lower house and it sends a message to Labor and prevents them from being complacent. Before Australian elections there is always some propaganda going around about wasted votes, this is usually part of a deliberate campaign to try and prevent people from voting for smaller parties. Because the news has many mentions of wasted votes in US elections (which are watched closely in Australia) it seems that some Australians don t realise that there are significant and fundamental differences between the political systems in Australia and the US. Volunteering Last time I checked the Greens were still accepting volunteers to hand out how to vote cards, so if you want to do more for the Greens than just vote for them then this is one way to do it. If you want an uncensored Internet with freedom of speech and a lot of investment in infrastructure (as well as good support for all human rights) then you really want to help the Greens win more seats at the election on Saturday.

16 August 2010

Russell Coker: The Gift of Fear

I have just read The Gift of Fear and Other Survival Signals that Protect Us From Violence by Gavin de Becker. Like many self-help books it has a concept that can be described in a paragraph and explained in a few pages. The rest of the book shares anecdotes that help the reader understand the concept, but which are also interesting for people who get it from the first chapter. When I read the book I considered the majority of the content to be interesting stuff added to pad it out to book size because the concept seemed easy enough to get from the start, but from reading some of the reviews I get the impression that 375 pages of supporting material aren t enough to convince some people maybe this is something that you will either understand from the first few chapters or never understand at all. Gavin s writing is captivating, he has written a book about real violent crime in a style that is more readable than many detective novels, from the moment I finished the first chapter I spent all my spare time reading it. I was a little disappointed at the lack of detailed statistics, but when someone has done all the statistical analysis chooses to provide the results in the form of anecdotes rather than statistics I m prepared to tolerate that especially when the anecdotes are so interesting. I spent quite a bit of time reading the Wikipedia pages relating to some of the people and incidents that are mentioned in this book. The basic concepts of his book are to cease worrying about silly things like airline terrorists (passengers won t surrender now so that s not going to work again) and to instead take note of any real fear. For example if you are doing the same things you usually do but suddenly feel afraid then you should carefully consider what you might have subconsciously noticed that makes you feel afraid and what you can do about it. This isn t going to change my behavior much as I have mostly been doing what the book recommends for a long time. I think that everyone should read this book.

15 August 2010

Kai Hendry: Debian powered Web applications

Blars dining Francois Freeing the Cloud talk at Debconf10, showed members of the Debian community keen to help see opensource Web applications. The proposed open&free implementation of Gravatar is a great example, whereby a user uploads an image of him/herself which corresponds to a hash of their email address. In order to make this Web "cloud" service rock, super quick downloads of that image are needed. Debian knows about mirroring data to distribute data amongst free software agents. And we also now know about CDN, aka content delivery networks and GEO DNS, which point to your local mirror. So lets pool our resources and offer CDN Web space (100MB?) to free software Web application projects. This will make the opensource Web services competitive in all important page load times and perhaps make it easier for user contributed data to be exported and "open". I have Debian server space in Germany I'm happy to contribute to Francois's avatar project and other similar opensource Web apps. If you think this is a good idea, please get in touch.

13 August 2010

Russ Allbery: git-pbuilder 1.13

This supporting script for using cowbuilder with git-buildpackage came up at DebConf, which also meant more attention on the bug requesting that it be included in git-buildpackage. I've therefore done a few new releases over the last few days including additional features that the example script in git-buildpackage supported that my version of the script hadn't. Changes since 1.10 include adding support for update, create, and login options, which call cowbuilder with the corresponding flag instead of doing a build, and support for taking the distribution on which to act from the name of the script in case anyone wants to create symlinks to the script as supported by the git-buildpackage example script. I also switched the license to an MIT license instead of the same terms as Perl (which didn't make much sense, since it's not written in Perl). You can get the latest version of the script from my scripts page, and will also be included in the git-buildpackage package directly down the road. Thank you very much to Guido G nther for his discussion, willingness to include the script, and bug fixes.

Next.

Previous.