Search Results: "pm"

9 September 2009

Matthew Palmer: Water Tanks, Reliability, and Redundancy

Water supply, in the developed world, is one of those things you just pretty much take for granted. You turn on the tap, and clean, cool, refreshing water comes out. Similarly, when I go to a fire in my shiny fire truck, no matter what time of the day or night, I expect to able to hook up to a hydrant and have a strong, steady supply of water available. This level of reliability is something that we in the IT industry can typically only dream of. Practically 100% reliability over a period of decades, without constant maintenance and tweaking? Not a hope. To even get close to that, we need clusters of fully-redundant servers, fancy database replication techniques, and probably something totally out of left-field like Erlang's ability to reload code on the fly. But how do the water utilities do it? Clusters of fully-redundant high-capacity pumps, fancy pipe re-routing techniques? Or something totally out of left-field like a big water tank and gravity? Where I live, we've got the latter. Thinking about how it probably operates and is managed, I'm blown away by the sheer simplicity and robustness of the whole design, and how it can handle all manner of failures. First off, consider how few components have to be working in order for the water supply to continue for a while after some sort of catastrophic failure. We need: Note that this list of elements doesn't include any moving parts, or even guaranteed continuous water supply from a dam or other huge supply store. You can lose your feed pump(s), supply lines, or anything else that's on the supply side of the water tank for some period of time and nobody on the consumption side will know or care. Need more resilience to supply-side failure? Just build a bigger tank. This blows my mind, it really does. Need to do pump maintenance? No problem, just make sure that the tank is large enough to service demand over the period of the "outage", and go for it. Can't find a pump supplier to give you a restoration SLA of less than a week? Just make sure you've got a water tank that'll provide for a week's consumption. Basically, any supply-side reliability problem can be solved with "build a bigger tank", and while there's a limit to how big we can make tanks, I'll bet we know a lot more about building huge water tanks to withstand some freaky failure conditions than we do about building pumps that won't fail for 100 years. If you do need more capacity than a single tank can provide, just parallelise -- horizontal scaling of the water supply. Woohoo! Your big water tank also provides cost savings in your other equipment. If you had to pressurise a water supply using pumps, not only would you need your redundant array of very expensive pumps (Hmm, RAVEP-5 sounds like a Doctor Who villain), but those pumps would need to be able to provide your peak consumption flow (toilet breaks during the Super Bowl, probably). I'd imagine that could get mighty expensive, and without providing much benefit for 99% of the time. I see this capacity problem at work all the time. Customers who have the occasional massive traffic spikes need to massively over-provision their average utilisation to successfully service that 1% of the time that they're doing heavy traffic. Yes, I know, cloud computing, horizontal scaling, capacity on demand, yadda yadda yadda. It's not a panacea, and the number of apps that are designed to properly scale horizontally over a large range of traffic volumes is miniscule. My general point, though, is that it when you've got a variable load, it costs a whole lot more to provision systems to supply peak demand than to provision systems that can deal with average demand. A suitably large water tank means you can easily deploy a much smaller capacity feed pump. All you have to do is make sure that when demand exceeds supply, you've got enough water in your tank to cover the difference between the demand and supply over that period. When your peak capacity increases, and is starting to strain your infrastructure, you've got a choice, too: you can upgrade the pump or increase your storage capacity, whichever is cheaper / easier / quicker / provides better kickbacks / whatever. This is all well and good, you say, but computer systems aren't water supplies. There's a lot more moving parts, inputs, and outputs, and those all have to be handled. This is quite true. Water-powered computers are not in high demand. However, I think we could produce some much more reliable systems if we looked for ways we could simplify capacity and redundancy issues, water supply style, instead of layering more and more cruft into our solutions.
Ironically, just after I started writing this post (and this gives you an idea of how long it's been sitting in my drafts folder), I saw an article in the RISKS digest about a water supply problem in Santa Cruz, caused by a power outage killing the refill pump for an extended period and resulting in the storage tank running dry. Hey, I never said that it was a foolproof system -- especially when you've got failures of imagination that result in the system that tells people that there's no power relying on the power source it's monitoring being up in order to be able to tell people that there's no power...

Brett Parker: LVM weirdness

I'm having some issues with LVM, basically it appears that I can't assign over a certain amount of my pv, where that certain amount is somewhere around 1.5TB... output from all the relevant chunks are:
ensenada:~# lvdisplay
  --- Logical volume ---
  LV Name                /dev/main/root
  VG Name                main
  LV UUID                ZR33WS-glE2-bUn6-adCX-n50C-37eX-Fc7Pde
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                7.45 GB
  Current LE             1907
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:0
  --- Logical volume ---
  LV Name                /dev/main/voldemort
  VG Name                main
  LV UUID                DrT0co-THRw-6Zcy-Q3PT-OiQj-4KBz-xfV0Un
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                1.37 TB
  Current LE             358400
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:1
ensenada:~# vgdisplay
  --- Volume group ---
  VG Name               main
  System ID
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  56
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               5.45 TB
  PE Size               4.00 MB
  Total PE              1428053
  Alloc PE / Size       360307 / 1.37 TB
  Free  PE / Size       1067746 / 4.07 TB
  VG UUID               ddVSIr-W9BU-XOWJ-gdSo-HZ4m-vw3M-S33Q3s
ensenada:~# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda3
  VG Name               main
  PV Size               5.45 TB / not usable 1.54 MB
  Allocatable           yes
  PE Size (KByte)       4096
  Total PE              1428053
  Free PE               1067746
  Allocated PE          360307
  PV UUID               sFxRYg-Ua60-zpMT-DMqo-tpD5-d24W-qfW31p
ensenada:~# lvcreate -n test -L 74G main
  Logical volume "test" created
ensenada:~# pvs
  PV         VG   Fmt  Attr PSize PFree
  /dev/sda3  main lvm2 a-   5.45T 4.00T
ensenada:~# lvs
  LV        VG   Attr   LSize  Origin Snap%  Move Log Copy%  Convert
  root      main -wi-ao  7.45G
  test      main -wi-a- 74.00G
  voldemort main -wi-ao  1.37T
ensenada:~# lvremove /dev/main/test
Do you really want to remove active logical volume "test"? [y/n]: y
  Logical volume "test" successfully removed
ensenada:~# lvcreate -n test -L 75G main
  device-mapper: reload ioctl failed: Invalid argument
  Aborting. Failed to activate new LV to wipe the start of it.
  LV main/test in use: not deactivating
  Unable to deactivate failed new LV. Manual intervention required.
ensenada:~# lvremove /dev/main/test
Do you really want to remove active logical volume "test"? [y/n]: y
  Logical volume "test" successfully removed
ensenada:~# lvcreate -n test -L 74G main
  Logical volume "test" created
ensenada:~# lvremove /dev/main/test
Do you really want to remove active logical volume "test"? [y/n]: y
  Logical volume "test" successfully removed
ensenada:~#
Please, someone, put me out of my misery - this is driving me nuts! Extra info:
ensenada:~# uname -a
Linux ensenada 2.6.26-2-amd64 #1 SMP Fri Aug 14 07:12:04 UTC 2009 x86_64 GNU/Linux
ensenada:~# dpkg -l lvm2
Desired=Unknown/Install/Remove/Purge/Hold
  Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
 / Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
 / Name                  Version               Description
+++-=====================-=====================-==========================================================
ii  lvm2                  2.02.39-7             The Linux Logical Volume Manager
ensenada:~#
ensenada:~# cat /etc/debian_version
5.0.1
(Erm, just remembered that I haven't got comments enabled at the mo, so if you've got fixes, please mail me! iDunno@sommitrealweird.co.uk - thanks!)

7 September 2009

Joey Hess: look who's packaging

So, Debian is still trying to decide if it likes upstream developers including debian directories in their source. Unlike the rpm world, where including a spec file in the tarball is expected, there has historically been some dislike about this being done on the deb side, mostly because of tool limitations. In the meantime, an interesting thing is happening in the wider world. Developing "apps" for mobile phones is appealing to lots of developers. And in an inversion of the linux distro status quo, part of making such an "app" is packaging it. Typically, for linux-based phones, into a .deb package, or something very close to it. Since N900 apps can fairly easily be ported to/from desktop linux, I won't be suprised if debian directories start popping up in lots more upstream sources. Like it or not.

Russell Coker: Gnash and use of Free Software

There is currently a discussion on a private mailing list about whether some money from a community organisation should be used to assist the development of Gnash (the free software Flash player) [1]. The main reason for this is that there are apparently some schools that depend on flash web sites to such a degree that they won t consider using a free OS that lacks Flash support. It has been shown that there are a number of issues related to contributing financially to free projects, the people who advocate financial contributions in this case assure us that such problems have been addressed but it will remain controversial to some extent. One thing that is not controversial is the fact that testing and debugging is universally a good thing. So I advocate doing such testing as a way to contribute to Flash development and therefore free software use in education. The Debian-Edu project has a web page with a link to flash sites that can be used for testing [2]. So I plan to now install Gnash on all Linux desktop systems that I run and get bug reports to help development. I encourage others to do the same. Also there is the Ming library for developing Flash files which could apparently do with some help in the development process [3]. While a non-free format such as Flash is not ideal, it s certainly a lot better than Silverlight! Note that I don t have strong feelings about the issues of financial support for Gnash (which is why I didn t contribute to the private discussion in question). But I am convinced that more people using and testing Gnash is a good thing.

6 September 2009

Stefano Zacchiroli: rc bug of the day - again

8 RC bugs in 8 days I got back from vacation about 10 days ago. For the first time in recent years, I've spent 2 weeks of vacation mostly without email (excluding 2 "breaks" of 10 minutes each in an internet point, just to verify that my electronic world was not collapsing). The net result is that I feel very recharged and more productive both at work and for other tasks. Including Debian. Hence, during last week, I've tried an experiment. When during the day I feel bored about a given task, instead of reloading compulsively the page of an online newspaper, instead of checking (once more) my email, instead of checking (once more) my feeds, I try to fix an RC bug. For the first week, the result has been encouraging: one RC bug fixed per day (on the average, some day I haven't fixed any, some other 2 or 3): In some sense, this try to be kudos to the amazing initiative of Sesse, a few releases ago (or was it Lenny? my memory still sucks, no matter how recharged I feel). He has been fixing one RC bug a day for several months. I'm quite sure I won't be able to deliver the same result, but maybe this post will motivate other people, so that collectively we can fix a handful of RC bugs per day. Of course in the beginning it's easy, because there are a lot of "simple" bugs to fix and also because (as you can notice in the list above) "clusters" of similar RC bugs, that require similar fixes, can be identified. But the effort is useful nevertheless, since it cleans up RC bug list enabling more capable people to fix more resistent bugs. I'm gonna tag RC reports like this one with "rc" and "debian". If people follow up with similar initiatives we can rinse out of planet a feed of pretty good news for the Debian development community :-) In the interest of sharing best practices, here is my RC bug squashing workflow:
  1. go to http://bts.turmzimmer.net/details.php, choose "both" (i.e. squeeze + sid) as distribution and sort by "bugnr" (so that you can avoid most recent reports, which are not yet NMU material)
  2. choose an inspiring bug form the list which is at least 1 week old (better at least 2 or 3), study its log; for the moment, I tend to prefer neglected bug logs, where the maintainer has been unresponsive
  3. apt-get source package , sudo cowbuilder --build package_1.2-3.dsc , install the result and check for bug reproducibility
  4. fix the bug ! (yes, this is the easy step :-) ), taking care of not changing anything else
  5. dch --nmu
  6. dpkg-source -b package-1.2/ , sudo cowbuilder --build package_1.2-3.1.dsc , check if the bug is fixed and the package otherwise working
  7. lintian package_1.2-3.1.changes , check that there are no regressions in lintian errors wrt latest uploaded version
  8. interdiff -z package_1.2-3.diff.gz package_1.2-3.1.diff.gz , check that no unexpected changes have slipped in
  9. debsign package_1.2-3.1*changes , dput -e 2 package_1.2-3.1*changes (as allowed by devref 5.11.1)
  10. nmudiff --delay 2 , in the text: briefly describe in what the patch consists, and mention devref 5.11.1
I welcome suggestions on how to improve the above workflow in order to be both more effective and more polite with maintainers (FWIW, all the maintainer feedback I've received thus far for the above NMUs has been totally positive: you rock guys.).

Frans Pop: debtree 0.7.3 - Oh what tangled webs we weave

It's been quite some time (almost two years) since my previous "release" of debtree, but now version 0.7.3 is available. And it still generates very nice graphs :-) The changes are relatively minor: a few nice fixes for corner cases that were not handled correctly, and an update of the default lists of "skip" and "end" packages which help to limit the size of graphs for a fair number of packages I tried (including konqueror and openoffice.org).
Reason to revisit debtree was a recent nice mail from a debtree user, but also the current discussions about udev and the FHS. I'm on the side of "let's please keep /usr mountable separately". Mostly because I like a (small) encrypted root with a separate (large) unencrypted /usr'. I'm also increasingly unhappy with the default size of Debian's desktop installs, especially now that it looks as if Squeeze will see installation of Recommends by default by tasksel (and thus Debian Installer). For comparison, the size of a default Gnome desktop install for Etch was 1360MB; for Lenny it is 1830MB; for Squeeze it looks like it will be well over 3000MB! Remember that for Sarge we installed both Gnome and KDE from CD1 with both together taking 1390MB? Sure, some of that is real functionality, but a lot is also (IMO) redundant visual effects that only serve to slow the desktop down and junk needed to do stuff automagically. And a heck of a lot is duplicated functionality. One of the main reasons I switched to Linux was because it gave me back control over my systems, but with KDE4 and pervasive stuff like hal and all the various "kits" Linux is on a fast track that's giving priority to flashiness over real functionality and eroding that control. Here's a fairly default dependency graph for hal (click for full image). Looks reasonable, right? hal dependency graph But that's only because most major dependencies, such as dbus, policykit and pm-utils have been pruned. Here's a complete graph, with only libc6 omitted (full image is 1.5MB). Truly a tangled web. Scary. full hal dependency graph One can also look at it from the other side. Today I upgraded my sid chroot and found I suddenly needed to install libavahi-client3, libavahi-common3, libavahi-common-data and libdbus-1-3. Why? Reason turned out to be libcups2, so I checked if I really needed that. And here's why I do. libgtk2.0-0 dependency graph Most of these dependencies of libgtk2.0-0 I can understand, but isn't gtk supposed to be a graphical toolkit library? Couldn't printing support be implemented in some more specialized Gnome printing toolkit library? But I'm probably missing something. Anyway, now that I have a bit more perl experience through my recent work on debmirror, maybe I should finally port debtree from shell script to perl...
See the debtree home page for a full overview of how to read the graphs, but here's a quick intro. Purple arrows are Pre-Depends, blue are Depends and black are Recommends; green connections show Provides. The green packages are currently installed in my sid chroot, while the white ones are not. The diamonds show where the graph has been pruned: dependencies for these packages are not shown.

5 September 2009

Mark Brown: Chasing patches into Linux

One thing that often seems to cause problems for people who work over many different areas of the Linux kernel is the process of making sure that patches actually get reviewed and applied. Where the relevant subsystem is actively maintained it s not a problem but that s not always the case. Sometimes maintainers are busy or on holiday and miss things, sometimes there are other problems. In these cases the onus is on the patch submitter to spot the problem and make sure that something is done to ensure that the patch doesn t get forgotten. There s a few workflows for dealing with this. My preferred one is to track the appearance of my patches in Stephen Rothwell s linux-next tree, which tracks individual development trees destined for merge into Linus tree. I create a git branch based on the current state of this tree then apply the patches I m submitting on top of that. This lets me spot any potential merge conflicts that they ll create but the main function is to allow me to come back to the branch later and track which of the patches has shown up in one of the trees that Stephen tracks. To do this I rebase the branch onto the current state of linux-next:
git rebase --onto next/master old-master
where old-master is the last linux-next commit in the branch. This will flag up any merge issues that have come up due to changes in other trees and will also handle patches that are already present in one of the trees in -next by dropping my local version. The end result is a branch based on the new linux-next with all the patches that were not yet applied in it. I can see what still needs to be looked at by examining the log
git shortlog next/master..
and take any appropriate action, such as following up with the relevant maintainer or trying to find out what s going on with the subsytem if it looks like the subsystem maintainers are inactive. One possible problem with this approach is that a patch may be applied and then subsequently dropped this is rare but it can happen. I deal with that by also keeping a normal unrebased development branch whch has the changes in Linus tree merged into it periodically and incremental patches for any review updates that occur during the submission process. By looking at the diff between that and other trees I can see any changes that have got lost along the way.

4 September 2009

Marc 'Zugschlus' Haber: TCP and mobile IP

Steinar H. Gunderson, sesse, has written an interesting article about TCP performance. I didn t find your blog s comment function, so I am commenting with a trackback. (note: which didn t work either, The auto-discovered trackback URI does not match our target URI ) I frequently use mobile internet, using various of the German GSM/UMTS network operators, out of a moving train. As you have written, this frequently causes packet loss which is not only not caused by congestion, but sends the congestion avoidance algorithms on a false path. For example, when the train passes through the 3575 m long Distelrasentunnel between Frankfurt and Fulda, my network link is broken for like two minutes. Passing through other parts of Germany sometimes gives me a ping response of hundreds of thousands of microseconds by virtue of the rather huge send buffer the UMTS equipment has. In these circumstances, ssh sessions frequently take tens of minutes to notice that the network is back before the session is useable again. Frequently, it doesn t come up again before an hour has passed. And I have not found a way to work myself around this. Can you explain what s happening here, and do you have any ideas to solve the issue?

Michal Čihař: Gammu stable version 1.26.0

Gammu 1.26.0 is out! There is reduced memory usage, tons of bugfixes, better support for reading messages from Motorola phones. Full list of changes: You can download from usual place: http://cihar.com/gammu/. Debian users will find packages in unstable soon, SUSE, Fedora and Mandriva users can check repository in OpenSuse Build Service and Ubuntu in my PPA.

3 September 2009

Uwe Hermann: Open-Source software for LightScribe CDs and DVDs?

Dear Lazyweb, is there any Free Software utility for putting those nice logos/images on LightScribe CDs or DVDs? I'm considering buying such a CD/DVD burner, but only if there's open-source software to control the device. I guess burning CDs/DVDs should be no problem using wodim/cdrecord, but adding images as labels on the CD/DVD is likely not possible without special software. I know about the LightScribe Linux software (which is binary-only and not open-source, it seems) and LaCie LightScribe Labeler for Linux (which only creates labels but does not write them to the CD/DVD, and is not open-source either, I think). Anyone aware of a Free Software utility for this? Or at least a public specification/datasheet which could be used for creating one? I might even be willing to join development if datasheets are available.

2 September 2009

Michal Čihař: Problems with phpMyAdmin website

Our website currently shows a bit wrong information about file downloads. The single reason for this is that we take this data from sourceforge.net and all they exports are currently broken in some way. Per file RSS (which was only working source of information few days ago) seems to be disabled recently, RSS for all files just repeats same metadata for all files and the old releases RSS does not list recent releases. I did not want to blame them again, but it has been already three weeks from my last complains and bug reports and all that changed is that they broke (or removed?) the last working feed.

1 September 2009

Biella Coleman: Hello Ladies (aka learning from the lady geeks)

3692689082_dc8494bf13 Whenever I attend Debconf, the first day is always a bit disorienting due to the confluence of jet lag with the sea of male programmers swarming at the venue. I am not sure I should be admitting this but one of the first things that pops to mind is the very cheesy one hit wonder whose video is filled with many bikini clad men, Its raining men. So for a few hours, maybe a day, I sometimes feel out of place but this this first wave of discomfort usually gives way to comfort. This is not only because I connect with friends but because Debconf as well as many other hacker events and this sits in marked contrast to professional academic conferences are remarkably accepting of diversity and difference, at least that has been my personal experience. Most notably, people could care less about how you look, disabilities are consistently accommodated, and some things, like gender changing, which would raise serious eyebrows for instance in most academic conferences, is accepted with next to no gossip. I raise this because this is how I personally have experienced the world of Free Software: overwhelming male but overwhelming accepting of difference as well. So even while I think it is key to confront the problems of discrimination in this world, it is not a simple story, for again, it is an environment that is also accommodates difference and quirkiness to some notable degree. The question of gender in F/OSS is complicated one and here I am not going to ruminate on sources of gender bias except to say that I am sure there are some in operation (as there are in most domains) some of which are internal to Free Software and other s external to Free Software. But what I have been struck by, especially over the course of the summer, is the explosion of sites, blogs, and debates that have confronted gender in Free Software. It has been nothing short of astounding and a really positive turn. The most notable example is this AWEEEEEEEEEEESOME pair: a wiki and a blog that confront gender head on. These are notable because the wiki, for example, catalogs all sorts of controversial events, comments on them, leaving a very visible trace of debate, one that is necessary to change the gender make-up and dynamic within Free Software project. Along with these, the FSF is finally hosting a min-summit, which is great (less great is that participation seems invite only but perhaps they have some good reasons for doing so). And today I just learned of this diversity page coming out the Python project. These are perfect examples of the recursive public in action raised in Chris Kelty s work. And I have long been impressed with the dialogue that has followed from some controversial events in the world of Free Software, including those related to gender and this summer seems to be a watershed of sorts and I look forward to their developments over time. Indeed and this may be controversial as well, but I think my academic field of media, law, and culture has something to learn from these gender politics for there are some very real, though probably unintentional forms of discrimination that are not under that much active discussion. The most glaring problem is the underwhelming presence of female scholars during conferences (and as we know, conferences are exceedingly important for one s professional development and career). What I find most striking about this trend is the number of female scholars is significant. That is, when it comes to scholars and this seems different from the world of tech, there is a sizable community of women scholars and activists so when there is a 5% female participation , as for example, with this event, one has to wonder why is it raining men at these conferences? It also seems that while the debate exists, it is not as vibrant as with what is going on right now in F/OSS. Take for example, this recent mailing list post which unapologetically highlighted the lack of female presence in the up and coming Free Culture event at the Berkman Center. While the post generated a handful of thoughtful responses, including this supremely classic, biting, and quite clever response from Georgetown professor Julie Cohen, the debate did not linger on (however short, the posts and discussion were quite fruitful). What to do? Frankly, blame and finger pointing are pretty counterproductive, mostly because the discrimination is quite unintentional and I believe change can be brought about via more constructive paths. More important is we need to make the issue visible, identify some possible sources, and then create projects that can remedy the problem. Along with Elizabeth Stark, who has also been keen to note and change the gender problems, I am currently whipping together a wiki with a list of female scholars, leaders, and technologists as a resource for folks organizing conferences. We already have a base list (with a remarkably long list of folks) and will hopefully in the course of the next month throwing it up on the wiki. I don t think it will ever be as cool as the Geek Feminism Wiki but it will hopefully do some good!

31 August 2009

Romain Beauxis: Some quick notes on netboot install

Since I recently got a new laptop, I wanted to install Debian on it, of course :) However, I really didn't feel like buying a new CDR and burning yet another single-use boot CD and did not have a USB key available. Hence, I looked at installing through netboot using the previous one as the boot server and the available documentation. Well it turned out to be much more easy than what I was expecting initially ! The quick notes around this are:
- First you need to setup iptables for ip forwarding and masquerading, in order to be able to connect the new laptop to the internet:
  • ifconfig eth0 192.168.1.1
  • echo 1 > /proc/sys/net/ipv4/ip_forward
  • iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE

- Then you need to enable dhcp and bootp. This means simply:
  • apt-get install dhcp3-server
  • Copy configuration example in Section 4.5.2.1 of the documentation
  • Edit with your own values, in particular check the nameserver used by the laptop sharing the connection. Also the tftp server will be your laptop, so set its address to 192.168.1.1 (in our case). Eventually, you need the MAC address of the new laptop (check ifconfig on it for instance)

- Eventually, you have to enable the tftp server, which only means:
  • apt-get install tftpd-hpa
  • The server serves files located in /srv/tftp (check ps -Af). You have to download the netboot tarball and untar its content in this directory.
The only strange thing here is that for some reason, the ethernet card would loose its address when the new laptop was using the DHCP to obtain an address. I didn't check it, but setting it at a regular intervals solved the problem: while true; do sleep 0.1; ifconfig eth0 192.168.1.1; echo "done"; done. Yeah, that's hacky, but I wanted a quick solution :) After all of that, you boot the new laptop, select the network boot and... voila ! In conclusion, since it is not that difficult to setup, it would probably be nice to have some sort of standard script to enable PXE boot on a computer. Then we could claim that the new Debian release is committed to sustainable development and allows to install without wasting a CD :-D

30 August 2009

Filip Van Raemdonck: 363 days lead time

I would like to point out that my last article was in no way meant to be critical towards libvirt, rather on the contrary. I think it's an awesome technology the way it provides a common management platform for various virtualization solutions available on Linux and even other platforms.

For reference, expensive proprietary virtualization solutions don't necessarily fair better.
I've in fact recently ran into a problem in a Vmware ESX / Virtual Center environment which, while rather different in manifestation, in the end was similar in nature: diverging state change in the environment and in configuration.

The Vmware problem surfaced after kernel changes on Linux guests, where, after rebooting into the new kernel and running the vmware-config-
tools script to rebuild the modules, nightly backups with help of the vcbmount utility would fail.
The only cryptic error message was:

Creating a quiesced snapshot failed because the (user-supplied) custom pre-freeze script in the virtual machine exited with a non-zero return code.


It didn't help that there wasn't any "user-supplied pre-freeze script" to begin with.

Searching for this error message returned lots of hits, several going at least one year back. However, aside from a few references to a documented Solaris guest issue, which was not applicable, none would actually provide any insight into what the problem might actually be. Unt
il I found following post on the Vmware communities site, which pointed out that
changes to the uuid.location weren't propagated properly to the vmx definition.

For the time being, the best way to work around this (in Linux guests) is to add another reboot right after rebuilding the vmtools modules.
Since you should be rebuilding
But Vmware should provide a proper fix their stuff, really. I've got this strong feeling that the libvirt bug will be resolved in less time than a year...

5 June 2009

Stefano Zacchiroli: turbogears 2 packaging - take 3

TurboGears 2 packaging - (another) status update Brief status update on TG2 packaging. As expected, this week has been very busy and I'm writing the update offline on a train, but nevertheless I've good news to share: ... the last item above would theoretically mean that TG2 is already usable, but it is not the case due to the infamous #507909 which breaks both TG2 and TG1 in unstable (the latter was working in Lenny, and I had to resort to a Lenny chroot for a recent course I've taught about web development). In short, turbojson is in unstable, in spite of the lack of several of its dependencies, making impossible any import of it. ..., stay tuned, keep faith, bear with me, almost there, ... etc.

Martin Michlmayr: Corporate participation in open source communities

Someone recently asked me a few question about corporate participation in open source communities and I thought I'd share my thoughts on this topic here. Are there differences between an open source project done for a corporation and one done for personal reasons? There are many different ways to run an open source project, led by a corporation or by someone else. Some projects that are run by corporations have few outside contributors. This is often the case with projects that require copyright assignment (i.e. contributors have to assign their copyright to the corporation). These projects may not gain all the benefits of a true open source community, such as outside contributions or high levels of peer review. However, they may still be very successful projects and may have high levels of quality. Projects done by a corporation may have better planning and may have more resources than other projects. When a corporation, especially a large one, starts or becomes involves in a project it can also give credibility to the project and attract a lot of interest to the project. This means that projects done by corporations may have a bigger impact and might also be more visible in terms of publicity. How do corporations successfully utilize an open source community? Corporations can benefit from an open source community in many ways. For example, they can often find people who will review their code or make code contributions. If people become excited about what the corporation does, they might also spread the word and create viral marketing for the corporation. Establishing a community around one's project is often also a good way to identify people to hire since you already have experience working with them and know their capabilities. How do open source communities successfully utilize their corporate relationships? Corporations can make several unique contributions. For example, large corporations can use their name to attract attention to a project and give it credibility. Furthermore, corporations have some capabilities that personal contributors often don't have access. They may have special testing equipment (such as servers with thousands of CPUs or hard drives) or access to a testing lab where a professional usability test can be done. Finally, corporations can sponsor developer conferences, which are typically very effective means for the community to come together and work on activities together. It is important for projects to remember that corporations are not charities and that they will invest in an open source project for a reason. Therefore, they have to ensure that the corporation will get tangible outcomes from their involvement or sponsorship, otherwise they may not stay involved in the long run. What are the risks for a corporation when working with an open source community? One risk is that the code (or other form of contribution) is not accepted. However, this is a risk any contributor to a project faces. Before making any sort of contribution, it is therefore important to become familiar with the project and its culture. Every project has their own "do's" and "don'ts" that have to be followed. Another risk is that a corporation will invest in a community project that later on is abandoned by the community. However, in this case, the corporation could take the lead and continue to maintain the project. What are the risks for an open source community when working with a corporation? One potential risk is that the corporation will assert too much control over the project. It's important for projects to ensure that the community as a whole has influence over the direction of a project rather than one particular player. Are certain certifications needed in order for someone to participate in open source projects for a corporation? Certifications are not needed to get involved in or start a project. However, it is important to become familiar with the open source community and the project one wants to contribute to. A good first step is to read the book Producing Open Source Software by Karl Fogel which is available online. As a next step, the community in which someone wants to get involved in should be studied, for example by reading the mailing list archives. This will help to become familiar with the culture of a project as well as the mechanisms to contribute to the project. How do open source communities communicate and collaborate with corporations? In the best case, employees from corporations would interact in the project like any other contributor. That is, they should use the existing communication channels, such as mailing lists, IRC or developer gatherings. Many companies are good at working "with the community" but the ideal scenario is for a company to be part of the community and to work "in the community", just like other contributors. This is the most effective way for them to make changes to the code and project. Of course, not every corporation will get involved in a project directly. That's why it makes sense for projects to collaborate with corporations in other ways. For example, projects can talk directly to companies to get samples of their hardware in order to add support for them in their software. Projects can also work directly with corporations to find out how their project can better meet the needs of enterprise users.

2 June 2009

Matthew Palmer: Local HTML docs for Prototype

Having (temporarily) flung myself back into a bit of professional Rails app development, I'm doing a JS-heavy site (just to maximise the WTF factor). Since I'm Old Skool, I'm going with Prototype, as it's the javascript library I'm least unfamiliar with. The problem, though, is that I do about a fifth of my work on the train, where I don't have access to the Internet, and the Prototype people haven't seen fit to provide a downloadable version of the docs in HTML. There's a PDF[1], (which lacks any sort of hyperlinking (how very 1990s) and whose page numbering in the TOC doesn't match up to the PDF page numbers), and a CHM which has hyperlinks that don't work half the time. Apart from that, all the docs are online-only, and not very scraping-friendly (wget --mirror doesn't rewrite links correctly when you're also mangling the filenames for local-friendly access -- patch in the making if I can be arsed). So, long story short, I need a non-traditional way to get myself some local docs. Turns out it isn't as hard as it sounds, as evidenced by the below command list:
	git clone git clone git://github.com/sstephenson/prototype.git
	cd prototype
	git submodule init
	git submodule update
	sudo gem install treetop
	rake doc:build
This will peg your CPU for an inordinate amount of time (a smidge over 10 minutes for me; -j 3 would have made things a lot quicker), but you'll end up with a doc/index.html which has some useful stuff in it. It'll look nothing like what's on the website, but it's something, and should be enough to keep me going on those long, lonely train journeys.
1. which looks like someone should be expecting a C&D from O'Reilly for look and feel, unless it's a semi-sooper-sekrit O'Reilly production that they don't want to put their name to directly.

Christian Perrier: On localization

An old friend is ranting about localization. I first considered not really reaction to Joey's post, mostly because he has very true points...and particularly when he has about the same reaction than mine about a recent post in ou mailing lists. So, I will stick to react to one part of Joey's post...the one I consider really inaccurate. "everyone uses the software in English anyway" Really, Joey, really, that's completely untrue. Of course, I do live in a country that very well known (or at least that has the reputation) for being deeply ignorant in English language. But, really....*all* computers I see around me, both in my personal life *and in my professionnal life* are nearly completely localized. It's probably now well know that I spend most of my professionnal time....providing desktop systems (or managing the people who provide them) to users in ONERA, "the French Aerospace Lab". That means about 2000 users in a scientific environment where the use of English is much more developed than the average environment. Most of these users speak a fairly decent English and are perfectly able to understand that language in some way. About 80% of these systems are Windows-based and 20% are Linux-based. Probably more than 95% of them are localized in French. Even on Windows systems, we mostly integrate free software. And for *each* of them, the most important feature we need, besides software being free, is it to be localized in French. I bet our friends from Extremadura can say the same. The success of Linux-based free systems there also happened because the software is available in Spanish. And, of course, there are tons of similar stories and counter-examples (M nchen, Bhutan, Venezuela, etc.). So, really, please don't spread out that idea. Oh, yes, geeks *do* use their software in English (and, even here, I need to be convinced if I remember well what I saw at conferences in Germany or Spain, for instance). And, yes, it's very well widespread among geeks that using software in English is "better" or "easier". Which I indeed always fight and will fight until I die..:-). We certainly agree that these l10n needs put a high load on development and sometimes slow it down. and, for sure, we sometimes need to make a balance and I really thank you for giving me the credit to try doing that balance. Anyway, all this (including your rant) had to be said and it's interesting to have that small discussion (I just should not start writing down a blog entry at 23:00...:-)) See you soon, hopefully, Joey !

1 June 2009

Stephan Peijnik: sphinx-aware Enums in Python

As I promised to keep you updated on recent developments on update-manager I am writing this article. Just as a disclaimer: I am not going to write about any recent developments here, but would rather like to point at a piece of code I added to update-manager that could be useful in other applications too. Now, as the title suggests there are sphinx-aware Enums in update-manager. Enums are common constructs in other programming languages like C and allow simple creation of constants with, for example, ascending values (first constant has value 0, second has value 1 and so on). Python unfortunately does not include support for Enums itself, but I found it rather easy to write classes that emulate such a construct. Nothing is new about Enums in Python and there are probably quite a few different implementations out there, but I believe mine is different. The sphinx-aware part means that my implementation automagically updates the docstrings of the created instances and thus allows sphinx autodata method to include sensible information in generated API documentation. I could go on writing about and praising my method, but I believe a short example gives you a better idea how my implementation works and what I wanted to achieve with this. Have a look at this page, which is part of update-manager s new API documentation. You should see rather well-looking documentation of the UpdateManager.Backend.RELOAD_CACHE_STATUS NegativeEnum, the defined constants, their values and some additional information about each value now. Still, nothing too fancy, HTML documentation generated from docstrings. What makes this special is the code from which it was generated:
RELOAD_CACHE_STATUS = NegativeEnum(
  BEGIN = "Started reloading package cache",
  DONE = "Finished reloading package cache")
This not only gives us a RELOAD_CACHE_STATUS enum, along with the RELOAD_CACHE_STATUS.BEGIN and RELOAD_CACHE_STATUS.DONE, but also some documentation, included in RELOAD_CACHE_STATUS docstring, that can be used by sphinx. You can find the Enum code, which is rather short and should be quite easy to understand, here. I hope you find this code as useful as I do.

Gustavo Noronha Silva: My first patch to WebKitGTK+ committed!

Well, not really my first patch. But the first thing I tried to mess with when I first started looking at WebKitGTK+ was the WebKitNetworkRequest object, because I was fancing the idea of writing stuff such as HTTP transactions monitoring, and things like that. So I wrote a big patch which exposed the internal WebCore object (ResourceRequest) fully through our own object. That was back in early 2008. We have come a long way since, and through all these months I got a broader perception of what kind of APIs we need, and how WebCore works. We also decided on going soup-only, which had a huge impact on what the final patch actually looks like. The patch which finally got committed this week is, how can I put it, VERY different from what I had originally written. You can take a look at the long discussions about it in the bug report I used to track progress. I think I should point out that Marco Barisione and Christian Dywan were crucial in helping me get going with my contribution to WebKit at that time. What this change gives us is basically the fact that a WebKitNetworkRequest now carries more than just the URI for the request (it actually carries with it a reference to the SoupMessage that will be used later in the request processing, which we are planning to expose in the near future), meaning that when WebKit API gives you a request, and you use it to cause a new load (for, say, opening in a new tab), you still get all the headers that were supposed to go with the request, so you don t lose things such as, for instance, Referer. So, now, after more than 5 years, the bug that complained that Epiphany did not set Referer (and Galeon before that) for new tabs is finally closed. By the way, this problem has been fixed for Mozilla s browser back in 2002, but the embedding API is still buggy up to now. There is still hope, since there s an attached patch that fixes the issue to be reviewed, and landed. If anyone is reading, it might be a good oportunity to get this fixed in there as well, so that users of applications that use Gecko s embedding API can also benefit!

Next.

Previous.