Search Results: "ed"

5 October 2010

Stefano Zacchiroli: Debian squeezes Chromium back in

does it? You might have read elsewhere that Debian has squeezed out Chromium. As of today, Debian has also squeezed Chromium back in. Does that mean that Debian Developers change their mind too often? Surely not. Rather, it means that the noise around the "squeeze out" has been a bit exaggerated and it would be similarly exaggerated to make noise about the "squeeze in" now. To avoid being incoherent with the latter intention, let me move to my main point here. As I've commented yesterday:
The lesson to learn here is that, as long as Squeeze (or any other Debian suite, FWIW) is not released, there is still a margin of variability in the software it contains. If that margin weren't there, then the suite would have been released in the first place. The point of a freeze, in Debian, is to stop package acceptance in the suite by default and undergo a thorough scrutiny of what goes in and what gets out.
Such a scrutiny helps stabilize a release fixing RC bugs affecting it and improves Debian quality. You really need to thank the Release Team for such a thorough scrutiny work. In "exchange" for that, you might also want to take part in the Debian gift economy by help fixing RC bugs: do NMU if you are a DD, provide patches if you are a developer, help triaging bugs if you are a user, organize a Bug Squashing Party if you have organization skills, etc. Those are the activities that will boost the Squeeze release.

Gerfried Fuchs: New Backports Suite created

The Backports Team is pleased to announce the availability of a new suite on backports: lenny-backports-sloppy. Please read carefully before considering using or uploading to it what this entails. The Background You might want to ask: What's that? Let me explain it. During the etch release discussions popped up on the backports list with two clashing groups: The standing at that time was to accept packages that were in testing after the release, which wasn't etch anymore but lenny. The same discussion started again before the lenny release, and given that we are facing the upcoming squeeze release we started internally to discuss how to noise down these long and tedious discussions, because both groups of people had valid opinions that shouldn't get ignored. So this is where the idea for lenny-backports-sloppy comes from. The Change lenny-backports-sloppy will please the group that is happy to upgrade from lenny + lenny-backports to squeeze + squeeze-backports. lenny-backports is meant only for packages from squeeze, even after the release. Technically that means it will get locked down for uploads after the release of squeeze and require manual approval (for e.g. point release update versions, or security updates that happen during the squeeze release cycle), while lenny-backports-sloppy will accept packages from wheezy. Uploading to lenny-backports will have to get approved by the Debian Backports Team after the squeeze release, just like uploads to lenny are currently approved by the Release Team. While lenny-backports-sloppy is created already and working we ask you to not upload packages there without prior discussion with the Backports Team. This is meant to ensure that the Uploader is aware about the expectations that come along with that: The package should have a good chance to get included in the next Debian release aka wheezy, and that the Uploader is willing to look after the package in the upcoming squeeze-backports suite after the release of squeeze to ensure upgradability. In case of questions, feel free to ask either directly on the debian-backports@lists.debian.org mailinglist, or contact team@backports.debian.org privately. We are also pleased to announce that the first upload to lenny-backports-sloppy already happened. From now on you will be able to install Postgresql 9.0 (which is not targeted at squeeze) from lenny-backports-sloppy. How to use If you want to use lenny-backports-sloppy you will have to add both lenny-backports and lenny-backports-sloppy to your sources.list. Backports from lenny-backports-sloppy may depend on packages in lenny-backports.
  deb http://backports.debian.org/debian-backports lenny-backports   main
  deb http://backports.debian.org/debian-backports lenny-backports-sloppy main
About Backports You are running Debian stable, because you prefer the stable Debian tree. It runs great, there is just one problem: the software is a little bit outdated compared to other distributions. That is where backports come in. Backports are recompiled packages from testing (mostly) and unstable (in a few cases only, e.g. security updates), so they will run without new libraries (wherever it is possible) on a stable Debian distribution. It is recommended to pick out single backports which fit your needs, and not to use all available backports. Thanks for reading this far, and enjoy!
Rhonda in the name of the Backports Team

/debian permanent link Comments: 1 Flattr this

Alexander Reichle-Schmehl: 1000th commit!

Wow! Just noticed, that we had the 1000th in Debians Publicity repository! Congrats everyone :)

Alexander Wirt: A new player in the house: lenny-backports-sloppy

The Backports Team is pleased to announce the availability of a new
suite on backports: lenny-backports-sloppy. Please read carefully
before considering using or uploading to it what this entails.
The Background You might want to ask: What's that? Let me explain it. During the etch
release discussions popped up on the backports list with two clashing
groups: The standing at that time was to accept packages that were in testing
after the release, which wasn't etch anymore but lenny. The same discussion started again before the lenny release, and given that
we are facing the upcoming squeeze release we started internally to discuss
how to noise down these long and tedious discussions, because both groups
of people had valid opinions that shouldn't get ignored. So this is where
the idea for lenny-backports-sloppy comes from.
The Change lenny-backports-sloppy will please the group that is happy to upgrade from
lenny + lenny-backports to squeeze + squeeze-backports. lenny-backports is
meant only for packages from squeeze, even after the release. Technically
that means it will get locked down for uploads after the release of squeeze
and require manual approval (for e.g. point release update versions, or
security updates that happen during the squeeze release cycle), while
lenny-backports-sloppy will accept packages from wheezy. Uploading to
lenny-backport will have to get approved by the Debian Backports Team after
the squeeze release, just like uploads to lenny are currently approved by
the Release Team. While lenny-backports-sloppy is created already and working we ask you to
NOT upload packages there without prior discussion with the Backports Team.
This is meant to ensure that the Uploader is aware about the expectations
that come along with that: The package should have a good chance to get
included in the next Debian release aka wheezy, and that the Uploader is
willing to look after the package in the upcoming squeeze-backports suite
after the release of squeeze to ensure upgradeability. In case of questions, feel free to ask either directly on the
debian-backports@lists.debian.org mailinglist, or contact
team@backports.debian.org privately. We are also pleased to annonce that the first upload to
lenny-backports-sloppy already happened. From now on you will be able
to install Postgresql 9.0 (which is not targeted at squeeze) from
lenny-backports-sloppy.
How to use If you want to use lenny-backports-sloppy you will have to add both
lenny-backports and lenny-backports-sloppy to your sources.list. Backports
from lenny-backports-sloppy may depend on packages in lenny-backports.
deb http://backports.debian.org/debian-backports lenny-backports main
deb http://backports.debian.org/debian-backports lenny-backports-sloppy main

Erich Schubert: Google Apps experiences

Has anyone experience with using Google Apps?The server I'm currently using for my private email will be shut down soon, so I need to decide upon a future way of handling my personal email. I have low requirements, they key feature I need is a good spam filter; Google Apps can certainly provide this. Since I'm the single user, the free "standard" edition should be easily good enough.Obviously, since I don't have a reachable mail server myself anymore, I'd either have to rent a server just for my personal email, or use some hosted solution like Gmail, even if it is not open source.So, did you try it? Did you return to a different solution? Any points I'm missing? Email me: erich (a) debian orgUpdate: I have received a lot of feedback, much of which is very positive on Gmail/Google Apps for personal use.

Russell Coker: Open Source Learning

Richard Baraniuk gave an interesting TED talk about Open Source Learning [1]. His project named Connexions which is dedicated to the purpose of creating Creative Commons free textbooks is a leader in this space [2]. He spoke about Catherine Schmidt-Jones who wrote 197 modules and 12 courses on music [3], that s a very significant amount of work! He also mentioned the translation of the work into other languages. I wonder how well the changes get merged back across the language divide. We have ongoing disputes in the free software community about whether various organisations do enough work to send patches back upstream, this seems likely to be more of a problem in situations where most of the upstream authors can t even understand the language in which the changes are written and when the changes involve something a lot more subtle than an change to an algorithm. This would be particularly difficult for Chinese and Japanese as those languages seem to lack quality automatic translation. He mentioned Teachers Without Borders [4] in passing. Obviously an organisation that wants to bring education to some of the poorer parts of the world can t have a curriculum that involves $250 of text books per year for a high school student (which was about what my parents paid when I was in year 12) or $500 of text books per year for a university student (which might be a low estimate for some courses as a single text can cost more than $120). Free content and on-demand printing (or viewing PDF files on a OLPC system) can dramatically lower the cost of education. It s widely believed that free content without the ability to remix is cultural imperialism. This is apparently one of the reasons that the connexions project is based on the Creative Commons Attribution license [5]. So anyone anywhere can translate it, make a derivative work, or collate parts of it with other work. I expect that another factor is the great lack of success of all the various schemes that involve people contributing content for a share of the profits, the profits just don t match the amount of work involved. Philanthropy and reputation seem to be the only suitable motivating factors for contributing to such projects. One of the stated benefits of the project is to have computer based content with live examples of equations. Sometimes it is possible to just look at an equation and know what it means, but often more explanation is required. The ability to click on an equation, plug in different values and have them automatically calculated and possibly graphed if appropriate can make things a lot easier. Even if the result is merely what would be provided by reading a text book and spending a few minutes with a scientific calculator the result should be a lot better in terms of learning as the time required to operate a calculator can break the student s concentration. Even better it s possible to have dynamic explanations tailored to the user s demand. To try this out I searched on Ohm s Law (something that seems to be unknown by many people on the Internet who claim to understand electricity). I was directed to an off-site page which used Flash to display a tutorial on Ohm s Law, the tutorial was quite good but it does seem to depart from the free content mission of the project to direct people off-site to proprietary content which uses a proprietary delivery system. I think that the Connexions project could do without links to sites such as college-cram.com. One of the most important features of the project is peer review lenses . The High Performance Computing Lens [6] has some good content and will be of interest to many people in the free software community but again it requires Flash. One final nit is the search engine which is slow and not very useful. A search for engine returned lots of hits about engineering which isn t useful if you want to learn about how engines work. But generally this is a great project, it seems to be doing a lot of good and it s got enough content to encourage other people and organisations to get involved. It would be good to get some text books about free software on there!

4 October 2010

Raphaël Hertzog: Can Debian offer a Constantly Usable Testing distribution?

Debian s testing distribution is where Debian developers prepare the next stable distribution. While this is still its main purpose, many users have adopted this version of Debian because it offers them a good trade-off between stability and freshness. But there are downsides to using this distribution and the Constantly Usable Testing (CUT) project aims to resolve those. This article will present the project and the challenges involved to make it happen. About Debian unstable & testing Debian unstable is the distribution where developers upload new versions of their packages. It happens frequently that some packages are not installable due to changes in other packages or due to transitions not yet completed. Debian testing, on the contrary, is managed by a tool that ensures the consistency of the whole distribution: it picks updates from unstable only if the package has been enough tested (10 days usually), if it s free of new release-critical bugs, if it s available on all supported architectures, and if it doesn t break any other package already present in testing. The Release Team (RT) controls this tool and provide hints to help it find a set of packages that can flow from unstable to testing. Those rules also ensure that the packages that flow into testing are reasonably free of show-stopper bugs (like a system that doesn t boot, or X that doesn t work at all). This makes it very attractive to users who like to regularly get new upstream versions of their software without dealing with the biggest problems associated to them. This is all very attractive, yet several Debian developers advise people to not use testing. Why is that? Known problems with testing Disappearing software The release team use this distribution to prepare the next stable release and from time to time they remove packages from it. Either because it s needed to ensure that other packages can migrate from unstable to testing, or because they have long-standing release-critical bugs without progress towards a resolution. It also happens that they remove packages on request of the maintainers because they believe that the current version of the software cannot be supported (security-wise) for 2 years or more. The security team also regularly issues such requests. Long delays for security and important fixes Despite the 10-day delay in unstable, there are always some annoying bugs (and security bugs are no exceptions) that are only discovered when the package already has migrated to testing. The maintainer might be quick to upload a fixed package in unstable, and might even raise the urgency to allow the package to migrate sooner, but if the packages gets entangled in a large ongoing transition, it will not migrate before the transition is completed. Sometimes it can take weeks for that to happen. This delay can be avoided by doing direct uploads to testing (through testing-proposed-updates) but this is almost never used, except during a freeze, where targeted bugfixes are the norm. Not always installable With testing evolving daily, updates sometimes break the last installation images available (in particular netboot images that get everything from the network). The debian-installer (d-i) packages are usually quickly fixed but they don t move to testing automatically because the new combination of d-i packages has not necessarily been validated yet. Colin Watson sums up the problem:
Getting new installer code into testing takes too long, and problems remain unfixed in testing for too long. [...] The problem with d-i development at the moment is more that we re very slow at producing new d-i *releases*. [...] Your choices right now are to work with stable (too old), testing (would be nice except for the way sometimes it breaks and then it tends to take a week to fix anything), unstable (breaks all the time).
CUT s history CUT finds its root in an old proposal by Joey Hess: it introduces the idea that the stable release is not Debian s sole product and that testing could become with some work a suitable choice for end-users. Nobody took on that work and there was no visible progress in the last 3 years. But recently Joey brought up CUT again on the debian-devel mailing list and Stefano Zacchiroli (the Debian project leader) challenged him to setup a BoF on CUT for Debconf10. It turned out to be one of the most heavily attended BoF (video recording is here), there is clearly a lot of interest in the topic. There s now a dedicated wiki and an Alioth project with a mailing list. The rest of this article tries to summarize the various options discussed and how they re supposed to address the problems identified. The ideas behind CUT Among all the ideas, there are two main approaches that have been discussed. The first is to regularly snapshot testing at points where it is known to work reasonably well (those snapshots would be named cut ). The second is to build an improved testing distribution tailored to the needs of users who want a working distribution with daily updates, its name would be rolling . Regular snapshots of testing There s general agreement that regular snapshots of testing are required: it s the only way to ensure that the generated installation media will continue to work until the next snapshot. If tests of the snapshot do not reveal any major problem, then it becomes the latest cut . For clarity, the official codename would be date based: e.g. cut-2010-09 would be the cut taken during September 2010. While the frequency has not been fixed yet, the goal is clearly to be on the aggressive side: at the very least every 6 months, but every month has been suggested as well. In order to reach a decision, many aspects have to be balanced. One of them (and possibly the most important) is the security support. Given that the security team is already overworked, it s difficult to put more work on their shoulders by declaring that cuts will be supported like any stable release. No official security support sounds bad but it s not necessarily so problematic as one might imagine. Testing s security record is generally better than stable s one (see the security tracker) because fixes flow in naturally with new upstream versions. Stable still get fixes for very important security issues sooner than testing, but on the whole there are less known security-related problems in testing than in stable. Since it s only a question of time until the fixed version comes naturally from upstream, more frequent cut releases means that users get security fixes sooner. But Stefan Fritsch, who used to be involved in the Debian testing security team, has also experienced the downside for anyone who tries to contribute security updates:
The updates to testing-security usually stay useful only for a few weeks, until a fixed version migrates from unstable. In stable, the updates stay around for a few years, which gives a higher motivation to spend time on preparing them.
So if it s difficult to form a dedicated security team, the work of providing security updates comes back to the package maintainer. They are usually quite quick to upload fixed packages in unstable but tend to not monitor whether the packages migrate to testing. They can t be blamed because testing was created to prepare the next stable release and there is thus no urgency to get the fix in as long as it makes it before the release. CUT can help in this regard precisely because it changes this assumption: there will be users of the testing packages and they deserve to get security fixes much like the stable users. Another aspect to consider when picking a release frequency is the amount of associated work that comes with any official release: testing upgrades from the previous version, writing release notes and preparing installation images. It seems difficult to do this every month. With this frequency it s also impossible to have a new major kernel release for each cut (since they tend to come out only every 2 to 3 months) and the new hardware support that it brings is something worthwhile to many users. In summary, regular snapshots address the not always installable problem and changes the perception of maintainers towards testing, so that hopefully they care more of security updates in that distribution (and in cuts). But they do not solve the problem of disappearing packages. Something else is needed to fix that problem. A new rolling distribution? Lucas Nussbaum pointed out that regular snapshots of Debian is not really a new concept:
How would this differentiate from other distributions doing 6-month release cycles, and in particular Ubuntu, which can already be seen as Debian snapshots (+ added value)?
In Lucas s eyes, CUT becomes interesting if it can provide a rolling distribution (like testing) with a constant flux of new upstream releases . For him, that would be something quite unique in the Free Software world . The snapshots would be used as starting point for the initial installation, but the installed system would point to the rolling distribution and users would then upgrade as often as they want. In this scenario, security support for the snapshots is not so important, what matters is the state of the rolling distribution. If testing were used as the rolling distribution, the problem of disappearing packages would not be fixed. That s why there have been discussions of introducing a new distribution named rolling that would work like testing but with adapted rules, and the cuts would then be snapshots of rolling instead of testing. The basic proposal is to make a copy of testing and to re-add the packages which have been removed because they are not suited for a long term release while they are perfectly acceptable for a constantly updated release (the most recent example being Chromium). Then it s possible to go one step further: during freeze, testing is no longer automatically updated which makes it inappropriate to feed the rolling distribution. That s why rolling would be reconfigured to grab updates from unstable (but using the same rules than testing). Given the frequent releases, it s likely that only a subset of architectures would be officially supported. This is not a real problem because the users who want bleeding edge software tends to be desktop users on mainly i386/amd64 (and maybe armel for tablets and similar mobile products). This choice if made opens up the door to even more possibilities: if rolling is configured exactly like testing but with only a subset of the architectures, it s likely that some packages migrate to rolling before testing when non-mainstream architectures are lagging in terms of auto-building (or have toolchain problems). While being ahead of testing can be positive for the users, it s also problematic on several levels. First, managing rolling becomes much more complicated because the transition management work done by the release team can t be reused as-is. Then it introduces competition between both distributions which can make it more difficult to get a stable release out, for example if maintainers stop caring of the migration to testing once the migration to rolling has been completed. The rolling distribution is certainly a good idea but the rules governing it must be designed to avoid any conflict with the process of releasing a stable distribution. Lastly, the mere existence of rolling would finally fix the marketing problem plaguing testing: the name rolling does not suggest that the software is not yet ready for prime time. Conclusion Whether CUT will be implemented remains to be seen, but it s off for a good start: ftpmaster Joerg Jaspert said that the new archive server can cope with a new distribution, and there s now a proposal shaping up. The project might start quickly: there is already an implementation plan for the snapshot side of the project. The rolling distribution can always be introduced later, once it is ready. Both approaches can complement each other and provide something useful to different kind of users. The global proposal is certainly appealing: it would address the concerns of obsolescence of Debian s stable release by making intermediary releases. Anyone needing something more recent for hardware support can start by installing a cut and follow the subsequent releases until the next stable version. And users who always want the latest version of every software could use rolling after having installed a cut. From a user point of view, there are similarities with the mix of usual and long term releases of Ubuntu. But from the development side, the process followed would be quite different, and the constraints imposed by having a constantly usable distribution are stronger: any wide-scale change must be designed in a way that it can happen progressively in a transparent manner for the user.
This article was first published in Linux Weekly News. If you want to see more articles like this, join Flattr and click on the flattr button below every article that you like.
Flattr this Share/Bookmark 21 comments Support my work

Jonny Lamb: Automatic Connections

My network blocks SIP on 3G, so when I start using its 3G service after leaving a wireless network my N900 goes absolutely crazy about not being to connect. Although this is a specific annoyance I have, it's a general problem with loads of similar cases, such as "I only want my work jabber/skype/MSN/etc. account enabled when I'm at work". So, I wrote a small daemon and a fun UI to make this easy. It's actually really simple: once you configure which connections accounts should listen for, it uses libconic to listen for connect/disconnect events and enables and disables accounts as appropriate. You should also be able to override its actions temporarily without it clobbering your setting. Click for full-sized image
A sneaky peek at the UI. See if you can guess where I've been visiting recently. I think the UI is pretty easy to use, and should cover most use cases. I just uploaded it to extras-devel. It's called Automatic Connections.

Iustin Pop: ICFP 2010 short report

I've recently returned from ICFP 2010 (International conference on functional programming). This was the second time I went there, and like last year, it was a very interesting conference. This year, I've even managed to do one keysigning with a fellow Debian Developer (hello Sylvain!) While there were many talks, I only mention a few key ones, that I have found very cool. Note: I'm a layman in this field. The text below is my own interpretation of the talks and contains many errors and wrong facts corrections are welcome! Workshop on ML I wasn't present at this workshop, but other people pointed me to this very cool talk: Mirage project The abstract, to be found at http://www.cs.rit.edu/~mtf/ml2010/program.html#Program (search for Mirage), says:
Our framework, dubbed Mirage, extends the Objective Caml language with I/O extensions and a custom runtime to emit binaries that execute as a paravirtual operating system directly under Xen. The custom run-time is significantly simpler than a general-purpose operating system, with a static single-process 64-bit address space, and "zero-copy" I/O that works directly with the OCaml garbage collector. As a result, Mirage applications exhibit significant performance speedups for I/O and memory handling versus the same code running under Linux/Xen.
It's extremely interesting, IMHO. You write your code as a more-or-less normal OCaml program, but you compile it with this special tool-chain, and you boot it directly on ton of Xen, as a domU, without the intermediate OS. The project Git repo is at http://github.com/avsm/mirage, and that links a bit older presentation (http://anil.recoil.org/papers/2010-hotcloud-lamp.pdf) which IMHO is a required read for anyone interested in cloud-related stuff. ICFP proper ICFP, the main conference, is as usual, full of math, designed to scare people away. Err, I mean, designed to tell about new developments in this field. The papers are gathered in this reddit post: http://www.reddit.com/r/haskell/comments/dji4v/papers_from_icfp_2010/. XenServer The paper http://anil.recoil.org/papers/2010-icfp-xen.pdf describes the XenServer project, and more specifically the management interface, which is written entirely in OCaml, by a team of ~10 people. They also have a bit of bin-packing code (http://xenbits.xen.org/xapi/xen-api.hg?file/95b9e4f1b9dd/ocaml/xapi/binpack.ml), but much simpler than the ganeti-htools project on which I'm working (no balancing or similar). On the other hand, it is interesting to see that they committed themselves to write the entire management suite in OCaml, as opposed to our Python plus just a few bits of Haskell. I envy them, very much so ;) The fun tidbit was how stable they found the OCaml tool-chain. In three years, they had just one bug related to this (the tool-chain), a segfault due to wrong formatting code in the handling path of a seldom-triggered Python exception, which led to a %s to be translated as is to C code, which meant printf got the wrong specifiers. So in an OCaml project, a serious bug was due to lack of type checking in string formatting in a Python component :) Overall I have found interesting parallels between their experience and my own with Haskell; from is this language/tool-chain stable enough for use to concerns about using FP for a significant project (but of course, orders of magnitude difference between their case and mine). Super-compilation There were two papers on this topic; one at ICFP proper and one at the Haskell symposium. If I understood things right, super-compilation is the opposite of lazy evaluation. But let's start a bit back. First, programs written in a functional languages contain many pure function. Pure functions, being side-effect-free, exhibit many interesting properties, and one of them is that they permit decoupling of the computation of a value from the declaration of such computation. One technique derived from this is lazy computations. Lazy computation means that even though you write y = sin(x) + 1, the actual call to sin and the increment is not actually done until you need the actual value of y (not just for storing it in y, that is). Super-compilation is the opposite of this. While laziness postpones doing a computation to as late as possible, in the hope it won't be needed at all, super-compilation runs the actual computation way ahead of time, even before the program starts, that is at compile time. Since programs use input data at runtime, which is not known at compile time, the trick is to pre-compute as much as possible at this stage, given incomplete information; and the papers showed that it's possible to pre-compute much more than you'd usually believe. Optimising compilers already do some form of CAF (constant applicative form) optimisation, but the techniques presented in both papers are going beyond simple CAFs. The details are in the papers, and are complicated, but one simple example is converting this:
ones = map inc [1..]
(note that is an infinite list, and you don't want to fully compute that at any time, neither at compile nor at runtime), into:
ones = [2..]
The details on how this is done depend on the exact super-compilation implementation, and not all implementations can do the same optimisations, but in the presentations at this ICFP the idea was to split things into static/non-static parts, and build on that, or to do partial evaluation, etc. The result is that super-compilation shows significant increases in speed (in some synthetic benchmarks up to 80%), with a few benchmarks not being sped up at all, but most are. The downside is an explosion in code size, as many more things are pre-computed. Still, as a research topic, this is very interesting. My own paper Surprisingly, it seems I survived giving my talk (just an experience report, not actually fancy stuff like the rest). Either the people actually found it interesting (less likely), or they were very polite (more probably). Anyway, the paper is on the papers page (shameless self-promotion) (if you care). Haskell symposium This is co-located with ICFP, and is more practical than ICFP (which is, as I said, way too much math). The papers are again available on reddit: http://www.reddit.com/r/haskell/comments/dkzn4/papers_from_the_haskell_symposium_2010/. Orc: Concurrent Orchestration in Haskell How to easily write code for synchronising concurrent work? As an example, how complicated is to solve this problem:
[from the Orc literature] we might wish to contact two airlines simultaneously seeking price quotes. If either quote comes back below a threshold price, say $300, then let's buy a ticket immediately. On the other hand if both quotes exceed the threshold, then buy the cheapest ticket. Additionally, buy a ticket if the other airline does not give a timely quote, or notify the user if neither airline provides a timely quote.
Now, in my experience, this would require manual management and synchronisation of threads, and for example in Python the early abort of querying the price from airline B if airline A returned a good price would be non-trivial; in addition, adding two extra conditions would be again not a simple composition of rules. In Orc however, it takes all of 10 lines of code to do this, safely. I won't copy the code here, see the paper for details, or the API docs. And the best part is, this is just a cabal install orc away! I'll be very interested to see how I can use this in practice, as it is seems indeed a very powerful library. Nikola: Embedding Compiled GPU Functions in Haskell Interesting project. Using almost 100% Haskell code, and it marshalls the data to/from the GPU; in the example given, the performance is as good as hand-written CUDA code of course, this doesn't happen in all cases. It it's interesting how they manage to compile the GPU-related code to actual object code at Haskell compile time, which makes it transparent for the user. Given the prevalence of high-speed GPUs, this could be a nice way to utilise them without learning yet another language (or two, since NVidia and ATI have different stacks). They've also developed some interesting techniques (e.g. manual control of sharing, to prevent inlining) as part of their project. Scalable Event Handling for GHC This is a paper co-authored by Johan Tibell of Google Switzerland and Bryan O'Sullivan of Real World Haskell fame. Very interesting serving 20K QPS from a Haskell process while having 16K+ idle clients connected:
[ ] the performance of the epoll back end does not begin to fall off significantly until we have 50,000 idle connections open.
The talk started with the idea that we're on the path to serving 10 clients from a single machine, and how to improve Haskell's I/O manager to achieve (almost) 10 concurrent clients. The slides (which I don't have a link for) were a bit more cool on numbers than the paper. Oh, and this will come with GHC 7.0, for free :) An LLVM Backend For GHC The eternal fight to get better than the C backend in other words, to not have to translate Haskell to C It is interesting to see how many languages target LLVM nowadays, as opposed to plain old C via gcc. A fun trivia item was that, according to the author, there are many features in LLVM that are not actually used by 90% of the languages targeting it (e.g. the garbage collector) (if I heard correctly in the talk). The results are interesting, although not too much overall speedup is gained; this seems to be due to the hard-to-optimise Cmm intermediate language inside GHC. In any case, a few benchmarks get more than 10% speedup, even in the current form. This will also come in GHC 7.0, although not enabled by default; probably needs to mature some more Conclusion I've enjoyed this conference a lot, and I've learned a lot from it. Given that many of the talks are about research topics, it's harder to apply them in day-to-day work, but I think the pay-offs will be long term rather than immediate.

Christian Perrier: [running] Latest bubulle running news

Since I ran Bois d'Arcy hal-marathon back on Sept 12th, breaking my record down to 1h37, I continued working on resistance, in preparation for the Sein-Eure marathon on Oct 17th, two weeks from now. I'm still working empirically. No program, no fractioned sessions, work on speed, constant cardio training, etc. Just running...a lot...most often at marathon speed, and alternating trail and road running. Usually, I'm out 4 days per week: both Saturdays (short 10km run, or so), Sundays (long run, up to 20) and two week days, during lunch (skipping lunch and going out running). I'm now working one day per week in another facility of Onera, in Palaiseau. This is 20km away from center Paris and has woods close to it. So, I can train in trails rather than stupid roads...:-) I ran a small local race on Sept 26th: "la vir e des Deux Bois", in Villebon sur Yvette, a 18km trail with 330meters positive slope. I finished 27th out of 69 in 1h31, despite the high level of runners....most being trained trail runners who didn't want to run in the crowd of Paris-Versailles. Finally, this week-end (Oct 3rd), I (alone) ran a long trail to the Colline d' lancourt, one of my favourite spots in the neighbourhoods, marvelous when the sun is shining, with a 360 paronamic view up to 30 km in distance (yes, one can see the Eiffel Tower). This artificial "moutain" is the highest point in Paris region. That trail involves many funny...and muddy...single tracks in the forest. A great piece of cake. It also involves 300 meters positive slope, so that's hard training..:-) I now feel ready for the third marathon of this year! Let's rest for two weeks, now.

Dirk Eddelbuettel: More BLAS, BLASter, BLAStest: Updates on gcbd

Following up on my initial post announcing gcbd, here is a brief note on a new version. The initial post announced version 0.2.2 which was the first CRAN version of gcbd. I updated to 0.2.3 when I made the aforementioned first blog post about gcbd with the lattice plot of the BLAS and GPU benchmark results across six different implementations (from reference BLAS to two Atlas versions, Goto, MKL and a GPU-based one). There is now a new version 0.2.4 of gcbd on CRAN. I revised the paper ever so slightly based on some more feedback, and focussed the results sections by concentrating on just the log-axes lattice blot and the corresponding lattice plot of raw results---where the y-axis is capped at 30 seconds: GPU/CPU Benchmark Results in levels This chart--in levels rather than using logarithmic axes is done here--nicely illustrates just how large the performance difference can be for for matrix multiplication and LU decomposition. QR and SVD are closer but accelerated BLAS libraries still win. GPUs can be compelling for some tasks and large sizes. More discussion is still available in the paper which is also included in the gcbd package for R.

Tim Retout: Problem solving

I have developed an affection for crime drama - on Sunday nights, the TV channel Five USA shows something like four hours of CSI, back to back. There's obviously something fascinating about the problem-solving process... and there's always the too-good-to-be-true computer interfaces to laugh at. (I also watch The Mentalist, NCIS and Law & Order, for variety.) And also, on CSI:NY and Law & Order, I get to spot the places in New York I visited at DebConf. So yes, problem solving. Sometimes the real thing can be even more compelling. And who wants to slump in front of the TV for four hours, anyway? Much better to sit up in front of the computer for... erm... a lot longer. This was a more challenging bug - I had to break out gdb and everything. Turns out string manipulation with C is still error-prone. First reported upstream in October 2008 - quite a mystery. NMU to follow later in the week, since there are two other serious bugs open for gnucash.

3 October 2010

Gregor Herrmann: RC bugs 2010/38 - 2010/39

diocles' blog post reminds me that it's time again for some more RCBW propaganda. so here are the RC bugs I've worked on in the last two weeks: which RC bug are you going to tackle today?

Colin Watson: Pipeline library

When I took over man-db in 2001, one of the major problems that became evident after maintaining it for a while was the way it handled subprocesses. The nature of man and friends means that it spends a lot of time calling sequences of programs such as zsoelim < input-file tbl nroff -mandoc -Tutf8. Back then, it was using C library facilities such as system and popen for all this, and I had to deal with several bugs where those functions were being called with untrusted input as arguments without properly escaping metacharacters. Of course it was possible to chase around every such call inserting appropriate escaping functions, but this was always bound to be error-prone and one of the tasks that rapidly became important to me was arranging to start subprocesses in a way that was fundamentally immune to this kind of bug. In higher-level languages, there are usually standard constructs which are safer than just passing a command line to the shell. For example, in Perl you can use system([$command, $arg1, $arg2, ...]) to invoke a program with arguments without the interference of the shell, and perlipc(1) describes various facilities for connecting them together. In Python, the subprocess module allows you to create pipelines easily and safely (as long as you remember the SIGPIPE gotcha). C has the fork and execve primitives, but assembling these to construct full-blown pipelines correctly is difficult and error-prone, so many programmers don't bother and use the simple but unsafe library facilities instead. I wrote a couple of thousand lines of library code in man-db to address this problem, loosely and now quite distantly based on code in groff. In the following examples, function names starting with command_, pipeline_, or decompress_ are real functions in the library, while any other function names are pseudocode. Constructing the simplified example pipeline from my first paragraph using this library looks like this:
pipeline *p;
int status;
p = pipeline_new ();
p->want_infile = "input-file";
pipeline_command_args (p, "zsoelim", NULL);
pipeline_command_args (p, "tbl", NULL);
pipeline_command_args (p, "nroff", "-mandoc", "-Tutf8", NULL);
pipeline_start (p);
status = pipeline_wait (p);
pipeline_free (p);
You might want to construct a command more dynamically:
command *manconv = command_new_args ("manconv", "-f", from_code,
                                     "-t", "UTF-8", NULL);
if (quiet)
	command_arg (manconv, "-q");
pipeline_command (p, manconv);
Perhaps you want an environment variable set only while running a certain command:
command *less = command_new ("less");
command_setenv (less, "LESSCHARSET", lesscharset);
You might find yourself needing to pass the output of one pipeline to several other pipelines, in a "tee" arrangement:
pipeline *source, *sink1, *sink2;
source = make_source ();
sink1 = make_sink1 ();
sink2 = make_sink2 ();
pipeline_connect (source, sink1, sink2, NULL);
/* Pump data among these pipelines until there's nothing left. */
pipeline_pump (source, sink1, sink2, NULL);
pipeline_free (sink2);
pipeline_free (sink1);
pipeline_free (source);
Maybe one of your commands is actually an in-process function, rather than an external program:
command *inproc = command_new_function ("in-process", &func, NULL, NULL);
pipeline_command (p, inproc);
Sometimes your program needs to consume the output of a pipeline, rather than sending it all to some other subprocess:
pipeline *p = make_pipeline ();
const char *line;
line = pipeline_peekline (p);
if (!strstr (line, "coding: UTF-8"))
	printf ("Unicode text follows:\n");
while (line = pipeline_readline (p))
	printf ("  %s", line);
pipeline_free (p);
man-db deals with compressed files a lot, so I wrote an add-on library for opening compressed files (which is somewhat man-db-specific, but the implementation wasn't difficult given the underlying library):
pipeline *decomp_file = decompress_open (compressed_filename);
pipeline *decomp_stdin = decompress_fdopen (fileno (stdin));
This library has been in production in man-db for over five years now. The very careful signal handling code has been reviewed independently and the whole thing has been run through multiple static analysis tools, although I would always welcome more review; in particular I have no idea what it would take to make it safe for use in threaded programs since I generally avoid threading wherever possible. There have been a handful of bugs, which I've fixed promptly, and I've added various new features to support particular requirements of man-db (though in as general a way as possible). Every so often I see somebody asking about subprocess handling in C, and I wonder if I should split this library out into a standalone package so that it can be used elsewhere. Web searches for things like "pipeline library" and "libpipeline" don't reveal anything that's a particularly close match for what I have. The licensing would be GPLv2 or later; this isn't likely to be negotiable since some of the original code wasn't mine and in any case I don't feel particularly bad about giving an advantage to GPLed programs. For more details on the interface, the header file is well-commented. Is there enough interest in this to make the effort of producing a separate library package worthwhile? As well as the general effort of creating a new package, I'd need to do some work to disentangle it from a few bits and pieces specific to man-db. If you maintain a specific package that could use this and you're interested, please contact me with details, mentioning any extensions you think you'd need. I intentionally haven't enabled comments on my blog for various reasons, but you can e-mail me at cjwatson at debian.org or man-db-devel at nongnu.org.

Erich Schubert: Google changing the web

Recently, Google started changing the web in some interesting ways. A lot by contributing code. It started with projects such as the Chrome web browser, which quickly achieved a market shares over 10%, which is more than Apple Safari and Opera together.And the good side, Chrome did not actually add "yet another incompatible platform", but by being closely related to Safari (or more precisely WebKit), it is not that different.Other stuff that belongs into this category is the Google Web Toolkit and the closure compiler (which essentially is a JavaScript code checker and optimizer).But recently, Google goes a step further. They started tackling internet file formats. First they proposed a new Video codec known as WebM or VP8. Now they extracted an image format out of this codec, known as WebP.There is some serious doubt on the quality, the images do appear to be a bit blurry and lose details. I concur with JPEG even being a bit better, despite the block artefacts. The blurry results of WebP are not really convincing.The h.264 based image looks best to me, but H.264 is patent-encumbered for all I know, so we might not see this new format in wide use until 2028. This already ruled out other image formats such as JBIG or FIF. Or look at audio formats: we're still using MP3 everywhere, because everyone tries to push another patent-encumbered audio format that the others are not going to adopt. So MP3 remains the only thing widely accepted (Ogg Vorbis is nice, but it looks as if companies such as Apple are not going to adopt it, probably because they have interest in other formats. Maybe the Ogg Vorbis support in Chrome will help here on the long run).In particular, why would you pay royalties for a tiny bit of image quality, given the constantly sinking prices of bandwidth? The H.264 patents may make a difference on video data, they probably won't pay of for still images.Given that WebM is sometimes claimed to be a simplified H.264 (minus the patent issues?), WebP probably is as good as we can get to the H.264 based quality without running into the same patent issues?But enough on the image data. There is one thing, I'd really like to see Google change on the web. The worst thing about the web is called JavaScript (or more accurately, ECMAScript). There is just so much wrong with this language that we really need to replace it ASAP.Some things that are wrong with JavaScript: So my biggest wishlist item of Google to contribute to the Web via Chrome and Firefox is a new web scripting language. As much as I love python, the whitespace syntax of Python doesn't work for the web (we need to be able to store complex statements in HTML attributes without newlines), so we'll have to look for something else. Maybe Ruby will work?

Petter Reinholdtsen: Links for 2010-10-03

Tim Retout: NMUs

It's been raining this weekend in Southampton. This afternoon's not been too bad - I went for a walk around the city walls. Just a few hundred years ago, the River Test would have come right up to them, and my flat would have been in a defensive moat full of sea water, I think. I re-read zack's rcbw page, and noticed the section on 'blog posts' - there is also a propaganda element to the idea, in improving Debian's internal culture w.r.t. NMUs. So with my grey Sunday afternoon, I have uploaded a couple of NMUs for RC bugs: I am surprised that there are still such easy pickings from the RC bug list at this point in the release cycle. This Debian Developer lark is a piece of cake, innit?

Matt Zimmerman: Tips for frequent international travel

I travel pretty regularly, about 35% so far in 2010. When it goes wrong, travel can be exhausting, frustrating, complicated, stressful and even debilitating. I m always looking for ways to make my trips run more smoothly. On a recent flight to Taipei, I wrote down a few of the techniques which I ve successfully put into practice and found helpful. This is not an exhaustive list; I ve omitted a lot of the common and obvious tips I ve seen elsewhere.
  1. Make a packing list. This one may be obvious, but a lot of people neglect it. Perhaps they think making lists is boring and fussy, but really, it isn t. Without a packing list, it s easy to forget to do the things which will make your trip better. Use it every time, and bring a copy with you (or store it online) so you can add the things you wish you had brought or done. A simple, ever-improving packing list is the most effective technique I have found for making travel less stressful and more enjoyable.
  2. Carry a water bottle with a tight-fitting lid. I use a 32oz Nalgene bottle, which fits nicely into the seat next to me or under an armrest, and gives me enough water for even the longest flights. I fill it up after passing through security, at a cafe, bar or lounge, and generally decline the beverages offered by the cabin crew. Staying hydrated helps me feel better during the flight, and leaves me with less malaise when I arrive. I don t need to manage a tray table or armrest full of cups and other debris, so I can sit more comfortably, with the tray table folded away.
  3. Consolidate essential items using multipurpose equipment. For example, invest in a power adapter which has USB sockets onboard, and carry USB cables instead of wall chargers. Versatile items like this save on space and weight. I can charge two devices this way, but the equipment is smaller and lighter than even a single wall charger.
  4. Learn how to sleep on an airplane. Getting some sleep on a long flight really helps to offset the effects of traveling. There are several resources out there with practical advice on how to do it. One thing which really helped me was to buy a high quality eye mask which blocks out all of the light in the cabin. The one I use looks a little funny and is not cheap, but is very comfortable and effective. It s made of memory foam with a soft, washable cover and works much better than the ones the airlines give away for free. I no longer bother with a neck pillow, and use the flaps built into the seat to lean my head against. I m surprised at how many people don t know about this common aricraft feature: virtually every long-haul seat has something like this, even in economy, though it may not be obvious how to use it.
  5. Buy duplicates of things like toiletries, and keep them in your travel kit so you don t need to pack your everyday items (and risk forgetting them) each time. The less packing you need to do, the less time it will take, and the less opportunity there is for mistakes. This also saves time unpacking when you get home, and lets you buy a smaller size of the item where available.
  6. Optimize border crossings Carry the forms you ll need for customs, immigration, etc. in your carry-on. They don t always provide them at the counter or on board the plane, and it s a hassle to rush to fill in the form at the last minute. If you have a few of them with you, you can fill them out early (perhaps even before you fly) and then hustle to the front of the queue. For countries you enter frequently (especially your home country), programs like Global Entry (US) and IRIS (UK) will save you a lot of time by allowing you to use an automated kiosk to cross the border.

Matt Zimmerman: The paradox of Steve Jobs

Steve Jobs is a name that comes up a lot when talking to businesspeople, especially in the technology industry. His ideas, his background, his companies, their products, and his personal style are intertwined in the folklore of tech. I have no idea whether most of it is fiction or not, and I write this with apologies to Mr. Jobs for using him as shorthand. I have never met him, and my point here has nothing to do with him personally. What I want to discuss is the behavior of people who invoke the myth of Steve Jobs. In my (entirely subjective) experience, it seems to me that there is a pattern which comes up again and again: People seem to want to discuss and emulate the worst of his alleged qualities. Jobs has been characterized as abusive to his employees, dismissive of his business partners, harshly critical of mistakes, punishingly secretive, and otherwise extremely difficult to work with. Somehow, it is these qualities which are put forward as worthy of discussion, inspiration and emulation. Is this a simple case of confusing correlation with causation? Do people believe that Steve Jobs is successful because of these traits? Perhaps it is a way of coping with one s own character flaws: if Jobs can get away with such misbehavior, then perhaps we can be excused from trying to improve ourselves. Or is there something more subtle going on here? Maybe this observation is an effect of my own cognitive biases, as it is only anecdotal. As with any successful person, Jobs surely has qualities and techniques which are worthy of study, and perhaps even emulation. Although direct comparison can be problematic, luminaries like Jobs can provide valuable inspiration. I d just like to hear more about what they re doing right. Perhaps this is an argument for drawing inspiration from people you know personally, rather than from second-hand or fictitious accounts of someone else s life. I ve been fortunate to be able to work with many different people, each with their own strengths, weaknesses and style. I ve seen those characteristics first-hand, so I also have the context to understand why they were successful in particular situations. If there s one thing I ve learned about leadership, it s that it s highly context-sensitive: what worked well in one situation can be disastrous in another. Is your company really that much like Apple? Probably not.

Erich Schubert: Copyright kills Music

German RIAA-Equivalent GEMA still has not reached an agreement with YouTube. As a consequence, about half of the YouTube videos I get links for are not accessible because they contain a song that is managed by GEMA.Sorry, I will not buy a song that I can't preview this way.International music companies such as Universal Music Group, EMI and Sony Music definitely should rethink their cooperation with GEMA, they are harming their business.

Next.

Previous.