Search Results: "beh"

6 October 2010

Luca Falavigna: Introducing projectb

Many people know Ultimate Debian Database already, and do use of it to create useful QA tools, or other Debian related activities. There is another database which is widely used by all of us, every day, and runs behind the scenes without being noticed. It is called projectb, the huge database of the Debian archive. projectb is the place where dak stores all the information it needs to run the archive (including but not limited to source packages, related binaries, suite definitions and associations, overrides, and so on), and it tells dak to hit you with a REJECT if you try to upload a package twice, or if you do something The Wrong Way. OK OK, it eventually allows your packages to be accepted too, sometimes it also behaves good :) projectb is hosted on ftp-master.debian.org. Most fellow Debian Developers cannot login over there because access is restricted to FTP and Release people only, but can enjoy its clone on merkel.debian.org [1]. Please note that clone is synced on a regular basis, but it could be a little bit not up-to-date with the very latest changes. Just run the following: psql -d projectb -p 5433 Now that you have access to it, you could want look at its structure to start looking at it. I just provide a couple of trivial examples of what you can do with it, your imagination is the limit [2] ;) This query displays which dpkg version known suites ship:
SELECT s.source, max(s.version) AS version, su.suite_name AS suite
FROM source s
JOIN src_associations sa ON sa.source = s.id
JOIN suite su ON su.id = sa.suite
WHERE s.source = 'dpkg'
GROUP BY s.source, su.suite_name
ORDER BY version; This query displays changelog entries for debhelper which versions are greater than 8:
SELECT changelog
FROM changelogs
WHERE source = 'debhelper'
AND version > '8'; Working with projectb means working with real Debian archive data, so you are sure to always get solid information. You can also experience a limited (but functional) dak session on merkel. You cannot do much, data is read-only, but this is not a limit for a couple of commands. The most interesting is dak rm, where you can simulate a package removal, and discover reverse dependencies in advance, to save some work to your lovely FTP Team ;) Just run the following:
dak rm -n -R -m "Don't worry, it's a Dry Run" your_behated_package Wait a little bit, and you will be prompted with removal report. Just remember to always use -n and -R flags, though. Want even more? Join FTP Team, and you will given the chance to learn a lot more about Debian! DISCLAIMER: if you plan to write a tool, or perform an activity which relies on current projectb implementation, PLEASE TALK TO FTP TEAM FIRST. Internal structure can change a lot over times, and it is way better let FTP Team to handle exporting/parsing its data for day-by-day tasks. [1] This is probably supposed to change, as merkel is going to be decommissioned soon. [2] If you need a new feature, just talk to FTP Team.

Josselin Mouette: Mounting encrypted keys for dumm for gurus

It s often said that KDE and GNOME are too bloated, too complex, too slow, or whatever. I won t deny that these critics are often justified, and some parts of the code are badly designed. But there can also be reasons behind this bloat: they are called features. When you want to mount an encrypted USB disk, you can write your own script and even write your own udev rules so that it can be mounted with autofs. It looks fun to find a way to use software that has been obsolete and useless for 10 years, in a way that requires administrator rights just to add a new model of USB disk to your system, and puts the private key in a place that is readable for anyone stealing the hardware. But while it will turn out as an interesting read for those willing to understand how the device mapper and cryptsetup work, I think it s a bit abusive to present it as a correct implementation of an encrypted disk mounting setup. Discover gnome-disk-utility In etch, GNOME shipped with pmount, a nice utility, still included in Debian, that allows to mount your keys, encrypted or not. In lenny, it shipped with HAL, and allowed to store LUKS passphrases securely in the GNOME keyring. Whenever you plug a LUKS-encrypted disk on a lenny system running GNOME, it is immediately made accessible, and that s all. In squeeze, things go much farther thanks to udisks (the backend) and gnome-disk-utility (the frontend). Roland rightly pointed out that the g-d-u documentation is nonexistent - it consists only in a screenshot, which is outdated. Nevertheless, you will find it practical if you want to encrypt a USB drive, since you can format it, partition it, create encrypted volumes and the filesystems on them, in a few clicks and without root permissions. If you use nautilus, it will also mount them automatically using the same backend when you plug them. I don t know for you, but I think it is worth a few CPU cycles of my 3GHz processor and a few dozen megabytes of my 500GB drive.

Debian News: Who is using Debian?

Here are the Debian users who have submitted a request to appear in our website so far in this year. Please welcome them into our Project! And here are some of their reasons to choose Debian:
We have chosen Debian over other alternatives; because of its reliability and easiness of maintenance.
The decision was based on the selection of additional expandable components, availability of distribution, a regular cycle for issuing amendments, stability and support solutions for a wide range of equipment for a dedicated server solutions.
Stability is important for us.
The main reason for choosing Debian over other distributions is the package management, security, number of available packages, ease-of-use, stability, amount of quality documentation, and cost.
We choose Debian to easen the burden of system administration. Also because Debian has very well maintaned packages and for high security.
Debian is powerful and keeping originality of Linux.
We find the software that comes with Debian very useful and easy to use since we are not specialists in computer matters.
Debian is the most powerful Linux distribution! The best choice for our jobs, our servers and our lifestyle.
We chose Debian because it is stable, simple and fast to install and to use, and because it is Free Software.
We are proud to say that we use Debian/GNU Linux on more than 90% of our servers and this simply because of it s stability, security and great packaging system.
Debian offered the perfect balance between security, stability and ease of administration.
We chose Debian for its rock-solid stability, ease of installation and configuration, and awesome ability to not only run on old, second-hand hardware, but to turn that hardware into powerful production machines perfect for our needs. We also like Debian s Social Contract. None of the other options were as successful in this implementation.
It is refreshing and enjoyable to have an operating system which gives users easy access.
Thanks very much for the hard work, Webmaster Team.

4 October 2010

Raphaël Hertzog: Can Debian offer a Constantly Usable Testing distribution?

Debian s testing distribution is where Debian developers prepare the next stable distribution. While this is still its main purpose, many users have adopted this version of Debian because it offers them a good trade-off between stability and freshness. But there are downsides to using this distribution and the Constantly Usable Testing (CUT) project aims to resolve those. This article will present the project and the challenges involved to make it happen. About Debian unstable & testing Debian unstable is the distribution where developers upload new versions of their packages. It happens frequently that some packages are not installable due to changes in other packages or due to transitions not yet completed. Debian testing, on the contrary, is managed by a tool that ensures the consistency of the whole distribution: it picks updates from unstable only if the package has been enough tested (10 days usually), if it s free of new release-critical bugs, if it s available on all supported architectures, and if it doesn t break any other package already present in testing. The Release Team (RT) controls this tool and provide hints to help it find a set of packages that can flow from unstable to testing. Those rules also ensure that the packages that flow into testing are reasonably free of show-stopper bugs (like a system that doesn t boot, or X that doesn t work at all). This makes it very attractive to users who like to regularly get new upstream versions of their software without dealing with the biggest problems associated to them. This is all very attractive, yet several Debian developers advise people to not use testing. Why is that? Known problems with testing Disappearing software The release team use this distribution to prepare the next stable release and from time to time they remove packages from it. Either because it s needed to ensure that other packages can migrate from unstable to testing, or because they have long-standing release-critical bugs without progress towards a resolution. It also happens that they remove packages on request of the maintainers because they believe that the current version of the software cannot be supported (security-wise) for 2 years or more. The security team also regularly issues such requests. Long delays for security and important fixes Despite the 10-day delay in unstable, there are always some annoying bugs (and security bugs are no exceptions) that are only discovered when the package already has migrated to testing. The maintainer might be quick to upload a fixed package in unstable, and might even raise the urgency to allow the package to migrate sooner, but if the packages gets entangled in a large ongoing transition, it will not migrate before the transition is completed. Sometimes it can take weeks for that to happen. This delay can be avoided by doing direct uploads to testing (through testing-proposed-updates) but this is almost never used, except during a freeze, where targeted bugfixes are the norm. Not always installable With testing evolving daily, updates sometimes break the last installation images available (in particular netboot images that get everything from the network). The debian-installer (d-i) packages are usually quickly fixed but they don t move to testing automatically because the new combination of d-i packages has not necessarily been validated yet. Colin Watson sums up the problem:
Getting new installer code into testing takes too long, and problems remain unfixed in testing for too long. [...] The problem with d-i development at the moment is more that we re very slow at producing new d-i *releases*. [...] Your choices right now are to work with stable (too old), testing (would be nice except for the way sometimes it breaks and then it tends to take a week to fix anything), unstable (breaks all the time).
CUT s history CUT finds its root in an old proposal by Joey Hess: it introduces the idea that the stable release is not Debian s sole product and that testing could become with some work a suitable choice for end-users. Nobody took on that work and there was no visible progress in the last 3 years. But recently Joey brought up CUT again on the debian-devel mailing list and Stefano Zacchiroli (the Debian project leader) challenged him to setup a BoF on CUT for Debconf10. It turned out to be one of the most heavily attended BoF (video recording is here), there is clearly a lot of interest in the topic. There s now a dedicated wiki and an Alioth project with a mailing list. The rest of this article tries to summarize the various options discussed and how they re supposed to address the problems identified. The ideas behind CUT Among all the ideas, there are two main approaches that have been discussed. The first is to regularly snapshot testing at points where it is known to work reasonably well (those snapshots would be named cut ). The second is to build an improved testing distribution tailored to the needs of users who want a working distribution with daily updates, its name would be rolling . Regular snapshots of testing There s general agreement that regular snapshots of testing are required: it s the only way to ensure that the generated installation media will continue to work until the next snapshot. If tests of the snapshot do not reveal any major problem, then it becomes the latest cut . For clarity, the official codename would be date based: e.g. cut-2010-09 would be the cut taken during September 2010. While the frequency has not been fixed yet, the goal is clearly to be on the aggressive side: at the very least every 6 months, but every month has been suggested as well. In order to reach a decision, many aspects have to be balanced. One of them (and possibly the most important) is the security support. Given that the security team is already overworked, it s difficult to put more work on their shoulders by declaring that cuts will be supported like any stable release. No official security support sounds bad but it s not necessarily so problematic as one might imagine. Testing s security record is generally better than stable s one (see the security tracker) because fixes flow in naturally with new upstream versions. Stable still get fixes for very important security issues sooner than testing, but on the whole there are less known security-related problems in testing than in stable. Since it s only a question of time until the fixed version comes naturally from upstream, more frequent cut releases means that users get security fixes sooner. But Stefan Fritsch, who used to be involved in the Debian testing security team, has also experienced the downside for anyone who tries to contribute security updates:
The updates to testing-security usually stay useful only for a few weeks, until a fixed version migrates from unstable. In stable, the updates stay around for a few years, which gives a higher motivation to spend time on preparing them.
So if it s difficult to form a dedicated security team, the work of providing security updates comes back to the package maintainer. They are usually quite quick to upload fixed packages in unstable but tend to not monitor whether the packages migrate to testing. They can t be blamed because testing was created to prepare the next stable release and there is thus no urgency to get the fix in as long as it makes it before the release. CUT can help in this regard precisely because it changes this assumption: there will be users of the testing packages and they deserve to get security fixes much like the stable users. Another aspect to consider when picking a release frequency is the amount of associated work that comes with any official release: testing upgrades from the previous version, writing release notes and preparing installation images. It seems difficult to do this every month. With this frequency it s also impossible to have a new major kernel release for each cut (since they tend to come out only every 2 to 3 months) and the new hardware support that it brings is something worthwhile to many users. In summary, regular snapshots address the not always installable problem and changes the perception of maintainers towards testing, so that hopefully they care more of security updates in that distribution (and in cuts). But they do not solve the problem of disappearing packages. Something else is needed to fix that problem. A new rolling distribution? Lucas Nussbaum pointed out that regular snapshots of Debian is not really a new concept:
How would this differentiate from other distributions doing 6-month release cycles, and in particular Ubuntu, which can already be seen as Debian snapshots (+ added value)?
In Lucas s eyes, CUT becomes interesting if it can provide a rolling distribution (like testing) with a constant flux of new upstream releases . For him, that would be something quite unique in the Free Software world . The snapshots would be used as starting point for the initial installation, but the installed system would point to the rolling distribution and users would then upgrade as often as they want. In this scenario, security support for the snapshots is not so important, what matters is the state of the rolling distribution. If testing were used as the rolling distribution, the problem of disappearing packages would not be fixed. That s why there have been discussions of introducing a new distribution named rolling that would work like testing but with adapted rules, and the cuts would then be snapshots of rolling instead of testing. The basic proposal is to make a copy of testing and to re-add the packages which have been removed because they are not suited for a long term release while they are perfectly acceptable for a constantly updated release (the most recent example being Chromium). Then it s possible to go one step further: during freeze, testing is no longer automatically updated which makes it inappropriate to feed the rolling distribution. That s why rolling would be reconfigured to grab updates from unstable (but using the same rules than testing). Given the frequent releases, it s likely that only a subset of architectures would be officially supported. This is not a real problem because the users who want bleeding edge software tends to be desktop users on mainly i386/amd64 (and maybe armel for tablets and similar mobile products). This choice if made opens up the door to even more possibilities: if rolling is configured exactly like testing but with only a subset of the architectures, it s likely that some packages migrate to rolling before testing when non-mainstream architectures are lagging in terms of auto-building (or have toolchain problems). While being ahead of testing can be positive for the users, it s also problematic on several levels. First, managing rolling becomes much more complicated because the transition management work done by the release team can t be reused as-is. Then it introduces competition between both distributions which can make it more difficult to get a stable release out, for example if maintainers stop caring of the migration to testing once the migration to rolling has been completed. The rolling distribution is certainly a good idea but the rules governing it must be designed to avoid any conflict with the process of releasing a stable distribution. Lastly, the mere existence of rolling would finally fix the marketing problem plaguing testing: the name rolling does not suggest that the software is not yet ready for prime time. Conclusion Whether CUT will be implemented remains to be seen, but it s off for a good start: ftpmaster Joerg Jaspert said that the new archive server can cope with a new distribution, and there s now a proposal shaping up. The project might start quickly: there is already an implementation plan for the snapshot side of the project. The rolling distribution can always be introduced later, once it is ready. Both approaches can complement each other and provide something useful to different kind of users. The global proposal is certainly appealing: it would address the concerns of obsolescence of Debian s stable release by making intermediary releases. Anyone needing something more recent for hardware support can start by installing a cut and follow the subsequent releases until the next stable version. And users who always want the latest version of every software could use rolling after having installed a cut. From a user point of view, there are similarities with the mix of usual and long term releases of Ubuntu. But from the development side, the process followed would be quite different, and the constraints imposed by having a constantly usable distribution are stronger: any wide-scale change must be designed in a way that it can happen progressively in a transparent manner for the user.
This article was first published in Linux Weekly News. If you want to see more articles like this, join Flattr and click on the flattr button below every article that you like.
Flattr this Share/Bookmark 21 comments Support my work

3 October 2010

Matt Zimmerman: The paradox of Steve Jobs

Steve Jobs is a name that comes up a lot when talking to businesspeople, especially in the technology industry. His ideas, his background, his companies, their products, and his personal style are intertwined in the folklore of tech. I have no idea whether most of it is fiction or not, and I write this with apologies to Mr. Jobs for using him as shorthand. I have never met him, and my point here has nothing to do with him personally. What I want to discuss is the behavior of people who invoke the myth of Steve Jobs. In my (entirely subjective) experience, it seems to me that there is a pattern which comes up again and again: People seem to want to discuss and emulate the worst of his alleged qualities. Jobs has been characterized as abusive to his employees, dismissive of his business partners, harshly critical of mistakes, punishingly secretive, and otherwise extremely difficult to work with. Somehow, it is these qualities which are put forward as worthy of discussion, inspiration and emulation. Is this a simple case of confusing correlation with causation? Do people believe that Steve Jobs is successful because of these traits? Perhaps it is a way of coping with one s own character flaws: if Jobs can get away with such misbehavior, then perhaps we can be excused from trying to improve ourselves. Or is there something more subtle going on here? Maybe this observation is an effect of my own cognitive biases, as it is only anecdotal. As with any successful person, Jobs surely has qualities and techniques which are worthy of study, and perhaps even emulation. Although direct comparison can be problematic, luminaries like Jobs can provide valuable inspiration. I d just like to hear more about what they re doing right. Perhaps this is an argument for drawing inspiration from people you know personally, rather than from second-hand or fictitious accounts of someone else s life. I ve been fortunate to be able to work with many different people, each with their own strengths, weaknesses and style. I ve seen those characteristics first-hand, so I also have the context to understand why they were successful in particular situations. If there s one thing I ve learned about leadership, it s that it s highly context-sensitive: what worked well in one situation can be disastrous in another. Is your company really that much like Apple? Probably not.

30 September 2010

Evgeni Golov: What to flattr?

The month is coming to an end, and I d like to recommend you some things on flattr.com :)OpenRheinRuhr Ein Pott voll Software Die OpenRheinRuhr ist eine Messe mit Kongress rund um das Thema Freie Software . Die OpenRheinRuhr 2010 findet am 13. und 14. November im Rheinischen Industriemuseum in Oberhausen statt. Freie Software ist aus Firmen, Verwaltungen, Bildungseinrichtungen und Privathaushalten nicht mehr wegzudenken. In der bev lkerungsreichsten Region Europas bietet der OpenRheinRuhr e.V. durch Ausstellungen, Vortr ge und Workshops Informationen ber Freie Software. Auch Themen jenseits der Technik, wie B rgerrechte im Netz oder die Handhabung von Lizenzen werden behandelt. Vortr ge in mehreren Tracks sprechen Menschen mit unterschiedlichem Kenntnisstand an vom Anf nger bis zum Profi. Entwickler & Projekte bekommen die Gelegenheit zum Erfahrungsaustausch. Weitere Informationen und Anmeldem glichkeiten unter: http://openrheinruhr.deBitlBee The IRC geek s solution to instant messaging. Tunnels instant messaging traffic (supporting all popular IM protocols and Twitter) to a virtual IRC channel and virtual IRC queries. Can be installed locally, but also available as a public service for people who can t/don t want to install it.
KiBi s blog This blog mainly features Debian-related posts on various topics: X.Org package maintenance (including calls for help, and status updates towards users), Debian GNU/kFreeBSD (GNU userland running on a FreeBSD kernel), and Debian s Graphical Installer (now based on X.Org). Also some upstream stuff, like the bugzilla replica type for SD (Simple Defects).
phpMyAdmin phpMyAdmin is a tool intended to handle the administration of MySQL over the Web. It can create, rename, and drop databases, create/drop/alter tables, delete/edit/add fields, execute any SQL statement, manage keys on fields, create dumps of tables and databases, export/import CSV data, and administrate one single database and multiple MySQL servers.
Debian Backports Over the last years I did numerous backports for Debian and also run backports.org. Since September 2010 we moved backports.org to backports.debian.org so its an officialDescriptions shamelessly stolen from flattr.Some words on these things:

27 September 2010

Raphaël Hertzog: How to generate different dependencies on Debian and Ubuntu with a common source package

There are situations in which a given package needs to have different dependencies in Debian and in Ubuntu. Despite this difference it s possible to keep a single source package that will build both variants of the package. Continue reading to discover a step-by-step explanation. 1. When is that needed? While it is possible to have different dependencies depending on the distribution on which the package is built, it should be usually avoided when possible. This infrastructure should only be used as a last resort when there are no better alternatives. Here are some examples of when it might be needed: 2. Using substitution variables in debian/control The dependency that varies between both distributions can t be hardcoded in debian/control, instead you should put a substitution variable (substvar) that will be replaced at build time by dpkg-gencontrol. You can name it $ dist:Depends for example: 
[...]
Depends: bzip2, $ shlibs:Depends , $ misc:Depends , $ dist:Depends 
[...]
Note that you typically already have other substitution variables ($ shlibs:Depends comes from dpkg-shlibdeps and $ misc:Depends from debhelper and its dh_* scripts). 3. dpkg-gencontrol needs a -V option The value used to replace this new variable needs to be communicated to dpkg-gencontrol. You can use the -V option for this, the syntax would be something like this: 
dpkg-gencontrol [...] -Vdist:Depends="foo (>= 2), bar"
If you use debhelper, you have to pass the option to dh_gencontrol after two dashes (--): 
dh_gencontrol -- -Vdist:Depends="foo (>= 2), bar"
If you use CDBS, you can set the DEB_DH_GENCONTROL_ARGS_ALL make variable: 
include /usr/share/cdbs/1/rules/debhelper.mk
DEB_DH_GENCONTROL_ARGS_ALL = -Vdist:Depends="foo (>= 2), bar"
The value given to dpkg-gencontrol is static in all those examples, now let s see how we can use give a different value depending on the distribution that we re targetting. 4. Using dpkg-vendor in debian/rules dpkg-vendor is a small tool (provided by the dpkg-dev package) that parses the /etc/dpkg/origins/default file (provided by the base-files package) to know the current distribution and its ancestry. It can be used in debian/rules to adjust the behavior depending on the current distribution. You can check its man-page to learn about the various options supported but we re only going to use --derives-from <vendor> in this sample. With this option the script exits with zero if the current distribution is or derives from the indicated distribution, or with 1 otherwise. Now combining all together, we can use dpkg-vendor to dynamically define the content of the substitution variable in debian/rules. Let s suppose that you want $ dist:Depends to be foo (>= 2) on Ubuntu (and its derivatives) and bar everywhere else. Using debhelper s 7 tiny rules file, this example could be: 
ifeq ($(shell dpkg-vendor --derives-from Ubuntu && echo yes),yes)
	SUBSTVARS = -Vdist:Depends="foo (>= 2)"
else
	SUBSTVARS = -Vdist:Depends="bar"
endif
%:
	dh $@
override_dh_gencontrol:
	dh_gencontrol -- $(SUBSTVARS)
If you use CDBS, it could be this: 
include /usr/share/cdbs/1/rules/debhelper.mk
ifeq ($(shell dpkg-vendor --derives-from Ubuntu && echo yes),yes)
	DEB_DH_GENCONTROL_ARGS_ALL = -Vdist:Depends="foo (>= 2)"
else
	DEB_DH_GENCONTROL_ARGS_ALL = -Vdist:Depends="bar"
endif
Do you want to read more tutorials like this one? Click here to subscribe to my free newsletter, you can opt to receive future articles by email.
Flattr this Share/Bookmark 9 comments Support my work

23 September 2010

Dirk Eddelbuettel: R Project and Google Summer of Code: Wrapping up

As this year's admin, I wrote up the following summary which has now been posted at the R site in the appropriate slot. My thanks to this year's students, fellow mentors and everybody else who helped to make it happen. GSoC 2010 logo

Projects 2010 As in 2008 and 2009, the R Project has again participated in the Google Summer of Code during 2010. Based on ideas collected and disussed on the R Wiki, the projects and students listed below (and sorted alphabetically by student) were selected for participation and have been sponsored by Google during the summer 2010. The finished projects are available via the R / GSoC 2010 repository at Google Code, and in several cases also via their individual repos (see below). Informal updates and final summaries on the work was also provided via the GSoC 2010 R group blog.

rdx - Automatic Differentiation in R Chidambaram Annamalai, mentored by John Nash. Proposal: radx is a package to compute derivatives (of any order) of native R code for multivariate functions with vector outputs, f:R^m -> R^n, through Automatic Differentiation (AD). Numerical evaluation of derivatives has widespread uses in many fields. rdx will implement two modes for the computation of derivatives, the Forward and Reverse modes of AD, combining which we can efficiently compute Jacobians and Hessians. Higher order derivatives will be evaluated through Univariate Taylor Propagation. Delivered: Two packages radx: forward automatic differentiation in R and tada: templated automatic differentiation in C++ were created; see this blog post for details.

A GUI for Graphics using ggplot and Deducer by Ian Fellows, mentored by Hadley Wickham. Proposal: R puts the latest statistical techniques at one's fingertips through thousands of add-on packages available on the CRAN download servers. The price for all of this power is complexity. Deducer is a cross-platform cross-console graphical user interface built on top of R designed to reduce this complexity. This project proposes to extend the scope of Deducer by creating an innovative yet intuitive system for generating statistical graphics based on the ggplot2 package. Delivered: All of the major features have been implemented, and are outlined in the video links in this blog post.

rgeos - an R wrapper for GEOS by Colin Rundel, mentored by Roger Bivand. Proposal: At present there does not exist a robust geometry engine available to R, the tools that are available tend to be limited in scope and do not easily integrate with existing spatial analysis tools. GEOS is a powerful open source geometry engine written in C++ that implements spatial functions and operators from the OpenGIS Simple Features for SQL specification. rgeos will make these tools available within R and will integrate with existing spatial data tools through the sp package. Delivered: The rgeos project on R-Forge; see the final update blog post.

Social Relations Analyses in R by Felix Schoenbrodt, mentored by Stefan Schmukle. Proposal: Social Relations Analyses (SRAs; Kenny, 1994) are a hot topic both in personality and in social psychology. While more and more research groups adopt the methodology, software solutions are lacking far behind - the main software for calculating SRAs are two DOS programs from 1995, which have a lot of restrictions. My GSOC project will extend the functionality of these existing programs and bring the power of SRAs into the R Environment for Statistical Computing as a state-of-the-art package. Delivered: The TripleR package is now on CRAN and hosted on RForge.Net; see this blog post for updates.

NoSQL Interface for R by Yasuhisa Yoshida, mentored by Dirk Eddelbuettel. Proposal: So-called NoSQL databases are becoming increasingly popular. They generally provide very efficient lookup of key/value pairs. I'll provide several implementation of NoSQL interface for R. Beyond a sample interface package, I'll try to support generic interface similar to what the DBI package does for SQL backends Status: An initial prototype is available via RTokyoCabinet on Github. No updates were made since June; no communication occurred with anybody related to the GSoC project since June and the project earned a fail.

Last modified: Wed Sep 22 19:39:43 CDT 2010

Julien Danjou: Gnus news is good news!

As I already wrote too many times, I've started to use Gnus 6 months ago, and never looked back. At that time, I joined the ding mailing list in order to ask some dumb questions and, once, send a patch. There were very low activity on that list. Until Lars, the original Gnus author, came back. Three weeks ago, he started to wrote a new wash function to render HTML mails properly, with pictures. It's named gnus-html', and is (for now) based on w3m (but not on emacs-w3m, which is not part of Emacs). Last week, I've sent a set of patches to replace the usage of curl by the standard url-retrieve' function to fetch images, plus various enhancement. It seems that my work was good enough that Lars offered me write access to the git repository. I can therefore mess up the Gnus entirely. Hurrah! I've continued to work on gnus-html' and recently merged a set of patches improving image retrieval (which is now done in parallel) and starting to use url-cache' to cache image for a defined period of time. Of course, I found a bunch of tiny bug and special case while reading RSS feeds and various HTML mails, and fixed them all along. Lars added a libxml binding for Emacs 24, providing the html-parse-string' function. His future plan seems to be the abandon of w3m in favor of a native parsing via libxml to render HTML, and therefore, HTML mails. I should also mention the new nnimap' back-end; Gnus has been designed to read NNTP newsgroups, and not mails. Consequently, it had a very poor behaviour when used with a back-end such has IMAP. Lars took a week to rewrite entirely our dear nnimap' back-end, and make it act in a more expected way. There's still a bunch of bug and code to write, but it is at least usable and seems faster than the old code. Last thing I did was to rewrite the icon support in the group buffer. When I started to use Gnus, I was curious and tried to configure this. I never managed to make it work, and now know and understand why it was broken. So I ended rewriting entirely, and now it works. I never though I would understand, fix, and commit this code when reading the Gnus documentation this winter, but hell yeah, I did. Now I've still several little project to improve things in all sort of area. We'll see what I'll do next. :-) Flattr this

22 September 2010

Gunnar Wolf: Damage control: Cleaning up compromised SSH keys

This morning, my laptop was stolen from my parked car while I was jogging. I do not want to make a big deal out of it. Still, even though I am sure it was not targetted at my data (three other people at least were reporting similar facts in the same area), and the laptop's disk will probably just be reformatted, I am trying to limit the possible impact of my cryptographic identification being in somebody else's hands. GPG makes it easy: I had on that machine just my old 1024D key, so it is just matter of generating a revocation certificate. I have done that, and uploaded it to the SKS keyservers - Anyway, here is my revocation certificate: 
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: A revocation certificate should follow
iHIEIBEIADIFAkyaOZwrHQJBIGNvbXB1dGVyIGNvbnRhaW5pbmcgdGhpcyBrZXkg
d2FzIHN0b2xlbgAKCRDYDvNai7UnrzWAAKC34eF76JQjxrZqSjNwcC0dU/5VbACg
gMIMmYg91Sl3y8KsZXdGj/rV7UE=
=rdlT
-----END PGP PUBLIC KEY BLOCK-----
But What worries me more is access to the computers my ssh key works for. Yes, the ssh key uses a nontrivial passphrase, but still SSH keys cannot be revoked (and this makes sense, as SSH should not add the delay, or potential impossibility, to check with a remote infrastructure whenever you want to start a session). So, I generated a new key (and stored it at ~/.ssh/id_rsa.new / ~/.ssh/id_rsa.new.pub) and came up with this snippet: 
  1. $ OLDKEY=xyHywJuHD3nsfLh03G1TqUEBKSj6NlzMfB1T759haoAQ
  2. $ for host in $(cat .ssh/known_hosts   cut -f 1 -d \  cut -f 1 -d ,  
  3.                 sort   uniq); do 
  4.     echo == $host
  5.     ssh-copy-id -i .ssh/id_rsa.new.pub $host && 
  6.     ssh $host "perl -n -i -e 'next if /$OLDKEY/;print' .ssh/authorized_keys"
  7. done
Points about it you might scratch your head about: Caveats: Oh, by the way: If you noticed me knocking on your SSH ports... please disregard. Possibly at some point I connected to that machine to do something, or it landed in my .ssh/known_hosts for some reason. I currently have 144 hosts registered. I am sure I triggered at least one raised eyebrow. And I will do it from a couple of different computers, to make it less probable that I miss some I have never connected from while at the particular computer I am sitting at right now. So... Any ideas on how to make this better?

Michal &#268;iha&#345;: Fixing phpMyAdmin bugs in Launchpad

Once in a time when looking at my Debian Developer's Packages Overview, I just start to think there is way too much bug reports for phpMyAdmin package in Launchpad. Today the threshold was 30 :-). Managing them is usually not that hard, because vast majority of them are just duplicates of same issue. Basically all problems come with the fact that dbconfig-common really does not behave nicely when database server is not configured and user tries to configure it. I know people should just answer no to first question, whether to use dbconfig-common, but they don't and it then horribly fails. Today I finally decided to ignore all errors from dbconfig-common in our packaging scripts, so you will get an error, but the package will be installed. As it seems that this is what most Ubuntu users expect after all. This also shows huge difference between Debian and Ubuntu users - such bug has never appeared in Debian bug tracker, but has zillions of duplicates in Launchpad in various incarnations (LP#618852, LP#621569 and the most favorite LP#456674).

21 September 2010

Rapha&#235;l Hertzog: Everything you need to know about conffiles: configuration files managed by dpkg

The Debian policy dictates that package upgrades must take care of preserving user changes to configuration files. This article will explain you how most packages ensure this. This is important knowledge for anyone who has to manage upgrades: knowing how it works lets you easily automate most of it and deal correctly with the fallout. How dpkg manages configuration files Most packages rely on dpkg to properly install configuration files. Dpkg keeps a checksum of the last installed version of configuration file. When it must install a new version, it calculates the checksum of the currently installed file and if it doesn t match anymore, it knows that the user has edited the file. In that case, instead of overwriting the configuration file, it asks the user what to do. You probably already have seen those questions, they look like this: 
Configuration file  /etc/bash.bashrc'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** bash.bashrc (Y/I/N/O/D/Z) [default=N] ?
In this specific example, if you answer Y or I (for yes or install ), dpkg will install the new version of /etc/bash.bashrc but it will also backup the current version in /etc/bash.bashrc.dpkg-old. If you answer N or O (for no or old ), dpkg will install the new version in /etc/bash.bashrc.dpkg-dist and /etc/bash.bashrc is left untouched. The two other answers allow you to examine the differences before taking a decision. Note that if you choose to start a shell, the new version is currently available as /etc/bash.bashrc.dpkg-new (and since Squeeze there are convenient environment variables $DPKG_CONFFILE_OLD and $DPKG_CONFFILE_NEW in case you want to create a custom review script). All configurations files managed by dpkg are called conffiles because that s the name of the field where they are recorded in the dpkg database. You can display the list of conffiles for any package: 
$ dpkg --status bash
[...]
Conffiles:
 /etc/skel/.profile ecb6d3479ac3823f1da7f314d871989b
 /etc/skel/.bashrc 2afdd6c53990f2387a7ef9989af0bc07
 /etc/skel/.bash_logout 22bfb8c1dd94b5f3813a2b25da67463f
 /etc/bash.bashrc 5b3c3bc73d236e4e1b6f9b6c1ed5964e
[...]
The command dpkg-query --showformat='$ Conffiles \n' --show bash can give you the same information if you need to retrieve only that field. The 32 characters after the filename are the MD5 checksum of the original configuration file provided by the package. Avoiding the conffile prompt Every time that dpkg must install a new conffile that you have modified (and a removed file is only a particular case of a modified file in dpkg s eyes), it will stop the upgrade and wait your answer. This can be particularly annoying for major upgrades. That s why you can give predefined answers to dpkg with the help of multiple --force-conf* options: If you use Apt, you can pass options to dpkg with a command-line like this: 
$ apt-get -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" dist-upgrade
You can also make those options permanent by creating /etc/apt/apt.conf.d/local: 
Dpkg::Options  
   "--force-confdef";
   "--force-confold";
Bringing up the conffile prompt at any time The conffile prompt is only displayed when dpkg detects that the package provides an new version of the conffile. Thus reinstalling the same package will not bring up the prompt. But you can instruct dpkg to ask nevertheless with the --force-confask option. This is a new feature in Debian Squeeze. It will only ask for files that are locally modified.
If you want to read more articles like this one, click here to subscribe to my free newsletter. You can also follow me on Identi.ca, Twitter and Facebook.
Flattr this Share/Bookmark 4 comments Support my work

20 September 2010

Steve Kemp: A new Blog::Spam release.

I've just released a new Blog::Spam module to CPAN. The Blog::Spam module is 99% of the code behind the blog & forum spam detection service. On the whole I'm pleased with the way that development has gone, and although I'm going to keep tinkering it is essentially "complete". Now I need to go back and fix bugs in chronicle, asql, and rinse. ObSubject: "You are just an ordinary man in a cape!" - Batman Begins.

17 September 2010

Jo Shields: Mono mythbusting, September 2010 edition

There are corners of the Internest where foolish people congregate, and invent stories. These foolish stories are then read as gospel by trusting people, and reposted, until the original made-up source is concealed from view. As an attempt to stem this flow of disinformation, here are some commonly held but incorrect beliefs about the Mono framework, and an explanation of the reality of the situation, as far as I understand it. The next Mono version is co-developed with Microsoft There is a grain of truth behind this one, but it s a gross mischaracterisation. Mono 2.8, when it ships, will bundle, for convenience, a number of Free Software libraries, which are released by Microsoft under a license considered Free Software by the Free Software Foundation, the Ms-PL. These are: So, to summarise, there are five Free Software libraries written by Microsoft under the Ms-PL included in Mono 2.8 but only two of those five are new, and none of them were co-developed in any meaningful sense. Mono development is dead There have been reports that Mono development has ended, on the basis that no commits have been made to Novell s Subversion server for a few weeks. However, these reports miss one minor detail Mono has moved to Github.com. There were 35 commits from 9 different people to the main mono.git repository (of dozens of repositories under the main Mono project) in the last 2 days, at time of writing far from dead. Mono lacks features found in Microsoft.NET Mono lacks libraries found in Microsoft.NET, such as the Windows Presentation Framework GUI toolkit. In terms of features, i.e. things implemented at a compiler or runtime level, Mono is typically more advanced. This is helped in no small part by Mono s Free Software development model, allowing experimentation and what if changes to the core runtime which a sluggish corporate behemoth like Microsoft cannot accommodate. To give three real-world examples, Mono allows embedding of its compiler as a service, and provides a REPL shell this is a planned feature for .NET 5.0, but has been available for years in Mono; Mono.Simd provides a number of data structures which will run on any version of Mono or Microsoft.NET, but will use optimized CPU extensions like SSE when run on a sufficiently new version of Mono, on an appropriate architecture as far as I m aware, there is nothing like this available or planned for Microsoft.NET. Mono is able to produce fully compiled, static executables which do not need to JIT anything at runtime this is used for iPhone compilation, for example, where JITters are not permitted. There is no comparable feature in Microsoft.NET. Clearly, Microsoft.NET can only be thought of as more featureful if one defines features in terms of does it have lots of libraries? in terms of functionality, Mono is ahead. Mono can sneak onto your system without your knowledge If you don t have the mono-runtime package installed, you can t run Mono apps. It is possible to install some Mono apps alongside the awful-yet-popular mononono equivs package, since the popular equivs script fails to place Conflicts on the correct packages (F-Spot will be blocked, Tomboy will not). No package in Debian or Ubuntu embeds its own copy of the Mono runtime, and we have no plans to make any changes to packaging which would allow execution of any C# application without mono-runtime installed. If one is using a different OS, then things may be different e.g. The Sims 3 for Mac & PC uses embedded Mono, which you wouldn t know about without looking. Canonical are pursuing a pro-Mono agenda, and are responsible for it being pushed in Debian Mono development has been happening in Debian for longer than Canonical has existed the first upload was made in April 2002. Ubuntu is made primarily from software already available in Debian deemed of sufficient quality, and when F-Spot and Tomboy became parts of the default Ubuntu desktop system in 2006, both pieces of software were already available in Debian and deemed of sufficient quality. Nobody working on Mono in Debian is paid by Canonical actually, that s not entirely true, three packages related to accessibility are officially maintained by someone who was hired by Canonical after doing the initial packaging work when he worked for Novell. But the Mono runtime itself, Canonical have no influence over its direction in Debian. As for a pro-Mono agenda , they ve always taken an extremely pragmatic approach to language choice, never showing any real preference for one language or another when it comes to app selection. They don t exhibit any overt anti-Mono policies, which is not the same thing. The names of people contributing to Mono in Debian are not secret check the pkg-mono team page on Alioth. The System.Windows.Forms namespace is protected by Microsoft patents The truth is, nobody knows for sure if SWF, or any other part of Mono, or any other framework such as Vala or Python, is covered by Microsoft patents. The way the US patent system is contrived, it is actively dangerous to check whether something contains any patents when you write it, as you are liable for triple damages should it later emerge that there WAS a patent, even if your searching missed the fact. You cannot patent an API or namespace only a specific implementation of a software concept, in those countries where software patents are permitted. There has never been any evidence shown that Mono s implementation of SWF, or indeed any part of Mono, infringes any Microsoft-held patents, because if that were the case, then the code would be rewritten to avoid the issue much like the approach taken by Linux kernel developers when a patent becomes apparent. The belief within the Mono community is that the core parts of Mono as defined in ECMA334/335 are safe (they are covered by the Microsoft Community Promise patent grant); any of the Ms-PL libraries mentioned above is definitely safe (Ms-PL includes an explicit patent grant); and the rest of the package is likely safe too (on the basis that it is a named component of the Open Innovation Network s list of protected software and on the basis that there s unlikely to be anything patentable in one of many implementations of basic ideas like database connectors). Nobody knows for sure, because that s how the system works. But, again, nobody knows for sure that Microsoft patents don t apply to other frameworks such as Python there is simply a belief and an assumption that they do not. MonoDevelop contributors removed GPL code from MonoDevelop, in an attack on the GPL This is somewhat disingenuous, given MonoDevelop is LGPL. The MonoDevelop team excised the remaining GPL code (and there wasn t much of it) in order to grant greater Freedom to developers. Previously, the entire MonoDevelop IDE was a GPL combination, meaning any add-ins for the IDE also needed to be GPL, regardless of developer choice. Now, any developer can write an add-in for MonoDevelop, using the license of their choice, whether another Free license like the Mozilla Public License, or a proprietary closed-source add-in. They are still welcome to produce GPL add-ins, if they want to, as well. Mono won t run random Microsoft.NET apps anyway, so isn t really cross-platform Actually, this one is often true. When developers write apps for Windows primarily, they rarely take the time to think is this the correct way to do it? and will often plough on with an assumption about the Windows way of doing things. It might be simple things like filesystem assumptions (assuming a case-insensitive filesystem, or assuming a backslash is used to separate directories, not a forward slash). It might be more involved, such as using P/Invokes into Windows-specific C libraries, when a more cross-platform alternative is possible. So, often, random applications for Microsoft.NET won t run on Mono. The reverse is also true F-Spot or GNOME Do are fairly heavily tied to Linux (or to UNIX-like OSes with X11, at any rate), through the libraries they invoke being fairly platform-specific. You can write platform-specific Java, with one quick piece of JNI, too. It should be noted, however, that Mono makes the chance of .NET applications being ported to Linux (and/or Mac) much more likely, since even in the worst case scenario, a company only needs to fix a portion of their source to make it cross-platform. The Mono team have a tool called MoMA, which will scan an application and its libraries, and give you detailed reports on the app s portability. This info is used at both ends by app vendors who want to become more portable, and by the Mono team who want to fill in the most frequently encountered blanks. And, it should be stressed, writing cross-platform apps is entirely possible if one desires it e.g. the IRC client Smuxi is pure cross-platform C#, and the executables compiled on Mono on Linux will run fine on Microsoft.NET on Windows. Portability in this direction is important too consider how many people have been introduced to Free Software thanks to availability of Free apps on Windows like OpenOffice.org, Firefox, Pidgin or GIMP. You can download Tomboy for Windows, and work is ongoing on fixing Banshee portability issues (which are mostly caused by gStreamer). Mono on Android is a terrible idea which will be super slow There are two efforts to enable developers to write Android applications with Mono a paid proprietary product from Novell called MonoDroid, and an untitled porting effort by big-name Android hacker Koushik Dutta. I have no insight into MonoDroid since it hasn t been released yet, but Koush did some benchmarks of his work which exhibited some remarkable figures compared to Dalvik (and even compared to Sun Java for ARM). Those numbers haven t been updated to reflect the Froyo Dalvik JITter, but Mono on Android is still very exciting from a performance perspective. Mono isn t really Free Software The source code is all available under Free Software licenses. So, um, yes it is. We don t need Mono because we have $LANG By the same logic, we don t need $LANG because we have Mono. By all means, use the language of your choice other developers will use the language of their choice too. If you want to use Vala or Python or Java, then by all means, go ahead. It doesn t mean there s no room for more languages, better suited to other usage patterns. Haskell is great for some types of development, so is Fortran, and so is C#. You might not need Mono because of $LANG, but there are others in the world with different needs. Mono apps are all slow, and crash your computer, and stuff No userland app can crash your Linux system, unless there s a bug in the kernel, or you have some severe problems with your hardware. If you ve ever observed a crash as a result of running a Mono app, then it s really coincidence that Mono is tickling whichever part of your system is busted. As for slow startup time may well be slower than for apps written in C or Vala. There s a delay due to the JITter needing to compile all the libraries used by the application (we may AOT the most common libraries in a future Mono package version, at upstream s recommendation). Once an app is running, it should be fast compared to a Python app, and memory-light compared to a Java app. The new garbage collector in Mono 2.8 should also offer significant improvements to performance, especially under heavy workloads. Ubuntu s default GTK+ themes require Mono This is my favourite recent nonsense to emerge from the Internest. There are reports mostly restricted to IRC and blog comments, that (paraphrasing here) removing Mono removes the Ubuntu themes . Here s the reality: in Ubuntu 10.04, a new visual style was implemented throughout the distro. As part of this, small icons (e.g. notification area icons) were set to be black-and-white by default, and colourised when attention is needed (e.g. the power-off icon turns red when a reboot is needed, the messaging indicator turns green when there s a new message). All of these new monochrome icons are in a package named ubuntu-mono . Removing the ubuntu-mono icon package will also remove the new Ubuntu GTK+ themes, in the light-themes package. So there s your explanation: the themes have a dependence on some monochrome icons, not on the Mono framework.

Matthew Garrett: USB runtime power management

I've just committed some patches to the rawhide (not F14) tree that re-enable USB autosuspend on some devices. This set includes a workaround in the bluetooth input code that should handle the case where people were seeing their input devices become laggy when autosuspend was enabled, but there's still some chance that other bluetooth devices will behave slightly oddly. If that's the case then try:

echo on >/sys/class/bluetooth/hci0/device/power/control

and see if it improves things. If so then please file a bug and include information about the device you're trying to connect to.

The other thing this patchset does is enable USB HCD runtime power management by default. This means that if you have a host controller with nothing connected, or with connected but suspended devices, the host controller will be powered down. It'll automatically power back up if a device is connected or a connected device sends a wakeup event. The obvious failure mode here is that your USB ports stop working. If that happens, please file a bug and include the output of lspci and dmidecode (you may want to edit the dmidecode output to make sure that your serial numbers are removed). In both cases, assign the bug to me (mjg at redhat.com).

Once this seems fairly stable I'm going to be looking at powering down more mostly unused PCI devices. SD readers and firewire controllers are typically unused and both support generating wakeup events, so they're the next step.

Rapha&#235;l Hertzog: How to create Debian packages with alternative compression methods

While gzip is the standard Unix tool when it comes to compression, there are other tools available and some of them are performing better than gzip in terms of compression ratio. This article will explain where you can make use of them in your Debian packaging work. In the source package A source package is composed of multiple files. The .dsc file is always uncompressed and it s fine since it s a small textual file. The upstream tarballs can be compressed with gzip (orig.tar.gz), bzip2 (orig.tar.bz2), lzma (orig.tar.lzma) or xz (orig.tar.xz), so choose the one that you want if upstream provides the tarball compressed with multiple tools. Put it at the right place and dpkg-source will automatically use it. Note however that packages using source format 1.0 are restricted to gzip, and the main Debian archive currently only allows gzip and bzip2 (xz might be allowed later) even if the source format 3.0 (quilt) supports all of them. The debian packaging files are provided either in a .diff.gz file for source format 1.0 (again only gzip is supported) or in a .debian.tar file for source format 3.0 (quilt) . The latter tarball can be compressed with the tool of your choice, you just have to tell dpkg-source which one to use (see below, note that gzip is the default). In a native package, dpkg-source must generate the main tarball and you can instruct it to use another tool than gzip with the --compression option. That option is usually put in debian/source/options: 
# Use bzip2 instead of gzip
compression = "bzip2"
compression-level = 9
For 3.0 (quilt) source packages, this option is not very useful as the debian tarball that gets compressed is usually not very large. But some maintainers like to use the same compression tool for the upstream tarball and the debian tarball, so you can use this option to harmonize both. In native packages, it s much more interesting: for instance the size of dpkg s source package has been reduced of 30% by switching to bzip2, saving 2Mb of disk. In the binary packages .deb files also contain compressed tar archives and by default they use gzip as well: 
$ ar t dpkg_1.15.9_i386.deb
debian-binary
control.tar.gz
data.tar.gz
data.tar.gz is the archive that contains all the files to be installed and it s the one that you can compress with another tool if you want. Again this is mostly interesting for (very) large packages where the size difference clearly justifies deviating from the default compression tool. Try it out and see how many megabytes you can shove. Another aspect that you must keep in mind is that those alternative tools might use important amount of memory to do their job, both for compression and decompression. So if your package is meant to be installed on embedded platforms, or if you want to build your package on low-end hardware with few memory, you might want to stick with gzip. Now how do you change the compression tool? Easy, dpkg-deb supports a -Z option, so you just have to pass -Zbzip2 for example. You can also pass -z6 for example to change the compression level to 6 (it s interesting because a lower compression level might require less memory depending on the tool used). The dpkg-deb invocation is typically hidden behind the call to dh_builddeb in your debian/rules so you have to replace that invocation with dh_builddeb -- -Zbzip2 . If you are using a debhelper 7 tiny rules files, you have to add an override like in this example: 
%:
	dh $@
override_dh_builddeb:
	dh_builddeb -- -Zbzip2
If you are using CDBS, you have to set the variable DEB_DH_BUILDDEB_ARGS: 
include /usr/share/cdbs/1/rules/debhelper.mk
[...]
DEB_DH_BUILDDEB_ARGS = -- -Zbzip2
I hope you found this article helpful. Follow me on identi.ca or on twitter.
Flattr this Share/Bookmark One comment Support my work

15 September 2010

Iustin Pop: Virtual hardware failures

Note: The title is intentionally unclear; it can be read both as virtual failures and failures of virtual hardware , and I'm not sure which one is the better for the cases below. One of the funny things about virtualization is how virtual hardware fails. It often happens that my NFS-stored virtual machine images suffer an NFS glitch or something, at which point one can see in the kernel logs: 
kernel: mptscsih: ioc0: attempting task abort! (sc=ffff880018320100)
kernel: sd 0:0:0:0: [sda] CDB: Read(10): 28 00 00 00 00 38 00 00 08 00
kernel: mptscsih: ioc0: task abort: FAILED (sc=ffff880018320100)
kernel: mptscsih: ioc0: attempting target reset! (sc=ffff880018320100)
kernel: sd 0:0:0:0: [sda] CDB: Read(10): 28 00 00 00 00 38 00 00 08 00
kernel: mptscsih: ioc0: target reset: SUCCESS (sc=ffff880018320100)
kernel: mptscsih: ioc0: attempting task abort! (sc=ffff880018320100)
kernel: sd 0:0:0:0: [sda] CDB: Read(10): 28 00 00 00 00 38 00 00 08 00
kernel: mptscsih: ioc0: task abort: FAILED (sc=ffff880018320100)
kernel: mptscsih: ioc0: attempting task abort! (sc=ffff88001eefcc00)
kernel: sd 0:0:0:0: [sda] CDB: Read(10): 28 00 00 98 01 a7 00 00 08 00
kernel: mptscsih: ioc0: task abort: FAILED (sc=ffff88001eefcc00)
kernel: mptscsih: ioc0: attempting target reset! (sc=ffff880018320100)
kernel: sd 0:0:0:0: [sda] CDB: Read(10): 28 00 00 00 00 38 00 00 08 00
kernel: mptscsih: ioc0: target reset: SUCCESS (sc=ffff880018320100)
While the NFS transient issues are probably a real bug, I have this image of a state machine inside the driver trying over and over the same error paths, in a futile attempt at fixing the hardware , and not locking onto/matching the real hardware behaviour. Or, this most recent case during boot and resource probing: 
kernel:  e1000: 0000:00:08.0: e1000_probe: The EEPROM Checksum Is Not Valid
kernel:  /*********************/
kernel:  Current EEPROM Checksum : 0x031a
kernel:  Calculated              : 0x031a
kernel:  Offset    Values
kernel:  ========  ======
kernel:  00000000: 08 00 27 f6 f3 9b 00 00 ff ff 00 00 00 00 00 00
kernel:  00000010: 00 00 00 00 08 44 1e 00 86 80 0e 10 86 80 40 30
kernel:  00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
kernel:  00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
kernel:  00000040: 00 00 61 70 0c 28 c8 00 c8 00 00 00 00 00 00 00
kernel:  00000050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 06
kernel:  00000060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
kernel:  00000070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 1a 03
kernel:  Include this output when contacting your support provider.
kernel:  This is not a software error! Something bad happened to your hardware or
kernel:  EEPROM image. Ignoring this problem could result in further problems,
kernel:  possibly loss of data, corruption or system hangs!
kernel:  The MAC Address will be reset to 00:00:00:00:00:00, which is invalid
kernel:  and requires you to set the proper MAC address manually before continuing
kernel:  to enable this network device.
kernel:  Please inspect the EEPROM dump and report the issue to your hardware vendor
kernel:  or Intel Customer Support.
kernel:  /*********************/
kernel:  e1000: 0000:00:08.0: e1000_probe: Invalid MAC Address
kernel:  e1000: 0000:00:08.0: e1000_probe: (PCI:33MHz:32-bit) 00:00:00:00:00:00
kernel:  e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
This is an emulated Intel NIC under VirtualBox. I have never seen it before, and it went away after a reboot. The most funny thing is the This is not a software error! message well, it definitely is :), but the driver has no way to know that. I think that these software-driven failures probably trigger some obscure error handling paths; I assume that the usual failures for real versus virtual hardware are quite different in nature, and that recovery is again different. This leads to using the wrong strategy for recovery, but fortunately nothing is really broken, just in the wrong state Since I'm not familiar with low-level driver details, I might be wrong of course; I'm just speculating on what I think goes on inside the driver. I'm not even mentioning virtual clock issues, as those are no longer funny

Matthew Garrett: Thoughts on upstreams

Last month I gave a presentation on the interaction between Android and kernel upstream at Linuxcon. The video for that is now available here (requires registration). Contrary to stories you may have heard, I do not dropkick anyone through a window.

There's some parallels between the Android/upstream scenario and Canonical's approach to upstream. Mark wrote a lengthy defence of Canonical's focus on components that they feel need development, while not putting development effort into things they feel are good enough already. That's pretty consistent with the discussions I had with him at the Ubuntu development meeting in Oxford over six years ago. Back then the focus was on taking all the excellent software that already existed and concentrating on providing it as a single polished and integrated product. It was successful - what's easy to forget now is that the first release of Ubuntu was massively more usable out of the box than any other Linux distribution available at the time, and it's absolutely undeniable that its release spurred increased efforts on the part of competitors. But I don't think the same focus is being applied any more.

The most obvious (and most controversial) example is the Ayatana project. Ayatana's a pretty explicit statement that Canonical don't view the existing Gnome UI as being suitable for their vision of the Linux desktop. That's absolutely fine. However, unlike many of the papercut projects, Ayatana is a set of complete reimplementations of functionality that behave differently to their upstream equivalents. There's no meaningful sense in which that's not a fork of Gnome. And, let me emphasise, I don't think that's a bad thing.

Let's go back to Android. While we talk about how much we'd have preferred it if Google had interacted with upstream before implementing wakelocks, the realistic outcome is that there's no way that we'd have accepted them even if they'd been posted years before hardware shipped. If there'd been a productive outcome to that aspect of the conversation it would have involved Android having a very different power management policy. Sometimes you're not going to convince people that something is better without implementing it and seeing how well it works. The problem with wakelocks is not that they exist or that Google feels they work better than the alternative, but that they end up integrated into drivers in a way that can cause divergence between mainline and Android.

Ayatana potentially has the same outcome. At some level, well-integrated applications have to be aware of the environment in which they're running. The application indicator patches are an example of that. Carrying conditional code increases maintenance burden, reduces test coverage and is a net loss in the long run. If upstream Gnome never gains support for this functionality, and if Ubuntu continues to depend on it, then Canonical have a delta that they'll have to carry forever. That benefits nobody.

Forking because you believe that your approach is better is a completely valid development model, but in the long run can cause problems if you don't have a long-term strategy for how to resolve that fork. For all we criticise Google's ability to get Android code into the mainline kernel, they've put orders of magnitude more effort into doing so than Canonical have in terms of getting Ayatana's code into mainline Gnome. This isn't a function of Google's relative size - the Android kernel team is on the order of 10 people. It's not enough of a difference to explain the disparity.

Canonical would be perceived as much better team players if there was an indication of their long-term plan in terms of Unity and the Ayatana projects and getting that code into mainline Gnome and integrating with the Gnome shell. It's completely unsurprising that they're viewed with distrust until that happens.

Michal &#268;iha&#345;: Czech phpMyAdmin translation complete, how is yours?

I finally made some progress on Czech translation of phpMyAdmin and it has again reached 100%. It was first time since merging GSoC projects, which introduced quite a lot of new strings. Not that they would be hard to translate, but it was rather hard to find time for that. However there are still lot of translations which are quite behind and would need some love. So if you can translate, you're welcome to join our translators and translate! PS: If you're looking for more work, we're also translating documentation :-).

14 September 2010

Rapha&#235;l Hertzog: How to customize dpkg-source s behaviour in your Debian source package

dpkg-source is the program that generates the Debian source package when a new package version is built. It offers many interesting command-line options but they are often not used because people don t know how to ensure that they are used every time the package is built. Let s fill that gap! It is possible to forward some options to dpkg-source by typing them on the dpkg-buildpackage command line but you d have to remember to type them every time. You could create a shell alias to avoid typing them but then you can t have different options for different packages. Not very practical. The proper solution has been implemented last year (in dpkg 1.15.5). It is now possible to put options in debian/source/options. Any long option (those starting with -- ) can be put in that file, one option per line with the leading -- stripped. Here s an example: 
# Bzip2 compression for debian.tar
compression = "bzip2"
compression-level = 7
# Do not generate diff for changes in config.(sub guess)
extend-diff-ignore = "(^ /)config.(sub guess)$"
Notice that spaces around the equal sign are possible contrary on the command line. You can use quotes around the value but it s not required. The debian/source/options file is part of the source package so if someone else grabs the resulting source package and rebuilds everything, they will use the options that you defined in that file. You can also use debian/source/local-options but this time the file will not be included in the resulting source package. This is interesting for options that you want to use when you build from the VCS (Version Control Repository, aka git/svn/bzr/etc.) but that people downloading the resulting source package should not have. Some options (like --unapply-patches) are only allowed in that file to ensure a consistent experience for users of source packages. You can learn more about the existing options in the dpkg-source manual page. Read it, I m sure you ll learn something. Did you know that you can tell dpkg-source to abort if you have upstream changes not managed by an existing patch in debian/patches? It s --abort-on-uptream-changes and it s only allowed in debian/source/local-options. Be sure to subscribe to the RSS feed or to the email newsletter to not miss useful documentation for Debian contributors! Flattr this Share/Bookmark 3 comments Support my work

Next.

Previous.