Search Results: "spa"

22 January 2011

Julien Viard de Galbert: Triage X Bugs of 2011 first Weeks (TXBW9-10-11)

I ve been lasy about the reporting lately, so this is a 3 weeks update at once ! The bug count according to udd is down to 818. I guess this is due to Kibi s triaging on input related bugs, but I helped to close a few one too ! I kept my week numbering as before, so now I know the first bugs I pinged are more that 11 weeks old. I guess if some submitter did not answer in that delay, they will probably never do it. I think the bug count will go down again ! ;) The X Strike Force (still) needs you ! You can have a look at the X Strike Force Bug Closing Procedure and check XSF unstable bugs sorted by date.

Russell Coker: Continuously Usable Testing of SE Linux

Joey has proposed a new concept of Continuously Usable Testing for Debian [1], basically testing should be usable at all times and packages that aren t usable should be dropped. But to properly achieve this goal we need continual testing of usability.The Plan For SE LinuxTo do this for SE Linux I m setting up a Xen server which will have a number of different DomUs for testing a variety of server applications. The system has 1.5G of RAM and 160G of mirrored storage. An image of a typical server will take about 4G of disk space, so we could have something like 40 images online and ready for testing. I have already setup Squid on another system on the same LAN to cache Debian packages, so running apt-get dist-upgrade on a number of DomUs won t take that long. With 256M for a typical server image I could have 5 images running at the same time. If the hardware isn t enough then I can expand it, I hope to get some donations of DDR-266 or DDR-333 RAM (or maybe DDR-400) to upgrade the system to 4G of RAM, I can add more hard drives, and I could even install more servers.I want to have testing be very usable for SE Linux throughout the development cycle so that I don t have to rush things before release.At this stage I m not sure whether to track Unstable or Testing for this. I guess it might be best to track Testing most of the time and only track Unstable for daemons that are changing rapidly. It might get boring testing every version that comes through Unstable, but if people want to do this then I won t stop them.Setting up the TestsWhat I need are interested people who want to install server configurations for testing. If you have some favorite combination of daemons that you want tested for SE Linux support (even if it s daemons that have no current policy) then I can give you root access to a DomU to develop test cases. Ideally there would be automated tests used for most things for example testing a mail server by using swaks to deliver mail and a POP or IMAP client script to retrieve it. But some things can t be tested properly without human intervention.For the automated tests I want to script the creation of DomUs, upgrading the packages in the DomU, testing it, and then shutting down the DomU if it all works. If at any time the tests fail (or the upgrade fails) then it would wait for human intervention. That would be me fixing SE Linux problems and other people fixing the application problems. I think that discovering SE Linux issues will only be a part of this project.For the manual tests I will grant access to create and destroy the DomUs in question to people who can run the tests.I m thinking of having a couple of DomUs running permanently for things which are test candidates but also useful for the project, such as a MediaWiki instance. It really depends on the interest of people who might use such things.Also I m thinking of setting up some Ubuntu DomUs too, I probably should join Ubuntu and get involved with SE Linux there.Sharing the ImagesI have a web server in Germany with almost unlimited bandwidth and storage. For every image that is created I want to upload a version to the server in Germany to allow anyone in the world to test it. There are lots of possible ways of using this for software development. For example if you had a patched version of Apache that you wanted to test then you could download every image that had Apache installed and test that they all work. That would be easier than configuring Apache in different ways and also possibly provide better coverage.Also if someone can t figure out how to configure a daemon correctly then downloading a Xen image of a working configuration could be helpful (if a little bandwidth intensive). Note that deploying such an image in production would be a really bad idea, among other things there are lots of places where passwords are stored and you wouldn t want to risk missing one.I also plan to share the scripts used in running the Dom0 and anything else that seems useful along the way.What We NeedThe main thing we need is volunteers to configure virtual machines with their favorite daemons. Note that I don t plan to have only one daemon per DomU, if we can get multiple daemons running that don t conflict (EG file server and mail server) or multiple daemons that can interact (EG database server and a mail server or anything else that can be a database client) running on the same system then that s a good thing. So there will be some degree of interaction with other people.I m happy to accept contributions from people who aren t interested in SE Linux. But SE Linux will run on all DomUs.Finally I also need more RAM for a HP D530S, DU875PA (that s a Celeron 2.4GHz). I ll accept donations of complete systems too once my HP system gets full, preferably relatively low power systems as they will be housed in a location that s not as well ventilated as I would like (cost and availability of IP addresses were the main criteria). A laptop with a broken screen would be great!The system won t go live until Monday, but I think that probably people won t be ready to do much work with less than two days notice anyway.

21 January 2011

Raphaël Hertzog: People behind Debian: Michael Vogt, synaptic and APT developer

Michael and his daughter Marie

Michael has been around for more than 10 years and has always contributed to the APT software family. He s the author of the first real graphical interface to APT synaptic. Since then he created software-center as part of his work for Ubuntu. Being the most experienced APT developer, he s naturally the coordinator of the APT team. Check out what he has to say about APT s possible evolutions. My questions are in bold, the rest is by Michael. Who are you? My name is Michael Vogt, I m married and have two little daughters. We live in Germany (near to Trier) and I work for Canonical as a software developer. I joined Debian as a developer in early 2000 and started to contribute to Ubuntu in 2004. What s your biggest achievement within Debian or Ubuntu? I can not decide on a single one so I will just be a bit verbose. From the very beginning I was interested in improving the package manager experience and the UI on top for our users. I m proud of the work I did with synaptic. It was one of the earliest UIs on top of apt. Because of my work on synaptic I got into apt development as well and fixed bugs there and added new features. I still do most of the uploads here, but nowadays David Kalnischkies is the most active developer. I also wrote a bunch of tools like gdebi, update-notifier, update-manager, unattended-upgrade and software-properties to make the update/install situation for the user easier to deal with. Most of the tools are written in python so I added a lot of improvements to python-apt along the way, including the initial high level apt interface and a bunch of missing low-level apt_pkg features. Julian Andres Klode made a big push in this area recently and thanks to his effort the bindings are fully complete now and have good documentation. My most recent project is software-center. Its aim is to provide a UI strongly targeted for end-users. The goal of this project is to make finding and installing software easy and beautiful. We have a fantastic collection of software to offer and software-center tries to present it well (including screenshots, instant search results and soon ratings&reviews). This builds on great foundations like aptdaemon by Sebastian Heinlein, screenshots.debian.net by Christoph Haas, ddtp.debian.org by Michael Bramer, apt-xapian-index by Enrico Zini and many others (this is what I love about free software, it usually adds , rarely takes away ). What are your plans for Debian Wheezy? For apt I would love to see a more plugable architecture for the acquire system. It would be nice to be able to make apt-get update (and the frontends that use this from libapt) be able to download additional data (like debtags or additional index file that contains more end-user targeted information). I also want to add some scripts so that apt (optionally) creates btrfs snapshots on upgrade and provide some easy way to rollback in case of problems. There is also some interesting work going on around making the apt problem resolver a more plugable part. This way we should be able to do much faster development. software-center will get ratings&reviews in the upstream branch, I really hope we can get that into Wheezy. If you could spend all your time on Debian, what would you work on? In that case I would start with a refactor of apt to make it more robust about ABI breaks. It would be possible to move much faster once this problem is solved (its not even hard, it just need to be done). Then I would add a more complete testsuite. Another important problem to tackle is to make maintainer scripts more declarative. I triaged a lot of upgrade bug reports (mostly in ubuntu though) and a lot of them are caused by maintainer script failures. Worse is that depending on the error its really hard for the user to solve the problem. There is also a lot of code duplication. Having a central place that contains well tested code to do these jobs would be more robust. Triggers help us a lot here already, but I think there is still more room for improvement. What s the biggest problem of Debian? That s a hard question :) I mostly like Debian the way it is. What frustrated me in the past were flamewars that could have been avoided. To me being respectful to each other is important, I don t like flames and insults because I like solving problems and fighting like this rarely helps that. The other attitude I don t like is to blame people and complain instead of trying to help and be positive (the difference between it sucks because it does not support $foo instead of it would be so helpful if we had $foo because it enables me to let me do $bar ). For a long time, I had the feeling you were mostly alone working on APT and were just ensuring that it keeps working. Did you also had this feeling and are things better nowadays ? I felt a bit alone sometimes :) That being said, there were great people like Eugene V. Lyubimkin and Otavio Salvador during my time who did do a lot of good work (especially at release crunch times) and helped me with the maintenance (but got interested in other area than apt later). And now we have the unstoppable David Kalnischkies and Julian Andres Klode. Apt is too big for a single person, so I m very happy that especially David is doing superb work on the day-to-day tasks and fixes (plus big project like multiarch and the important but not very thankful testsuite work). We talk about apt stuff almost daily, doing code reviews and discuss bugs. This makes the development process much more fun and healthy. Julian Andres Klode is doing interesting work around making the resolver more plugable and Christian Perrier is as tireless as always when it comes to the translations merging. I did a quick grep over the bzr log output (including all branch merges) and count around ~4300 total commits (including all revisions of branches merged). Of that there ~950 commits from me plus an additional ~500 merges. It was more than just ensuring that it keeps working but I can see where this feeling comes from as I was never very verbose. Apt also was never my only project, I am involved in other upstream work like synaptic or update-manager or python-apt etc). This naturally reduced the time available to hack on apt and spend time doing the important day-to-day bug triage, response to mailing list messages etc. One the python-apt side Julian Andres Klode did great work to improve the code and the documentation. It s a really nice interface and if you need to do anything related to packages and love python I encourage you to try it. Its as simple as: 
import apt
cache = apt.Cache()
cache["update-manager"].mark_install()
cache.commit()
Of course you can do much more with it (update-manager, software-center and lots of more tools use it). With pydoc apt you can get a good overview. The apt team always welcomes contributors. We have a mailing list and a irc channel and it s a great opportunity to solve real world problems. It does not matter if you want to help triage bugs or write documentation or write code, we welcome all contributors. You re also an Ubuntu developer employed by Canonical. Are you satisfied with the level of cooperation between both projects? What can we do to get Ubuntu to package new applications developed by Canonical directly in Debian? Again a tricky question :) When it comes to cooperation there is always room for improvement. I think (with my Canonical hat on) we do a lot better than we did in the past. And it s great to see the current DPL coming to Ubuntu events and talking about ways to improve the collaboration. One area that I feel that Debian would benefit is to be more positive about NMUs and shared source repositories (collab-maint and LowThresholdNmu are good steps here). The lower the cost is to push a patch/fix (e.g. via direct commit or upload) the more there will be. When it comes to getting packages into Debian I think the best solution is to have a person in Debian as a point of contact to help with that. Usually the amount of work is pretty small as the software will have a debian/* dir already with useful stuff in it. But it helps me a lot to have someone doing the Debian uploads, responding to the bugmail etc (even if the bugmail is just forwarded as upstream bugreports :) IMO it is a great opportunity especially for new packagers as they will not have to do a lot of packaging work to get those apps into Debian. This model works very well for me for e.g. gdebi (where Luca Falavigna is really helpful on the Debian side). Is there someone in Debian that you admire for his contributions? There are many people I admire. Probably too many to mention them all. I always find it hard to single out individual people because the project as a whole can be so proud of their achievements. The first name that comes to my mind is Jason Gunthorpe (the original apt author) who I ve never met. The next is Daniel Burrows who I met and was inspired by. David Kalnischkies is doing great work on apt. From contributing his first (small) patch to being able to virtually fix any problem and adding big features like multiarch support in about a year. Sebastian Heinlein for aptdaemon. Christian Perrier has always be one of my heroes because he cares so much about i18n. Christoph Haas for screenshots.debian.net, Michael Bramer for his work on debian translated package descriptions.
Thank you to Michael for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

4 comments Liked this article? Click here. My blog is Flattr-enabled.

Richard Darst: DebConf organization/volunteer meeting

As I just announced, the DebConf team will have an "organizational meeting". Every year, many teams need new people, since it is such a intense job for a few months. This year, I'm trying to get people who have not previously been involved in DebConf organization to help with these jobs.The meeting will be held on #debconf-team on irc.debian.org at 20:00 UTC on 25th January 2011. The main point of the meeting is to go through and fill out the teams list. This doesn't mean talking about how jobs should be done this year, just figuring out who is interested, sign a team up, and let the team start thinking about the details. The team can discuss details among themselves, and with the larger DebConf team on the lists and in future meetings.In past years, this meeting has helped to kickstart the teams and really began the DebConf cycle. This year, I am hoping to also make it a volunteer recruitment meeting. The DebConf team works hard, often to the point of burn-out, and thus would love new people. We need new people.In the next few days, I'll try to make more posts in my DebConf series to help make people familiar with the way DebConf works. You could consider this meeting to be the ultimate goal of the DebConf article series: helping to recruit new people at this meeting.Of course, if you want to join the DebConf team any other time, just drop by and offer a hand.

20 January 2011

John Goerzen: Research on deduplicating disk-based and cloud backups

Yesterday, I wrote about backing up to the cloud. I specifically was looking at cloud backup services. I ve been looking into various options there, but also various options for disk-based backups. I d like to have both onsite and offsite backups, so both types of backup are needed. Also, it is useful to think about how the two types of backups can be combined with minimal overhead. For the onsite backups, I d want to see:
  1. Preservation of ownership, permissions, etc.
  2. Preservation of symlinks and hardlinks
  3. Space-efficient representation of changes ideally binary deltas or block-level deduplication
  4. Ease of restoring
  5. Support for backing up Linux and Windows machines
Deduplicating Filesystems for Local Storage Although I initially thought of block-level deduplicating file systems as something to use for offsite backups, they could also make an excellent choice for onsite disk-based backups. rsync-based dedup backups One way to use them would be to simply rsync data to them each night. Since copies are essentially free, we could do (or use some optimized version of) cp -r current snapshot/2011-01-20 or some such to save off historic backups. Moreover, we d get dedup both across and within machines. And, many of these can use filesystem-level compression. The real upshot of this is that the entire history of the backups can be browsed as a mounted filesystem. It would be fast and easy to find files, especially when users call about that file that they deleted at some point in the past but they don t remember when, exactly what it was called, or exactly where it was stored. We can do a lot more with find and grep to locate these things than we could do with tools in Bacula (or any other backup program) restore console. Since it is a real mounted filesystem, we could also do fun things like make tarballs of it at will, zip parts up, scp them back to the file server, whatever. We could potentially even give users direct access to their files to restore things they need for themselves. The downside of this approach is that rsync can t store all the permissions unless it s running as root on the system. Wrappers such as rdup around rsync could help with that. Another downside is that there isn t a central scheduling/statistics service. We wouldn t want the backup system to be hammered by 20 servers trying to send it data at once. So there d be an element of rolling our own scripts, though not too bad. I d have preferred not to authorize a backup server with root-level access to dozens of machines, but may be inescapable in this instance. Bacula and dedup The other alternative I thought of system such as Bacula with disk-based volumes . A Bacula volume is normally a tape, but Bacula can just write them to disk files. This lets us use the powerful Bacula scheduling engine, logging service, pre-backup and post-backup jobs, etc. Normally this would be an egregious waste of disk space. Bacula, like most tape-heritage programs, will write out an entire new copy of a file if even one byte changes. I had thought that I could let block-level dedupe reduce the storage size of Bacula volumes, but after looking at the Bacula block format spec, this won t be possible as each block will have timestamps and such in it. The good things about this setup revolve around using the central Bacula director. We need only install bacula-fd on each server to be backed up, and it has a fairly limited set of things it can do. Bacula already has built-in support for defining simple or complicated retention policies. Its director will email us if there is a problem with anything. And its logs and catalog are already extensive and enable us to easily find out things such as how long backups take, how much space they consume, etc. And it backs up Windows machines intelligently and comprehensively in addition to POSIX ones. The downsides are, of course, that we don t have all the features we d get from having the entire history on the filesystem all at once, and far less efficient use of space. Not only that, but recovering from a disaster would require a more extensive bootstrapping process. A hybrid option may be possible: automatically unpacking bacula backups after they ve run onto the local filesystem. Dedupe should ensure this doesn t take additional space if the Bacula blocksize aligns with the filesystem blocksize. This is certainly not a given however. It may also make sense to use Bacula for Windows and rsync/rdup for Linux systems. This seems, however, rather wasteful and useless. Evaluation of deduplicating filesystems I set up and tested three deduplicating filesystems available for Linux: S3QL, SDFS, and zfs-fuse. I did not examine lessfs. I ran a similar set of tests for each:
  1. Copy /usr/bin into the fs with tar -cpf - /usr/bin tar -xvpf - -C /mnt/testfs
  2. Run commands to sync/flush the disk cache. Evaluate time and disk used at this point.
  3. Rerun the tar command, putting the contents into a slightly different path in the test filesystem. This should consume very little additional space since the files will have already been there. This will validate that dedupe works as expected, and provide a hint about its efficiency.
  4. Make a tarball of both directories from the dedup filesystem, writing it to /dev/zero (to test read performance)
I did not attempt to flush read caches during this, but I did flush write caches. The test system has 8GB RAM, 5GB of which was free or in use by a cache. The CPU is a Core2 6420 at 2.13GHz. The filesystems which created files atop an existing filesystem had ext4 mounted noatime beneath them. ZFS was mounted on an LVM LV. I also benchmarked native performance on ext4 as a baseline. The data set consists of 3232 files and 516MB. It contains hardlinks and symlinks. Here are my results. Please note the comments below as SDFS could not accurately complete the test.
Test ext4 S3QL SDFS zfs-fuse
First copy 1.59s 6m20s 2m2s 0m25s
Sync/Flush 8.0s 1m1s 0s 0s
Second copy+sync N/A 0m48s 1m48s 0m24s
Disk usage after 1st copy 516MB 156MB 791MB 201MB
Disk usage after 2nd copy N/A 157MB 823MB 208MB
Make tarball 0.2s 1m1s 2m22s 0m54s
Max RAM usage N/A 150MB 350MB 153MB
Compression none lzma none gzip-2
It should be mentioned that these tests pretty much ruled out SDFS. SDFS doesn t appear to support local compression, and it severely bloated the data store, which was much larger than the original data. Moreover, it permitted any user to create and modify files, even if the permissions bits said that the user couldn t. tar gave many errors unpacking symlinks onto the SDFS filesystem, and du -s on the result threw up errors as well. Besides that, I noted that find found 10 fewer files than in my source data. Between the huge memory consumption, the data integrity concerns, and inefficient disk storage, SDFS is out of the running for this project. S3QL is optimized for storage to S3, though it can also store its files locally or on an sftp server a nice touch. I suspect part of its performance problem stems from being designed for network backends, and using slow compression algorithms. S3QL worked fine, however, and produced no problems. Creating a checkpoint using s3qlcp (faster than cp since it doesn t have to read the data from the store) took 16s. zfs-fuse appears to be the most-used ZFS implementation on Linux at the moment. I set up a 2GB ZFS pool for this test, and set dedupe=on and compress=gzip-2. When I evaluated compression in the past, I hadn t looked at lzjb. I found a blog post comparing lzjb to the gzip options supported by zfs and wound up using gzip-2 for this test. ZFS really shone here. Compared to S3QL, it took 25s instead of over 6 minutes to copy the data over and took only 28% more space. I suspect that if I selected gzip -9 compression it would have been closer both in time and space to S3QL. But creating a ZFS snapshot was nearly instantaneous. Although ZFS-fuse probably doesn t have as many users as ZFS on Solaris, still it is available in Debian, and has a good backing behind it. I feel safer using it than I do using S3QL. So I think ZFS wins this comparison. I spent quite some time testing ZFS snapshots, which are instantaneous. (Incidentally, ZFS-fuse can t mount them directly as documented, so you create a clone of the snapshot and mount that.) They worked out as well as could be hoped. Due to dedupe, even deleting and recreating the entire content of the original filesystem resulted in less than 1MB additional storage used. I also tested creating multiple filesystems in the zpool, and confirmed that dedupe even works between filesystems. Incidentally wow, ZFS has a ton of awesome features. I see why you OpenSolaris people kept looking at us Linux folks with a sneer now. Only our project hasn t been killed by a new corporate overlord, so guess that maybe didn t work out so well for you <grin>. The Cloud Tie-In This discussion leaves another discussion: what to do about offsite backups? Assuming for the moment that I want to back them up over the Internet to some sort of cloud storage facility, there are about 3 options:
  1. Get an Amazon EC2 instance with EBS storage and rsync files to it. Perhaps run ZFS on that thing.
  2. Use a filesystem that can efficiently store data in S3 or Cloud Files (S3QL is the only contender here)
  3. Use a third-party backup product (JungleDisk appears to be the leading option)
There is something to be said for using a different tool for offsite backups if there is some tool-level issue, that could be helpful. One of the nice things about JungleDisk is that bandwidth is free, and disk is the same $0.15/GB-mo that RackSpace normally charges. JungleDisk also does block-level dedup, and has a central management interface. This all spells nice for us. The only remaining question would be whether to just use JungleDisk to back up the backup server, or to put it on each individual machine as well. If it just backs up the backup server, then administrative burdens are lower; we can back everything there up by default and just not worry about it. On the other hand, if there is a problem with our main backups, we could be really stuck. So I d say I m leaning towards ZFS plus some sort of rsync solution and JungleDisk for offsite. I had two people suggest CrashPlan Pro on my blog. It looks interesting, but is a very closed product which makes me nervous. I like using standard tools and formats gives me more peace of mind, control, and recovery options. CrashPlan Pro supports multiple destinations and says that they do cloud hosting, but don t list pricing anywhere. So I ll probably not mess with it. I m still very interested in what comments people may have on all this. Let me know!

Simon Josefsson: Introducing the OATH Toolkit

I am happy to announce a project that I have been working quietly on for about a year: the OATH Toolkit. OATH stands for Open AuTHentication and is an organization that specify standards around authentication. That is a pretty broad focus, but practically it has translated into work on specifying standards around deploying and using electronic token based user authentication such as the YubiKey. YubiKey OATH s most visible specification has been the HOTP algorithm which is a way to generate event-based one-time passwords from a shared secret using HMAC-SHA1. HOTP has been published through the IETF as RFC 4226. Built on top of HOTP is the time-based variant called TOTP, which requires a clock in the token. OATH do some other work too, like specifying a data format for transferring the token configuration data (e.g., serial number and shared secret) called PSKC.

The aim of my project OATH Toolkit is to provide an implementation of various OATH related technologies. I m intentionally leaving it open ended because you never know what they may specify that I find interesting. However, the primary goal has been to focus on HOTP and TOTP. Throughout 2010, the project was called HOTP Toolkit but that name made it difficult to support TOTP in a non-confusing way. During the last month, after discussion with Daniel Pocock on Dynalogin which is a potential consumer of my package, I took the time to create a fork of the HOTP Toolkit and the OATH Toolkit was born. Now what does the OATH Toolkit actually do? Primarily it provides a library called liboath that implements HOTP and TOTP. Liboath is a relatively small library, and my goal is to keep it well documented and of high quality. There is GTK-DOC generated API documentation. Of course there is a command line tool to go with it, called oathtool which makes working with HOTP/TOTP from the command line easier. It can generate and validate one-time passwords. Let s say you want to generate the first four OTP based on the dummy key 1234.
jas@latte:~$ oathtool -w4 1234
376439
299783
041392
819202
158134
jas@latte:~$
By default the tool is using HOTP, but you can switch it into TOTP mode with the totp parameter. The output OTP will now depend on the current time on your machine, unless you specify the time manually using the now parameter.
jas@latte:~$ oathtool now= 2011-01-20 15:46 UTC totp 1234
527971
jas@latte:~$
The tool can do more, check the oathtool man page for all the details. The final component of the OATH Toolkit is a PAM module pam_oath. With it, you can login to your machine using an OTP and optionally a password. Right now the user and password management is simplistic, but that should improve over time. To setup single-factor authentication for su you would create a file containing the user information and HOTP key as /etc/users.oath like this:
HOTP root 1234
Then configure PAM to use the pam_oath module like this in /etc/pam.d/su:
auth requisite pam_oath.so debug usersfile=/etc/users.oath window=20
The user file will be rewritten every time you su to hold the current state. There is a README for the PAM-module with more documentation. That s it for an intro! From the OATH Toolkit webpage we link to binary packages for Debian and Ubuntu so please try the OATH Toolkit yourself and provide feedback to the oath-toolkit-help mailing list.

Richard Darst: The DebConf talk selection and scheduling process

The DebConf talk selection process is in charge of selecting the talks at DebConf. Some of the reasons for having a talks team are to help ensure a high quality of talks, maintain a somewhat academic feeling for the conference, and to let attendees know what talks can be expected to be good.Even though there is a big talk selection process in advance, DebConf has always allowed attendees to schedule new talks during the conference - at DebConf10, these were called "ad-hoc" talks. At DC10, attendees could sign up for a room+time slot on a wiki page, and each night they would be added to the main schedule. In past years, this distinction had been called "official" (pre-selected) vs "non-official" (new ideas during DebConf) talks, which was extremely confusing to people. "Ad-hoc" talks are a very important part of DebConf: not all development meetings are pre-planned.Since we pretty much always surplus of talk rooms and times and we let people schedule talks during DebConf, people have asked: what is the purpose of talk selection? Are we going to let all talks be given? Are we going to reject talks just to say we selected them? When talks were "rejected" in the past, the speakers were told to just ask for a room during DebConf if they still wanted to give it. This question was raised on our mailing list. Even if there is a surplus of talk rooms, there are reasons for talk selection: it allows attendees to know what is most prepared, helps the video team to capture the best talks, lets talks be scheduled in the best room. You could also say it helps to organize talks into tracks and to collect abstracts for proceedings.For DebConf10, we introduced a "conference track" concept, a set of related talks which would be scheduled in the same room in sequence. This allowed a more structured conference for attendees, and perhaps recruiting speakers to fill in requested talks for the tracks. Example tracks from DebConf10 are Science, Enterprise, and Community Outreach. I would have liked to see some tracks more Debian related, such as: project infrastructure, project communication, project social issues, ..., but what we had was a good start that can be improved on, if the new team desires to.As for scheduling, there are two main parts: one phase before DebConf, assigning the selected talks to rooms and making the advance schedule. The other phase happens during DebConf and handles assigning rooms to people who want to do "ad-hoc" talks. The two phases aren't always done by the same person. The schedulers need to balance the expected audience and room capacity, need for video coverage, BOF vs auditorium style seating, not making talks overlap that are likely to appeal to the same audience, ... . The scheduling team works with the talk team, but doesn't necessarily have to be the same people.I'm very happy that there was good discussion about the selection and scheduling process for DebConf10. In the end, a talk selection and scheduling wiki page was created. This page had two main purposes. First, it discussed and documented the selection and scheduling process, so that it was transparent to people that year and provides a reference for future teams. Secondly, other people have added their proposed talks workflows, that they can be considered in future years.
Talk selection has many competing demands - because of the completeness of the wiki page, I'll just refer you there. The rest of this post mainly frames the types of questions the talks team has to face. If you have thoughts on the DebConf talk selection process, feel free to add your ideas to the talk selection process wiki page, or better yet, help out with the team this year.

19 January 2011

Eddy Petri&#537;or: Picking up the pieces

Update: After trying KiBi's suggestion to take advantage of this information, I looked for more info on the issue and found this conversation. A git upgrade to the backported git 1:1.7.1-1.1~bpo50+1 version, and git svn rebase started pulling the right stuff in. Yes, I left the svn-remote.svn.rewriteRoot stuff set to the old value and the svn-remote.svn.url to the new value.

Keywords:

Unable to determine upstream SVN information from working tree history

when running git svn reabase


As I said yesterday, I am going to come back to being active in Debian.

I remember looking a little at my page on the Debian Wiki and it was clear that it was stale. (I sometimes find it amusing how I presented myself my name is Eddy Petrisor .) For some reason now the wiki seems to be down.

Two days ago I tried to get the Wormux/Warmux upstream source but git svn appears not to like the rename although I even modified the .git/config and .git/svn/.metadata, but it still wasn't satisfied and answered in a bad mood with this message (after a long waiting period):

$ git svn rebase
Unable to determine upstream SVN information from working tree history


And, yes, I am aware of the compromise and I changed my password for the project on gna.org.

I guess is better if I try to take a look at the RC bugs for the moment.

Robert Millan: Finally, amd32 is taking shape

When AMD launched its now widely used 64-bit architecture in year 2000, it started marketing it as a significant step forward because of it having longer word and pointer size. One just has to check the advertising material and notice those big 64 being touted as the main improvement. But it is commonly accepted that most applications don t need a 64-bit address space for anything. Building them in LP64 model is just a waste of memory due to increase of pointer size. Even though this made most applications lag behind, the new architecture still was an improvement in terms of speed because of the AMD revised ISA, featuring changes like: To summarize, most of the merit from AMD64 architecture was in fixing some of the insanity of Intel 386 ISA (instruction set architecture), rather than the increased pointer size which was a source of inefficiency most of the time. Perhaps AMD didn t evaluate this correctly, or perhaps its marketing side won over technical merit. However, although the new hardware is biased towards LP64 data model, it s not actually enforced. It was a matter of time until an independent project took over and attempted to fix this, combining the AMD64 ISA with ILP32 data model. I ve been reading with much interest in the binutils and gcc mailing lists that such project is beginning to take shape. A port of binutils, GCC, GDB and Linux is already available. Future plans include porting Glibc which will make it possible to build a standard GNU derivative out of this. Sadly, I don t have the time to devote to this project myself, but I ll continue following its progress. I m looking forward to bringing this speed boost to my machines.

Evgeni Golov: funny spam

yes, I do collect funny spam ;)Today I will present you some spammy funny comments I got on my (wordpress-powered) blog in the last months.
  1. 2010/09/24 at 03:08
    I cant believe, Facebook is currently down with a DNS failure. I guess Facebook having some issues. Businesses are reporting a near impossible 480% increase in productivity
  2. 2010/10/25 at 11:07
    Not quite on topic BUT really important: Please guys, donate something for Haiti! I just came back from a trip down there and I have to say the situation is really terrible! It s soon christmas time, so please be so kind and do something good! Thanks
  3. 2010/10/27 at 11:07
    Hey, I can t view your site properly within Opera, I actually hope you look into fixing this.
  4. 2010/11/18 at 12:26
    Are you watch Sarah Palin s TV show? I saw the trailer & wtf? -__- She s like, This is so much better than being in politics. It s like she s doing this just because she is losser. o.O What do you think? Do you believe she can be the next american presiden? ..
  5. 2010/11/30 at 15:40
    Wende im Proze gegen J rg Kachelmann. Die des Publikumslieblings haben ihr Mandat niederlegt. Der Proze in Mannheim soll dennoch planm ig fortgesetzt werden. Es werde keinen Antrag auf Aussetzung des Prozesses geben, f gte Rechtsanwalt Ralf Hoecker hinzu. Der neue Strafverteidiger Johann Schwenn (Hamburg) und die seit Verfahrensbeginn anwesende Pflichtverteidigerin Andrea Comb werden top vorbereitet sein, sagte er weiter.
  6. 2010/12/09 at 17:31
    How I can download documents from WikiLeaks?
    Thanks
  7. 2010/12/10 at 10:09
    Hi, you should check out http://www.voteonwikileaks.com. It s sort of like a crowdsourced collection of arguments against Wikileaks. Considering you re a blogger, i think you d find it to be an interesting read
  8. 2010/12/16 at 18:38
    It appears as though Julian Assange will at least be out on bail any minute but what about Bradley Manning? Solitary confinement for seven months so far without being convicted of anything, without a trial, even. That s bad!
  9. 2010/12/24 at 09:03
    Gibt es ein Jailbreakme f r Nerdomaten? Gen gend Bugs sollte es im IE6 geben :D
  10. 2010/12/29 at 13:22
    Der hammer bei den dingern ist aber eigentlich, dass sie nicht mit win7 laufen sondern mit: XP!!!!!!! Das ist doch der Hammer oder? hab auch ein Beweisvideo vom Hochfahren des Teils
  11. 2010/12/30 at 05:03
    Widespread criticism of RealVNC vulnerability is fixed
    A security report claimed that RealVNC software virtual network a high-risk vulnerability could allow a malicious attacker does not need a password to login to a remote system
  12. 2011/01/05 at 18:10
    is it true that it is not possible to use that sky thing with verizon?
Sorry, some are in German ;)What s so funny about them you ask? Well, the do contain actual content (even if not always matching to the posts they were attached to), they look relevant to the time they were posted and still are spam: the links in the homepage field of the comment led to spamsites. Numbers 9 and 10 even funnier: they are exact copies of already present comments to the same post.And oh, I think WikiLeaks should leak a cable about downloading stuff from WikiLeaks ;)

Nikita Youshchenko: Trinity project: KDE3 alive?

Trinity project in an attempt to keep KDE3 desktop compatible with ongoing changes elsewhere, and to some extent to fix bugs and add features into KDE3. It is for those us long-time KDE users who is upset by the way that KDE has chosen for KDE4. Unfortunately Trinity project looks small and underpowered, so it s future is unclear. Did not try it yet. There are debs, although some people think that it is hard to deal with those, because of KDE3 package namespace now used by KDE4, and also because of binary compatibility issues within Trinity. Why I don t like KDE4? Because I have no interest in eye candy, but due to heavy use of thin clients with apps running on remote server I do care about desktop resource consumption. A medium-class server (2 dual-core 2 GHz cpus, 16G ram) perfectly hosts 50 KDE3 sessions, served over vnc. Anything near to that with KDE4? I m not aware of any good-working kde4-over-vnc-over-network configuration, even for a single user over unloaded network. Looks like within KDE4, even with all effects turned off, every tiny operation results into near-fullscreen update. Spice is promising but requires per-session VMs which is a serious drawback when number of users is high.

Dirk Eddelbuettel: RcppBDT 0.1.0

The family of Rcpp packages just grew by one: the first 0.1.0 release of RcppBDT is now on CRAN. RcppBDT stands for Rcpp Boost Date_Time. It employs what we call Rcpp modules: a mechanism which provides easier ways to expose C++ functions and classes to R (and which bears some resemblance to Boost.Python---see this vignette about Rcpp modules for more details). And thus RcppBDT provides R users with access to (some) Boost Date.Time functionality. I used Boost Date.Time because it is widely known, useful and well tested. It can also be used in pure template mode not requiring linking (if one foregoes string parsing and formatting which is fine here as we get this from R already; it can be added via a #define in the header file). With that, the package is rather portable as it only needs to find the Boost headers. (And no, I have not yet added a configure option for that.) Basic usage follows Rcpp modules and provides the C++ class via a Reference Class in R. That means using a new constructor and assessing functions via the $ operator. Here is an example: 
R> library(RcppBDT)
Creating a new generic function for "print" in "RcppBDT"
Creating a new generic function for "format" in "RcppBDT"
R> d <- new(bdtMod$date, 2011, 1, 18)
R> format(d)
[1] "2011-01-18"
R>
Not very exiting yet: we create a date, using explicit year, months and date arguments and then format it. Something more useful follows: 
R> d$getDayOfWeek()
[1] 2
R> d$getModJulian()
[1] 55579
R> d$getFirstOfNextMonth()
[1] "2011-02-01"
R>
We can use accessors to extracts parts of the date, or use functions to convert the date to different representations such as a modified Julian date. Moreover, given a date, we can apply helper functions such as get me the date of the beginning of the next month. More interesting still are helper functions such as the ones below. Note that this also shows the alternate access method using wrapper functions I added for the package, this may be more familiar to most R users: 
R> getNthDayOfWeek(third, Wed, Dec, 2010)
[1] "2010-12-15"
R> getLastDayOfWeekInMonth(Sat, Dec, 2010)
[1] "2010-12-25"
R> getFirstDayOfWeekInMonth(Sat, Dec, 2010)
[1] "2010-12-04"
R> getFirstDayOfWeekAfter(Wed, as.Date("2010-12-31"))
[1] "2011-01-05"
R>
This uses some of the constants Jan, Feb, ... , Dec; Sun, Mon, ..., Sat and first ... fifth defined in the package; normal integers can also be used. The third Wednesday in a month is also known as the IMM Date to Finance-heads; the example was borrowed from Whit's earlier rboostdatetime code on github. More examples are in the demo available with the package and accessible via demo(RcppBDT) once you install the package. As always, feedback would be welcome, both on the usefulness (or lack thereof) of the Boost Date.Time functionality as well as on the Rcpp modules wrapping. Additional Boost Date.Time functionality (durations, times, ...) may be added; contributions would be welcome. The rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

18 January 2011

Russell Coker: Conferences and People on the Autism Spectrum

There have been some recent discussions about issues related to the treatment of women at Free Software conferences, I ve written posts about Aspie Social Skills and Free Software [1] and Empathy, Autism, and Geeks [2] about this. But of course discussion continued on how Aspies supposedly cause problems that no-one seems to be noticing.Lisa of Chaotic Idealism wrote an interesting post about the NT social bias [3]. In summary NTs seem to think that Autism Spectrum Disorders are only about socialisation, not realising that for many people on the Spectrum it s sensory issues that are their main problem. Social problems are exacerbated by sensory issues and other causes of stress, so I think it s worth considering ways in which conferences can be planned to be less stressful for people on the Autism Spectrum, people who have SPD [4] that isn t associated with an ASD, and NTs who just get annoyed by loud noises etc.What Autism Conferences DoAutscape is one of only two conferences for people on the Autism Spectrum and the only one which clearly documents how they plan their conference [5]. The first noteworthy thing that they do is have badge colors to indicate what level of social interaction is desired by each delegate, I don t think that this is relevant to Free Software conferences as people who don t have sufficient social skills to suit at least a green badge probably won t be attending. But I think that when attending a conference about Free Software or any other equally geeky topic (if there is an equally geeky topic) it s worth keeping in mind the fact that there are probably a lot of people who would like to talk to you but lack the social skills to start a conversation.
No initiationRedPlease do not initiate any interaction with me.
Prior PermissionYellowPlease do not initiate unless I have already given you permission to approach me on a yellow badge.
Please initiateGreenI would like to socialise, but I have difficulty initiating. Please initiate with me.
NeutralWhite (or no badge)I am able to regulate my own interaction.
The next thing that they document is a black circle badge which indicates that the wearer shouldn t be photographed. Prior to reading that web site I wasn t aware of this being an Autistic issue, I was only aware of it being an issue for women who don t want zoomed-in pictures of themselves appearing on guys web sites (even pictures that aren t up-skirt or down-blouse can be unwelcome). A conference policy that prohibited photographs that zoom in on one person without that person s consent (or parental consent in the case of minors) would probably be a good idea.Another thing about badges is that it s a really good idea to have the delegate s name on both sides of the badge if the badge is attached in a way that permits it to turn. People on the Autism Spectrum tend to have some difficulty in recognising people and in remembering names. I find it inconvenient when someone expects me to recognise them but has their badge turned around so I can t see their name. Some people get really unhappy if they think that someone doesn t recognise them.Autscape has long breaks between activities and a leisure session each afternoon to allow delegates to recover from the stress of dealing with people. It seems to me that computer conferences in some cases could do with longer breaks between sessions. I find that a lot of the benefit of a conference is in what happens outside sessions and the standard practice of publishing videos of presentations makes personal meetings a more important part of the conference.Autscape has designated quiet spaces. I think that for computer conferences which have hack-labs (which seems to be most Free Software conferences nowadays even if they aren t labelled as such) it would be good to have some lab areas designated as quiet zones. I think that it is a really good thing to meet people you ve only known by email and then play some LAN based games against them and that this should be encouraged as part of a conference, but having that sort of thing separated from people who want to do some quiet coding is a really good idea. Whether people want to do quiet coding, read email, or just escape from the stress of a conference a quiet hack-lab would be a good place for it.An issue that s related to quiet spaces is the ability to escape from social situations. One of my pet hates is corporate meetings on boats, when the boat is in motion there is no escape. Corporate meetings that are only accessible by coach are also bad. Fortunately most conferences aren t like that.The Autscape web site states that they prohibit people from wearing perfume or aftershave to help people who are sensitive to smell. My observation of Free Software conferences is that encouraging everyone to have a shower every morning would be a good idea as there are stronger smells from unwashed people than from perfume.One thing that s interesting about the Autscape web site is that they have different color schemes available and have tested it in multiple web browsers including Safari and Konqueror (which apparently don t work so well). It s interesting to note that they test with such a variety of browsers including free software ones most corporations don t do that.Sensory Processing IssuesThe main SPD issues related to conferences seem to be noise and light related.For social events one difficulty that most people on the Spectrum seem to face is in listening to one person in a noisy crowded room. As well as that sudden noises and loud background noises can be very stressful. While it s sometimes impossible to avoid crowds (which can be another problem) it is often possible to select venues that have less noise problems. A venue with carpet on the floor and soft walls (anything other than brick or concrete) will be a lot quieter than one with hard surfaces that reflect noise. It seems that a quiet venue will benefit NTs as well, there seems to be a strong correlation between the price of food at restaurants and the amount of sound absorbing material on the floor and walls people who can afford a good dining experience seem to want it quiet.In terms of visual issues the relevant problems seem to be related to sudden transitions and lecture halls that are extremely dark. The only thing that can be done by conference organisers is to seek to have the lights in the lecture halls as bright as possible without preventing the viewing of the projector screen, this is usually done anyway.For social events the whole binge drinking at crowded bars thing doesn t work too well due to noise, poor lighting, crowds, and the smell of vomit. But there s no shortage of reasons to discourage binge drinking at conferences.FoodPsychology Today has a good article about preferences for food and picky eaters [6]. Lots of people on the Autism Spectrum have similar issues. The thing to do when arranging a conference meal is to have things neat and without needless combinations. Think about making food look more like what you would expect to see in a Japanese restaurant and less like Paella. Also having some very plain food on offer is a good thing, I think that bread-rolls for dinner and ice-cream for desert makes a viable meal. But any dish with a word like hash or mixed in it s name isn t a good option. Finally some combinations are really bad, I always find rare steak and potatoes on the same plate to be rather disgusting potato that s blood-stained doesn t appeal. The majority of conference meals satisfy these criteria.Autscape provides options of eating outside the main dining hall for people who can t tolerate the noise. For a Free Software conference it might be a better idea to provide seating outside the main area for people who are going to make noise. I don t mind people who want to get really drunk at the conference dinner, but I would prefer them to be in a separate room. Also when assisting a drunk friend to leave the venue before they cause more problems it would be convenient if there was a good place to take them to. The one occasion when I had to strongly encourage a friend to leave a conference dinner to reduce his embarrassment the next day (and reduce the annoyance for everyone else) there was no good place to take him so he just came back!ConclusionI don t expect that anyone will make any significant changes to conference plans after reading this post. I have tried to focus on things that would benefit NTs as well, but there are practical issues that get in the way such as the cost and availability of venues that are large enough. But there are some smaller ideas that can be implemented with relative ease.I hope that the people who are making claims about Autism and conference behavior will refer to this in future. If you think that there is a problem with the way people on the Spectrum act at conferences then the solution will more likely involve the suggestions I make here than anything else.

John Goerzen: Backing Up to the Cloud

I m recently taking some big-picture looks at how we do things, and one thing that I think could be useful would be for us to back up a limited set of data to an offsite location states away. Prices are cheap enough for this to make it useful. Services such as Amazon S3 and Rackspace Cloud Files (I ve heard particularly good things about that one) seem to be perfect for this. I m not quite finding software that does what I want, though. Here are my general criteria:
  1. Storage fees of $0.15 per gigabyte-month or less
  2. Free or cheap ($0.20 per gigabyte or less) bandwidth fees
  3. rsync-like protocol to avoid having to re-send those 20GB files that have 20MB of changes in their entirety every night
  4. Open Source and cross-platform (Linux, Windows, Mac, Solaris ideally; Linux and Windows at a minimum)
  5. Compression and encryption
  6. Easy way to restore the entire backup set or individual files
  7. Versatile include/exclude rules
  8. Must be runnable from scripts, cron, etc. without a GUI
  9. Nice to have: block or file-level de-duplication
  10. Nice to have: support for accurately backing up POSIX (user, group, permission bits, symlink, hard links, sparse files) and Windows filesystem attributes
  11. Nice to have: a point-and-click interface for the non-Unix folks to use to restore Windows files and routine restore requests
So far, here s what I ve found. I should note that not a single one of these solutions appears to handle hard links or sparse files correctly, meaning I can t rely on them for complete system-level backups. That doesn t mean they re useless I could still use them to back up critical user data just less useful. Of the Free Software solutions, Duplicity is a leading contender. It has built-in support for Amazon S3 and Rackspace Cloud Files storage. It uses rdiff, which is a standalone implementation of the rsync binary delta algorithm. So you send up a full backup, then binary deltas from that for incrementals. That makes it bandwidth-efficient for incremental backups, and storage-efficient. However, periodic full backups will have to be run, which will make it less bandwidth-efficient. (Perhaps not incredibly *often*, but they will still be needed.) Duplicity doesn t offer block-level de-deuplication or a GUI for the point-and-click folks. But it DOES offer the most Unixy approach and feels like a decent match for the task overall. The other service relying on Free Software is rsync.net, which supports rsync, sftp, scp, etc. protocols directly. That would be great, as it could preserve hard links and be compatible with any number of rsync-based backup systems. The downside is that it s expensive really expensive. Their cheapest rate is $0.32 per GB-month and that s only realized if you store more than 2TB with them. The base rate is $0.80 per GB-month. They promise premium support and such, but I just don t think I can justify that for what is, essentially, secondary backup. On the non-Open Source side, there s JungleDisk, which has a Server Edition that looks like a good fit. The files are stored on either S3 or Rackspace, and it seems to be a very slick and full-featured solution. The client, however, is proprietary though it does seem to offer a non-GUI command-line interface. They claim to offer block-level de-duplication which could be very nice. The other nice thing is that the server management is centralized, which presumably lets you easily automate things like not running more than one backup at a time in order to not monopolize an Internet link. This can, of course, be managed with something like duplicity with appropriate ssh jobs kicked off from appropriate places, but it would be easier if the agent just handled it automatically. What are people s thoughts about this sort of thing?

Stefano Zacchiroli: cross-distribution meeting on application installer

saluting the cross-distro app installer meeting I just want to wish luck to the cross-distribution meeting on applications installers, which is starting today. For some background, check out Vincent's nice blog post on the subject. As far as my memory goes, this is one of the first attempts if not the first one to get representative of as many distros as possible around the same table to design, hack, and discuss a common topic of interest. We don't know yet how it will turn out, but the approach to the meeting has been a commendable one: aim at as much diversity as possible (first gathering people informally to get some ballpark dates, then with an announcement sent to the most appropriate place), be transparent about the organization, and engage in keeping others informed ex post. As they say here in France, chapeau for this attempt at factual cross-distro collaboration. Debian is present at the meeting, represented by Enrico Zini and David Kalnischkies. Thanks to them (for being there), to Vincent Untz (for the organization), to Novell/SUSE (for sponsoring part of Debian attendees' expenses), and to Debian donors (whose donations have been used to sponsor the rest).

Benjamin Drung: Poll: How to call the library?

Dear lazy web, I am writing a library that implements the Units Policy of Ubuntu. This library can be used to format sizes (of files, disks, memory, etc.) for displaying them and the other way around. For example, the size 12345 bytes can be formatted to 12.3 kB, 12.1 KiB, or 12.1 KB depending on the configuration. An other function will give you the possibility to convert 12.3 kB, 12.1 KiB, or 12.1 KB back to 12345 bytes. The preferred binary unit can be configured by an text file (per system and per user) and by an environment variable. The library is written in C and doesn t have any dependency. Therefore it can be used by any program on any desktop environment. Binding to other languages like C++, Python, Java are planned. Before creating a project for this library, I need a name for the library. The current prototype is called libdisplayunits, but this name reflect only the half of the project. It does not indicate that the library can be used for inputting sizes. What do you think? What is the best name for the library? Please poll and/or comment below.

<script charset="utf-8" language="javascript" src="http://static.polldaddy.com/p/3972978.js" type="text/javascript"></script> <noscript> View This Poll
survey software </noscript>

17 January 2011

Joey Hess: the conversion

It's summer in Spain, and outside there's a dry heat, but in the hack lab, it's frigid. I'm sitting across the table from Otavio, and each of our laptops is showing the same identical screen of text, editing the same file. Slowly. The file is located in Colorado, and we're fighting serious conference lag, plus transatlantic lag. Type a few lines, wait a minute, hope it shows up, and restart the converter to see if it worked. We're struggling with figuring out how to make it work, and pair programming across the table like this helps -- although we also struggle with each others accents sometimes, so we're eventually mostly communicating via text on the computers. Eventually Otavio rolls off to bed. Later, everyone else in the conference goes to bed or whatever -- I don't really notice. At some point a cleaning woman comes in, and seems suprised to see me. I muster up enough Spanish to ask if this muy fio air conditioning can be turned off. A bit later, I make this commit and then go out, look at storks in early morning light, and go to bed. 
r59731   joeyh   2009-07-24 12:30:52 -0400 (Fri, 24 Jul 2009)   1 line
up to r15000; also skip the packages/po accient revs
That would have been 6 am local time. The log shows us back at it by 11 am local. There's so much detail in these logs, and it's hard to tell how any individual bit will turn out useful, but the value in aggregate is undeniable and that's why we struggle to retain it as technology marches on.
A few days later and the conversion is caught up to the commits documenting it. Everyone goes home. Later still, and work is done to verify the conversion. Problems are found, fixed. The worst of them involves the Danish language, and absolute evil. Tempers flare, and I have the last argument I'll ever have the pleasure of having with Frans. At some point in there, I do rather more coding in C++ than I ever have before, or ever wanted to. Frans's objections are addressed by that, more or less. We're using a weird tool that was developed as a kind of one-off for another project, but happens to meet our needs pretty well -- but we're stuck dealing with its bugs. And its need for copious handholding and historical research. The file we started out editing grows to two thousand lines, and now covers ten years of odd bits of history.
Suddenly it's a year and a half later. I'm in a cabin in winter, Otavio is down in summer, and I'm once again dealing with lag (dialup lag this time). We run the converter again -- one last time. And a time more.. and four or five more "last times". Finally, after a longish day, it's done.
This was not a major project, just some little bits of time here and there, maybe a week's worth in total, spread out over a few years and four or five people. It hardly seems worth writing about, just another codebase converted from Subversion to Git, a bit behind most of the other ones. Just thought I'd give you a glimpse behind the curtain.

Julien Viard de Galbert: Triaging bugs is not that hard !

There is a big thread on -devel that started about Forwarding bugs upstream, the idea is that it should be the maintainer s job. While it s true in general, some understaffed teams are so flooded by bugs that they just can t do it.
I ve seen more than once the X Strike Force asking a user to report upstream (pointing to the relevant BTS) and come back with the tracking URL to be attached to the bug. This was mostly for driver issue if I remember well.
And most users did it, so thanks! Then the thread evolved in lots of directions, I m not following all of it, but there was the question of the bug triaging and as I m doing it for the X packages, I want to share about my experience here. The available triaging docs The debian wiki already have a general Bug Triage page.
The thread pointed to the KDE Bug triaging page.
The X Strike Force has some pages, one on the wiki and a new one more specific to bug triaging on the alioth project s doc.
And when it comes to closing bugs with version information, there is a good read on the wiki: Bugs Version Tracking.
Finally don t hesitate to reopen the BTS documentation whenever you need to write to control@b.d.o ! Now if all those docs don t help you, then try to improve the docs as a first step ! ;) Talk with the maintainer(s) Well I did met Julien Cristau and Cyril Brulebois in person during mini-DebConf Paris/2010 that certainly helped. But we exchanged several mails, or met on IRC channel #debian-x.
But the triaging rules of the X Strike Force (linked above) were pretty clear anyway. My current process This process has evolved over time with the response of several contributors either to help or to fix to my mistakes, thanks!
  1. List bug from UDD: XSF unstable bugs sorted by date.
  2. Pick an old bug (once you answer and UDD runs, the bug moves down the list, so there are always old bugs on the top).
  3. Read the report, don t be afraid by the subject, it might be unrelated, the bug might be merged with other bugs, then reading the other bugs can help.
  4. If relevant try to reproduce it. (most bugs in X are hardware related or specific to one setup, so it s not that often that I have something to test).
  5. Decide what to do, most of the time, I will ping the submitter.
  6. I m a mutt user, and to my help the bts command as a way to open the bug mbox in mutt, that makes it really easy to answer the bug.
    So I launch bts -m show $n where $n is the bug number, and I can then answer the bug from mutt.
    The triaging procedure suggest to ping -submitter, but using mutt it s easier to just reply and add a -quiet to the bug number (this avoid spamming the debian-x with all my pings).
  7. In the message, always try to be kind and helpful, most of the time ask to test newer versions.
  8. Keep track of the pinged bugs, well I do it for my blog but it will be useful anyway, see below.
  9. When the submitter answers, just react accordingly, if the bug can be closed, then close it ! ;)
  10. If after some time, the bug are still not answered, consider closing them (I haven t started that yet, but this will probably get the bug number down faster:) ).
Note: Sometime I don t send the bug to -quiet so that other debian-x subscriber can have a look at the bug. They then can give me advices, or close the bug directly with a stronger reason than any I could have found. ;) To submitters, please keep the bug CCed when you answer, thanks. For the skeptics Yes as a newbie I did some mistakes, but I was always corrected in a nice way, with explanations. Also I try to have a conservative approach, I don t close bugs if I m not sure, I ask if I can close them, so it might take longer, but in the end the effect is the same for the bug, but not the same to the submitter ! ;) Also this work might look boring, but it s actually rewarding, most of the answers I received included a cheerful message thanking me for following up on the bug. Not to mention all I ve learned on Xorg. Conclusion As long as I have time for it, I ll continue to triage bugs and I d love to see others coming to help me on X or probably better go and help some other teams that also need help, and flood the planet with bug triaging reports ! ;) Get involved !

Dirk Eddelbuettel: Keeping simple things simple

My friend Jeff deserves a sincere congratulation for finally unveiling his rebranded R consultancy Lemnica. One notable feature of the new website is a section called esoteric R which discusses less frequently-visited corners of the R world. It even boasts its own CRAN package esotericR with the example sources. esoteric R currently holds two articles. Jeff had sent me the one about introducing closures a while back, and I like it and may comment at another time. What caught me by surprise when Lemnica finally opened was the other article: R calling C. It is a fine article motivated by all the usual reasons that are e.g. mentioned in the Google Tech Talk which Romain and I gave last October about our work around Rcpp. But it is just not simple. Allow me to explain. When Jeff showed this C language file 
#include <R.h>
#include <Rinternals.h>
SEXP esoteric_rev (SEXP x)  
  SEXP res;
  int i, r, P=0;
  PROTECT(res = allocVector(REALSXP, length(x))); P++;
  for(i=length(x), r=0; i>0; i--, r++)  
     REAL(res)[r] = REAL(x)[i-1];
   
  copyMostAttrib(x, res);
  UNPROTECT(P);
  return res;
and then needs several paragraphs to explain what is going on, what is needed to compile and then how to load it --- I simply could not resist. Almost immediately, I emailed back to him something as simple as this using both our Rcpp package as well as the wonderful inline package by Oleg which Romain and I more or less adopted: 
library(inline)  ## for cxxfunction()
src <- 'Rcpp::NumericVector x = Rcpp::NumericVector(xs);
        std::reverse(x.begin(), x.end());
        return(x);'
fun <- cxxfunction(signature(xs="numeric"), body=src, plugin="Rcpp")
fun( seq(0, 1, 0.1) )
Here we load inline, and then define a three-line C++ program using facilities from our Rcpp package. All we need to revert a vector is to first access its R object in C++ by instantiating the R vector as a NumericVector. These C++ classes then provide iterators which are compatible with the Standard Template Library (STL). So we simply call the STL function reverse pointing the beginning and end of the vector, and are done! Rcpp then allows us the return the C++ vector which it turns into an R vector. Efficient in-place reversal, just like Jeff had motivated, in three lines. Best of all, we can execute this from within R itself: 
R> library(inline)  ## for cxxfunction()
R> src <- 'Rcpp::NumericVector x = Rcpp::NumericVector(xs);
+         std::reverse(x.begin(), x.end());
+         return(x);'
R> fun <- cxxfunction(signature(xs="numeric"), body=src, plugin="Rcpp")
R> fun( seq(0, 1, 0.1) )
 [1] 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0
R>
Lastly, Jeff shows a more complete example wherein a new vector is created, and any potential attributes are copied as well. Naturally, we can do that too. First, we used clone() to make a deep copy (ie forcing creation of a new object rather than a mere proxy) and use the same R API function he accessed---but it our case both prefixed with ::Rf_ for R remapping (to protect clashed with other functions with identical names) and a global namespace identifier (as it is a global C function from R). 
R> library(inline)
R> src <- 'Rcpp::NumericVector x = Rcpp::clone<Rcpp::NumericVector>(xs);
+         std::reverse(x.begin(), x.end());
+         ::Rf_copyMostAttrib(xs, x);
+         return(x);'
R> fun <- cxxfunction(signature(xs="numeric"), body=src, plugin="Rcpp")
R> obj <- structure(seq(0, 1, 0.1), obligatory="hello, world!")
R> fun(obj)
 [1] 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0
attr(,"obligatory")
[1] "hello, world!"
R> obj
 [1] 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
attr(,"obligatory")
[1] "hello, world!"
R>
Both the obj variable and the new copy contain the desired data attribute, the new copy is reversed, the original is untouched---and all in four lines of C++ called via one inline call. I have now been going on for over one hundred lines yet I never had to mention memory management, pointers, PROTECT or other components of the R API for C. Hopefully, this short writeup provided an idea of why Romain and I think Rcpp is the way to go for creating C/C++ functions for extending and enhancing R.

Rapha&#235;l Hertzog: Review of my Debian related goals for 2010

Last year I shared my Debian related goals for 2010 . I announced that I would not be able to complete them all and indeed I have not, but I have still done more than what I expected. Let s have a look. Translate my Debian book into English and get it published: NOT DONE Or rather not yet done. It s still an important project of mine and I will do it this year. When I wrote this last year, I expected to find a publisher that would take care of everything but we failed to find a suitable one so we re going to do it ourselves. Cleanup the dpkg perl API and create libdpkg-perl: DONE libdpkg-perl has been introduced with dpkg 1.15.6. Create dpkg-buildflags: DONE dpkg-buildflags has been introduced with dpkg 1.15.7. It s not widely used yet and it won t really be until debhelper 7 supports it (see #544844). It would be nice to see progress on this front this year. Ensure the new source formats continue to gain acceptance: DONE The adoption rate has been steady, it clearly slowed down since the freeze though. I have implemented quite a few features to satisfy the needs of users, like the possibility to unapply the patches after the build, or the possibility to fail in case of unwanted upstream changes. Design a generic vcs-buildpackage infrastructure to be integrated in dpkg-dev: NOT DONE I believe it s something important on the long term but it never made it to my short-term TODO list and it s unlikely to change any time soon. Continue fixing dpkg bugs faster than they are reported: PARTLY DONE We have dealt with many bugs over the year, but we still have 20 more bugs than at the start of last year (370 vs 350). We re not doing bad compared to many other Debian teams but we can still benefit from some help. Start here if you re interested. Enhance our infrastructure to ease interaction between contributors and to have a better view of how each package is maintained: NOT DONE I am convinced that we need something to have a clearer idea of the commitments made by each contributor. I don t put the same amount of care in maintaining smarty-gettext that I do on dpkg. If we had a database of the stuff that we know we don t do well enough, it s easier to point new contributors towards those. Anyway, this project is still unlikely to come to the top of my priorities any time soon. Work on the developers-reference: NOT DONE We have switched the Maintainer field to debian-policy@lists.debian.org to have more review of the changes suggested through the bug tracking system but that has not changed much on the global situation. I still hope to become more active on it sometimes this year. Maybe by trying to make it more fun and creating the text for some of the wishlist bugs as blog articles first. Rewrite in C the last Perl scripts provided by the dpkg binary package: DONE Dpkg 1.15.8 was the first version working without Perl, I announced it in July. Integrate the 3-way merge tool for Debian changelogs in dpkg-dev: DONE dpkg-mergechangelogs is part of dpkg-dev since 1.15.7. I enjoy it regularly. Unfortunately it doesn t work well for cherry-picks. Would be nice to see this fixed, anyone up to the task? :-)
Click here to subscribe to my free newsletter and get my monthly analysis on what s going on in Debian and Ubuntu. Or just follow along via the RSS feed, Identi.ca, Twitter or Facebook.

One comment Liked this article? Click here. My blog is Flattr-enabled.

Next.

Previous.