Search Results: "zed"

7 February 2011

Alexander Reichle-Schmehl: What a night! What a day!

Wow, that was a night. I'm still quite tired from the weekend. Someone told me, that ~ 1 000 people subscribed to our live denting via identi.ca, and about 12 000 via twitter. And that does not even include those, who followed us via the respective web interfaces! While it was quite exhausting (we tried to watch about 7 irc channels for the important stuff, I partly had ~80 windows in my irssi (which is especially bad, since I only have keyboard shortcuts for the first 40) and there whas also a constant need to watch for mails and due to translations also our spamfilters), it was also very nice to think that somewhere there are release parties going on with people looking at our dents, maybe even having them running on a big monitor or projector ;) One thing that remains to be done, are the left-over dents we prepared as time killers: Since the release process went quite smoothly (and we didn't have the time to check all our dents for the limit of 140 chars ;) we still have some of them left. Since some of them where requested (and I poked other people to get me the numbers) here are the remaining dents: I especially like the one about debian-curiosa ;) Update 1: The 600 USD where collected during DebConf5, not DebConf3. Update 2: Small clarification about the number of subscriptions on identi.ca and twitter: They didn't followed a live-stream channel, but they do follow our general channel. We are not sure, how many people where subribed before we announced the live denting; some saw 700 subscribers on identi.ca, which would mean ~200 seem to have subscribed for the release dents (still not including the ones just following us the old fashioned way via a browser).

6 February 2011

Christian Perrier: Squeeze released

This is the gazillionth entry on Planet Debian about Squeeze being released. If I had to say one thing about it, it would be that I think the way it was handled this time has been really great. The debian-publicity team, lead by the tireless Reichles, did great job in doing as many buzz as possible around that release....but managed to do it in the "spirit" of free software (pushing release parties, animating a feed on identi.ca, preparing press releases, etc.). The Release Team did a tremendous job during the 6 months of the freeze, by carefully balancing between conservatism and liberalism in accepting (or denying) our gazillion of request for freeze exceptions The FTP, CD, installer, website, documentation teams organized things really well for their respective fields of responsibility. A special mention to the tremendous effort done about the release notes, release announcements documents (and of course, their translators). I'm currently busy upgrading my servers (at home and at work) and things are going smoothly (except the first two I did without carefully reading warnings about missing firmwares...they're now waiting for me to provide them with the right non free firmware packages, 30km away from here). And, for myself, I celebrated the release this morning by squashing my personal best time on an half-marathon (officials races excepted) by nearly 7 minutes, bringing it just below 1h45 (PB in official races is 1h37). Release power! :) And, during that afternoon, I also had the please of having my pet package (geneweb) and my most important package (samba) having their first release in unstable, targeted for wheezy. Squeeeeeeze meeeeeeeeeeeee!

Martin Michlmayr: Upgrading to Debian 6.0 (squeeze) on ARM

Debian 6.0 (squeeze) has been released. Here are some notes if you're running Debian on an ARM-based NAS device or plug computer and are planning to upgrade. First of all, make sure to read the release notes for Debian 6.0 on ARM. This document contains a lot of information on performing a successful upgrade. Second, during the kernel upgrade, you'll get the following message about the boot loader configuration:
The boot loader configuration for this system was not recognized. These
settings in the configuration may need to be updated:
 * The root device ID passed as a kernel parameter;
 * The boot device ID used to install and update the boot loader.
On ARM-based NAS devices and plug computers, you can simply ignore this warning. We put the root device into the ramdisk so it will be updated automatically. Finally, after doing the upgrade and before rebooting your system, make sure to run flash-kernel to activate the new kernel.

Benjamin Mako Hill: Editor-to-Reader Ratios on Wikipedia

It's been reported for some time now that the number of active editors on Wikipedia (usually defined as people who have edited at least 5 times in a given month) peaked in 2007 and has been mostly stable since then. A graph of the total number of active editors in every month since Wikipedia's founding is shown below. The graph shows the aggregate numbers for all language Wikipedias. English Wikipedia is the largest component of this and is generally more variable. That said, very similar patterns exist for most larger languages. /copyrighteous/images/active_editors_all_wikpiedia.png Felipe Ortega, who has provided many of these statistics, has warned against fatalist claims. Although there seems to be a decrease in the total number of active editors over the last two years, the situation seems to have somewhat stabilized in most languages. New editors in Wikipedia are replacing folks at almost the rate that they are leaving. It is also widely known that the number of readers of Wikipedia has been increasing during this period. According to the report cards released by the foundation using comScore data, the number of unique visitors to Wikipedia each month has increased by 61 million people in the last year -- over 17%. This discrepancy between rising readership and stable or sinking editorship should raise major concern. After all, the Wikimedia Foundation's mission is two part: (1) to empower and engage people around the world to collect and develop educational content and (2) to disseminate it effectively and globally. Although the Foundation report cards include measures of raw levels of editorship, a better metric of engagement and empowerment might be the proportion of readers who engage in editing. I could not find reliable data on the number of unique readers reached each month for more than a few months in the last year. What is available, however, is wonderful data on page views each month going back to 2008. Analysis of the data from available report cards show that, at least during the last year, there is a very stable ratio of 35 page views per unique visitor, as estimated by comScore. Using that measure, we can do a back of the envelope estimation on the proportion of users that are editors for the period where page view data is available, dating back from February 2008 ( marked with the grey dashed lined above). The graphs below show the very different results you get when you consider the change in the number of Wikipedia editors and the change in the editor-to-reader ratio. Once again, these data are combined data for all language Wikipedias although graphs look very similar for most larger languages. The results are striking. Although there has been a 12% decrease between February 2008 and December 2010 in the number of active editors, there is a 42% decrease in the proportion of readers who edit at least five times a month. We can see that fulfilling the first half of the Wikimedia mission remains a struggle. /copyrighteous/images/wikipedia_editors_combined_graph.png Although the graphs above do not say anything directly about the most active core contributors to Wikipedia, the fact that Wikipedia is being maintained by a tiny -- and shrinking -- proportion of its readership does mean that the idea behind Wikipedia is under threat. Although none are as big as Wikipedia, there are lots of good encyclopedias out there. The reason Wikipedia is different, interesting, and important is because -- unlike all those others -- Wikipedia is the encyclopedia that anyone can edit. Wikipedia is powerful because it allow its users to transcend their role as consumers of the information they use to understand the world. Wikipedia allows users to define the reference works that define their understanding of the their environment and each other. But 99.98% of the time, readers do not transcend that role. I think that's a problem. Worse, that the number is growing. The Wikimedia Foundation recently ran a major successful effort to attract donations in its annual fundraiser. Jimmy Wales' smiling mug is apparently enough in the way of motivation to get something like 1% of its readers to donate money to support the project. I think that's very good news. If the Wikimedia community can entice even half of those people to contribute through an increased involvement in the projects themselves, they might do more than ensure Wikipedia's continued growth. They would help take a step toward the empowerment and engagement of those users in sharing their own knowledge, and the continued fulfillment of a critically important mission.

Charles Plessy: Long life to Squeeze !

Debian just published Squeeze today. It is an excellent release, that I hope I will keep for a long time on my machines. Among the improvements, boot has been accelerated. On my laptop with a second hand SSD, it only takes 20 seconds to see the login screen. I was amazed when I saw this for the first time. I was often upgrading from Stable to Testing quite quickly, to install more recent versions of desktop applications. But the backports change the game. I hope that I will not switch to Testing before one year or two. For Squeeze, I will also prepare backports for the packages I maintain. I hope that more people will enjoy them. Most of these packages are independant from the graphical system, and more in general from its core. The backports will allow to use recent version of these packages on top of a stable distribution. Since Lenny, Debian Med displays the contents of its metapackages with tasks pages, that include draft packages as well. Not all will make it in Debian, and I would like to have the feedback from our users to better chose our priorities. Contact us on the debian-med@lists.debian.org mailing list ! These new packages will also be distributed for Squeeze through the backports. Thanks again to everybody who contributed to the release of Squeeze !

3 February 2011

Joey Hess: so long second thoughts

So Debian 6.0 "Squeeze" is due to be released this weekend. I've been doing my part annoying the release team with various risky last minute changes for not-formally-RC bugs. The best of these was making kFreeBSD CD images be actually used by d-i. Which had been silently ignoring them (so nobody noticed), and then I "broke" it to at least complain that it couldn't use them, and is now entirely fixed. Anyhow, there are these Debian release parties planned in 68 locations worldwide, this weekend. I have never been to a Debian release party. The timing is never right. The nearest one is 6 hours away, so I won't be making the trip. But yesterday I had a crazy idea, and I seem to have done enough that I can't back out now. announcing the Debian Party Line mumble.debian.net is a mumble server. Mumble is a nice system for doing group voice chats. I will be hosting a kind of virtual release party there, on February 5th and 6th. We'll follow the progress of the release, maybe have some scheduled events (if someone wants to do a talk or tutorial session). And at the end we'll hear what it sounds like when we all get Squeezed. So, should be fun, assuming my servers don't melt down due to bandwidth use. Hope to chat with you there! Please visit its page for details and to RSVP. squeeze countdown

Raphaël Hertzog: People behind Debian: Mike Hommey, Firefox/Iceweasel maintainer

Mike Hommey, our Iceweasel maintainer

For as long as I have known Mike, he s been maintaining one of the largest (and most widely used) package: Iceweasel, Debian s default web browser. It s effectively Mozilla s Firefox although it has been renamed to avoid some rules enforced by Mozilla s trademark policy. But we might see Firefox back in Debian read on to know Mike s plans. His story is also very interesting in that he went from simple user to Iceweasel maintainer, and now he s working for Mozilla Corporation. But you already knew that contributing to free software is a good way to grow skills and gain experience that are valuable on the job market, right? ;-) My questions are in bold, the rest is by Mike. Who are you? I am Mike Hommey. I am French, even if my name doesn t sound so. I have been a GNU/Linux user since 1995, using Debian since 1999, and started packaging in 2003. After a long time in the NM queue, I became a DD in 2005. The main reason I started being actively involved is quite selfish: I wanted some things done, and I just did them. My first packages were extensions for the Mozilla Suite, the old one that directly inherited from Netscape Communicator. Later on, I wanted some of these extensions to work with the new kid on the block: Phoenix; or maybe it was already Firebird. By then, integrating extensions was a hassle, and the mozilla-browser maintainer had come up with a script that would make it easier for extension packages. So I ported that script for use with Phoenix/Firebird and sent it to the BTS. Then Phoenix/Firebird became Firefox, and it grew a real extension manager. Except that is was entirely not suited for extensions installed by the packaging system. So I made it work better. And made a few more changes to make Firefox work better. All through patches or NMUs. Then I became co-maintainer, then Firefox couldn t keep its name and became Iceweasel, and, fast-forward until today, I m now taking care of Iceweasel/Xulrunner and Iceape in stable, unstable, experimental and an external repository. This 8 years-long story (wow, I hadn t even realized it had been that long until I wrote it) tells you something: newcomers shouldn t be afraid to stick their hands anywhere, even in big packages. I was only a Mozilla and Phoenix end-user when I started. Apart from Mozilla-related packaging, I also happen to (co-)maintain some other packages in Debian, most notably webkit, though I haven t been very active in the pkg-webkit team recently. There are several reasons for that, the main one being that there are other team members getting the job done. And I can t say that much about the pkg-mozilla team, since there are effectively no other team members. What s your biggest achievement within Debian? I am quite proud that Squeeze will be the first Debian release where Iceweasel works on all supported architectures. Previously, we only knew the package compiled on all these architectures. Starting with Squeeze, we know it passes several of the test suites coming with the source code. This ensures most of the browser is functional. It was actually far from being the case before, as it appeared when the test suites were first enabled. I think it is very important to have properly working packages on all our supported architectures. The current evolution of hardware shows how important it still is. Who would have thought a few years back that we would see ARM or MIPS based netbooks today ? What are your plans for Debian Wheezy? For Debian Wheezy, I would like to improve Iceweasel/Icedove/Iceape integration with the packaging system. Something that would allow to use the apt database to find extensions or plugins, on top of the current behaviour. I also want to improve desktop integration, most notably for KDE. Another thing I d like to see happen is to package google breakpad independently (its source code is currently embedded in chromium and iceweasel, though not built, as far as I know), and have our Mozilla products builds be able to send their crash reports upstream. It is very important that upstream gets direct visibility over what crashes on GNU/Linux systems. It helps making them aware of issues that would otherwise probably remain unnoticed, as there are much more users using distributions packages than upstream tarballs. And not all Debian users report crashes to the BTS. I know at least Fedora and SuSE are already doing it. More importantly, I want to avoid the sad situation we got with Squeeze, where we re going to release a 18 months old Iceweasel. We got there for a combination of reasons, one being that I was preparing for a freeze in December 2009 and focusing on getting 3.5 in shape instead of the then upcoming 3.6. The result was that we didn t get 3.6 packages before upstream released 3.6.3 in April 2010. By then it was too late, considering the uncertainty of the release schedule, to update reverse dependencies and get 3.6 ready for Squeeze. Things are different for 4.0. I m already getting ahead of things, and while reverse dependencies haven t been taken care of, all beta releases have been made available through http://mozilla.debian.net/, usually within hours after upstream release (sometimes even a few hours before official announcement, but always after availability on ftp.mozilla.org). A great share of our patches were also pushed upstream. Several things made that possible: There will be a couple major upstream releases until Wheezy comes along, and I want to make them available early in unstable, even if that means breaking reverse dependencies until they are fixed. Finally, I would like to get other people involved, starting with the Icedove maintainer. Volunteers are very much welcome, for any kind of involvement: bug triaging, documentation (which we are deeply lacking), testing, code People interested can contact the pkg-mozilla team list. As the Debian packaging gets less time consuming, I should be able to spend more time mentoring, which should help. If you could spend all your time on Debian, what would you work on? I actually have been working almost full time on Debian for a few months during a more-or-less purposeful unemployment period. That s when I attempted to add an alert popup when Iceweasel or extensions are upgraded (which was eventually removed because not working very effectively). That s when I worked on Iceweasel 3.6 and the first 4.0 betas. That s when I pushed more than 70 patches upstream, and got upstream commit access. I know for a fact that keeping up with upstream and answering bug reports is already time consuming, and time comes in finite quantity. Working full time on Debian allows to do more than that, as I ve been able to experience first hand. Unfortunately, that doesn t help pay the bills, so it would be best if time could be shared with other persons. Like, actual members in the pkg-mozilla team. ;) Anyways, if I were to work full time on Debian again, I d like to do more than packaging, and help improve some of our infrastructure, most notably the buildd network, to grow some automated way to get packages built on all or some of our architectures. Call that PPA if you want, though I don t think we need something exactly like PPA. We sure do have porter boxes, but there is a huge overhead to get packages built there: you need to first try, then see you lack build dependencies, then wait for admins to install them, then come back and launch your build, and finally come back again much later to see how it went. And that s when it goes smoothly. Repeat for each architecture. It s already cumbersome when it s a small package, so imagine when it s Iceweasel. I d like to be able to push a (signed) .dsc somewhere, say I want it built on that and that architecture, and get binary packages and logs back. It could then be up to developers to distribute them, or not. Sometimes, you re just interested in test suite results. Another related wish would be a way to access build trees of failed builds, possibly on the buildd where it failed. What s the biggest problem of Debian? Lack of contributors, lack of contributors, and lack of contributors. It can sound weird to say that in a community of 1000+ people. But the sad reality is that we ve seen, repeatedly, calls for help in various places from various understaffed teams (most often of effectively one member) for years. So there is obviously a problem getting people to invest time in these teams either on the long term, or at all. I know this is not a very significant metric, but it already should say a lot: I wouldn t be surprised if more than 2/3 of the number of source lines of code in Debian come from less than 5% of the number of packages in the archive, and see less than 5% of the DD/DM population involved. If someone feels like doing the actual math, I m curious to know how far I m from reality, and in what direction. We also lack manpower to be able to make our release development smoother. In a perfect world where manpower is not a scarce resource, we should work on $release+1 when $release is being frozen. Said otherwise, we should have been able to start working on Wheezy in August 2010, or even earlier. You ve been working for Mozilla Corporation for a few months. How does the work look like and what are you working on? I am actually not working full time for MoCo; I am only contracting for 4 days a week. That was a request I made to keep having enough spare time for Debian or other activities. I am working from home, and am pretty much free about when I work, as long as I work long enough. I take advantage of this freedom and the saved commuting time to live a healthier life by doing some sports in the daytime (which also happens to be cheaper). I mostly work with Taras Glek and other people on various aspects of Firefox startup performance. This involves pretty interesting low-level stuff, in which I actually got involved before being contracted. This work has nothing to do with packaging Iceweasel, though I can have a few opportunities to do things somehow related. You told in your blog that you were discussing with Mozilla how Debian could use the Firefox trademark. Can you tell us a bit about the progress you made? I have got an informal approval of the patches we are applying to the Iceweasel 4.0 beta packages. There are a few concerns about using the system cairo library, and the embedded copy of libpng needs to be used to have support for the animated PNGs that are used in the user interface and that are required in Firefox feature-set. Other than that, there are pretty much no objection on the technical side. We managed to get to some common ground as to how to manage stable updates and more generally, changes to our packages, but I m still waiting for an actual agreement draft. While I, as the maintainer, am fine with the way it should look according to my discussions with the people involved at Mozilla, I ll seek review from my fellow Debian Developers when I ll have something to show. I haven t decided yet how this will reflect on the Debian packages, though. For example, would Iceweasel stay? But I do hope Wheezy will get Firefox back. Is there someone in Debian that you admire for their contributions? Not someone, but all past, present and future release team members. This is a team that receives a lot of critics (and I indirectly gave one a few paragraphs ago), yet manages to keep doing an amazing job and its members are very committed to their task. I admire their dedication.
Thank you to Mike for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

8 comments Liked this article? Click here. My blog is Flattr-enabled.

2 February 2011

Jose Luis Rivas Contreras: No more TweetDeck for me, on AIR

I changed (once again) to awesome and AIR applications doesn t work here. One of the apps I heavily use based on AIR is TweetDeck. So I tried old clients that were once an alternative like HootSuite but not having support for both kind of RTs (Native and Quote) is a pain in the ass. Then I remembered I installed once TweetDeck for Google Chrome on my browser and never give it a chance, because it was not exactly the same as the AIR-based app. Now it only lacks proper relation between followers and no-followers in each account when there are multiple accounts configured. So yesterday was the day to try it, the right day. It even has notification support, and customized by column!

31 January 2011

Michal Čihař: OpenWRT and resolving private ranges

For quite a long time, I have a problem, that DNS in my OpenWRT box does not resolv local range on wan interface. Today I have leared that it is doing it in order to protect me:
dnsmasq[15709]: possible DNS-rebind attack detected: jmnet.czf
But hey, my wan interface has 10.* address and I want to resolve 10.* domains there! A little bit of googling revealed that this is caused by --stop-dns-rebind passed to dnsmasq. Okay, the easiest fix is to disable it. You need to change /etc/config/dhcp:
config 'dnsmasq'
        option 'rebind_protection' '0'
After reading source of /etc/init.d/dnsmasq I realized there is also way to whitelist some domains, what sounds like a slightly better idea. After fighting with syntax, I found the solution to be (again in /etc/config/dhcp):
config 'dnsmasq'
        list 'rebind_domain' '/czf/'
Next thing to explore is why is native IPv6 not working for me on the router though it should...

Filed under: English Openwrt 0 comments Flattr this!

Stefano Zacchiroli: who the bloody hell cares about Debian

A down-under talk on the role of Debian, A.D. 2011 I'm back from LCA 2011, which I've attended to share some thoughts about the role that Debian plays in the Free Software ecosystem, 18 years after its inception (yes, we are that old^Welder). The talk title---Who the bloody hell cares about Debian? [1]---was meant to be rather provocative. The idea was indeed to challenge the meme that, in the era of distributions that release every 6 months, a distribution with a release cycle of circa 2 years (like Debian, considering the past 5 years) is not a project that deserves your attention anymore. Is it really the case? In the talk I (obviously) claim it is not, using two main arguments. The first argument is based on the observation that Debian offers a set of pretty rare, if not unique, features among mainstream FOSS distributions. Those features consist of a mix of technical and "political" aspects: (1) a focus on package quality, with no distinction among first and second class packages; (2) a strong culture of software freedom, which refuses to offer non-free software (or firmware) by default to users and distribution developers (as parts of the infrastructure used to make Debian); (3) independence from commercial interests, with no single company or entity that could claim to babysit Debian; and (4) a decision making model based on a weighted sum of do-ocracy and democracy, which implies that by doing (rather than talking) everyone has a chance to have an impact on Debian. Considering all that and looking at the most popular FOSS distributions, one can easily identify Debian as one of the few remaining players who both care about Free Software and can be trusted in making choices not driven by profit. Mind you, I've nothing against companies in general and I'm very well aware that many FOSS companies carry a good deal of the burden of developing and promoting Free Software. Nonetheless, in days in which it is striking how quickly FOSS-friendly companies can become very much FOSS-unfriendly, I can't help putting my trust and efforts into community-driven projects, better if with no attached company label whatsoever. Furthermore, having distributions like Debian around can encourage other company-backed distributions to demand more and more independence and clarifications about the relationships between the community and the backing company. The second argument about the relevance of Debian is more pragmatic and rather straightforward: Debian is the root of a huge tree of derived-distribution (AKA "derivatives"), more than 120 according to popular distribution indexing sites. Each Debian derivative focuses its attention on, and directs its people power to, customizing Debian for a specific target and build entierly upon Debian work for all parts that do not need customization. That possibility is one key advantage of Free Software, after all. A well-known derivative example is Ubuntu, which is probably the most popular Debian derivative, enjoying a user base way larger than that of Debian itself. Ubuntu is heavily customized with respect to Debian and still had only about (at the time of Natty) 25% of packages which either differed from their Debian counterparts or that did not exist in Debian at all. Other Debian derivatives tend to be way less customized than that. Either way, if you are running a Debian derivative, chances are that you heavily depend on Debian and on its well-being. (Yes, it is so also if you didn't know about that, sorry.) Let's now fast-forward to the end of the talk, skipping my usual comments on how to keep the whole tree of Debian derivatives sustainable and beneficial for Free Software as a whole (i.e. by reducing as much as possible the viscosity of patch flow along the derivatives tree). Feedback People's reaction to such a provocative talk has been positive and we have enjoyed a fairly long Q&A session discussing the topics mentioned above as well as other Debian-related topics. A (commented) summary of the obtained feedback is reported below. Judging from the recurrent questions and suggestions I've received doing Debian talks in the past 8 months, I dare to say that LCA feedback is fairly representative of the feelings of many Debian users. Slides of the talk are available, while a video will eventually be posted at http://linuxconfau.blip.tv.
[1] Kudos to Francois Marier for suggesting the title. If the title doesn't ring a bell for you (it didn't for me in the beginning), you might want to check out the story of this ad. Update: minor rephrasing in the 4th paragraph

David Welton: Parallelized web site creating and selling - underperformingstocks.com

I often have ideas for little web sites, stuff I just get the urge to create for the fun of it. Some of them have done rather well, like langpop.com but even that doesn't make any money, and, sadly, as I age, that is becoming a more important factor in life. So I decided to try a little experiment. I was thinking some about "value" stocks, and fiddling around with Google's stock screener, and thought of the idea of a site that highlights underperforming stocks: http://underperformingstocks.com - so far so good - I went ahead and registered the domain, and put together the beginnings of a Rails application. At the same time though, I decided I have enough things to do in my evenings and weekends, so I decided that it might be interesting to start selling the site at the same time I'm building it. I'm not really sure how it will work out: perhaps people will see that there's not much there and not bid for it. Perhaps someone likes the idea behind it and is willing to buy on that. My idea, in any case, is to keep working on it as the auction progresses. As an additional twist, there are obviously a lot of features that could be added to the site, so I'll prioritize work on features suggested by users who have bid on the site. Here's the auction listing on Flippa.com: www.underperformingstocks.com is For Sale on Flippa! Edit: I'll also be posting updates to the twitter account I created for the project: http://twitter.com/#!/underpstocks

28 January 2011

Richard Darst: DebConf communication channels and decisions

This post is designed to describe communications channels and decision processes of DebConf. This is mostly designed to help new team members integrate into the team better and be able to contribute substantially. This only describes internal communication channels, not between DebConf and attendees.I should emphasize that this is the ideal process. Since time is so limited, things get rushed. You shouldn't worry about the steps above slowing you down, they are mostly what you'd do anyway if you are working transparently. Furthermore, once something has been delegated to you, just try to be public in your discussions and document what you will do. This is supposed to be a guide to help you take initiative, not slow you down.
#debconf-team (on OFTC) is the primary IRC channel. Most people idle here. Discussion and decisions often happen here, but saying things only in IRC isn't enough. IRC is ephemeral, so important information and decisions should be on the website, wiki, or mailing lists. IRC is good for getting advice from past members and keeping people up to date, and real-time communication among teams. Groups actively talking among each other usually use this channel. Not everyone can be expected to watch IRC all the time.The mailing lists, in particular debconf-team@ are for permanent records of meetings, and for getting feedback from all team members. For a decision to have weight, there should be a chance given for this list to provide feedback.The wiki is the place for permanent and organized documentation. Ideally, the wiki is a place for team members and others to go to find out just what we are doing. (Official information for attendees goes on the website, not the wiki.)Decisions in DebConf are hopefully made by delegation or rough consensus, but often time gets short and things have got to get done. Tradition and historical precedent play a big part: not everything can be re-designed every year. What follows below is a rough best-practice for a decision process, but I'm not saying that everything always is or has to be like this. Things get faster once you get more familiar with people.People first should read historical documents or chat with people and learn how things have worked in the past. People should know what you are working on, either through IRC or a mail to debconf-team@. This will get you feedback early and perhaps some additional advice and help.The group discusses and decides on some sort of proposal or course of action. This could be by IRC, lists, or whatever they need. It should be some public forum so that other people can watch and give feedback.The group's proposal is sent to list for feedback. Hopefully, they would have a preliminary write-up on the wiki so others can add to it there. Perhaps there is more than one proposal and they can be given side by side. Sending a proposal isn't just so that people can object: even if everyone thinks it looks good, it is important that there is some record of the process. You can get other opinions on the proposal here, and if it seems there is rough consensus, then you could consider it decided - just let someone know that you are implementing it.If it's not clear that there is agreement on the proposal, you should bring it up in a IRC meeting. Add it to an agenda, and we can discuss it. Discussing it at a meeting has the advantage that most people are there, so we can make sure that the idea isn't lost due to indecision. We want meetings to be efficient, so try to have at least a description on the lists first.Finally, things should be documented on the wiki or website. This is important to maintain transparency, and since this will be the first place people look for reference in future years.

Thomas Koch: Google Summer of Code 2011

This summer I'll be a full time student (again) to finish my computer science bachelor. So I thought it would be a nice occasion to apply for the Google Summer of Code. Would anybody like to recommend me a project? Something around Debian, Java, decentralized social software, quality assurance or developer tools?

Thank you!

Michal Čihař: phpMyAdmin improvements

As my Gammu Hackweek 6 project went smoothly, I've decided to spend some time today on phpMyAdmin as well. My work was mostly inspired by (IMHO unfair) comparing of Adminer and phpMyAdmin. I could surely create similar comparing where would be dozens of things why is phpMyAdmin better, but I don't want to go that way and I rather decided to spend my time on coding. First and easy task was bit which is in comparing described as "Status variables", where I've added documentation links, so phpMyAdmin now has both short executive summary and link to the official documentation. While looking at server status page, I've categorized some newly added variables, so they now appear in right place. I also reconsidered formatting of numbers and hopefully current solution will work for everybody and not produce contradicting bug reports like the old one. Then I did focus a little bit on testing AJAX and fixed some minor issues, mostly affecting Chrome/Chromium browsers, because they are strict in case of some errors. Last but not least I've added URL redirector inside phpMyAdmin, so no external links will now reveal what did user actually was doing in phpMyAdmin, but rather the url of the redirector. And when speaking about phpMyAdmin, I need to mention one thing as well: contribute yourself! You don't have to be developer, there is currently ongoing themes contest and we're always looking for translators.

Filed under: English Phpmyadmin Suse 0 comments Flattr this!

Amaya Rodrigo: Indignez Vous!

If There is any sort of food for thought worth reading, this is it.
INDIGNEZ-VOUS! GET ANGRY! CRY OUT by St phane Hessel


St phane Hessel, author of Indignez-vous!
After 93 years, it is almost the final act. The end for me is not very far off any more. But it still leaves me a chance to be able to remind others of what acted as the basis of my political engagement. It was the years of resistance to the Nazi occupation -- and the program of social rights worked out 66 years ago by the National Council of the Resistance!

It is to Jean Moulin [murdered founder of the Council] that we owe, as part of this Council, the uniting of all elements of occupied France -- the movements, the parties, the labor unions -- to proclaim their membership in Fighting France, and we owe this to the only leader that it acknowledged, General de Gaulle. From London, where I had joined de Gaulle in March 1941, I learned that this Council had completed a program and adopted it on March 15th, 1944, that offered for liberated France a group of principles and values on which would rest the modern democracy of our country.

These principles and these values, we need today more than ever. It is up to us to see to it, all together, that our society becomes a society of which we are proud, not this society of immigrants without papers -- expulsions, suspicion regarding the immigrants. Not this society where they call into question social security and national retirement and health plans. Not this society where mass media are in the hands of the rich. These are things that we would have refused to give in to if we had been the true heirs of the National Council of the Resistance.









From 1945, after a dreadful drama [WWII], it was an ambitious
resurrection of society to which the remaining contingent of the Council
of the Resistance devoted itself. Let us remember them while creating
national health and pensions plans such as the Resistance wished, as its
program stipulated, "a full plan of French national health and social
security, aimed at assuring all citizens the means of existence whenever
they are unable to obtain them by a job; a retirement allowing the old
workers to finish their days with dignity."

The sources of energy, electricity, and gas, mines, the big banks, were
nationalized. Now this was as the program recommended: "... the return
to the nation of big monopolized means of production, fruits of common
labor, sources of energy, wealth from the mines, from insurance
companies and from big banks; the institution of a true economic and
social democracy involving the ousting of the big economic and financial
fiefdoms from the direction of the economy."

General interest must dominate over special interest. The just man
believes that wealth created in the realm of labor should dominate over
the power of money.

The Resistance proposed, "a rational organization of the economy
assuring the subordination of special interests to general interest, and
the emancipation of 'slaves' of the professional dictatorship that was
instituted just as in the fascist states," which had used the interim
[for two years after the war] government of the Republic as an agent.

A true democracy needs an independent press, and the Resistance
acknowledged it, demanded it, by defending "the freedom of the press,
its honor, and its independence from the State, the power of money and
foreign influence." This is what relieved restrictions on the press from
1944 on. And press freedom is definitely what is in danger today.

The Resistance called for a "real possibility for all French children to
benefit from the most advanced education," without discrimination.
Reforms offered in 2008 go contrary to this plan. Young teachers, whose
actions I support, went so far as refusing to apply them, and they saw
their salaries cut by way of punishment. They were indignant,
"disobeyed," judging these reforms too far from the ideal of the
democratic school, too much in the service of a society of commerce and
not developing the inventive and critical mind enough. 2

All the foundations of the social conquests of the Resistance are
threatened today.

The motive of the Resistance: indignation (Indignez-vous!)

Some dare to say to us that the State cannot afford the expenses of
these measures for citizens any more. But how can there be today a lack
of money to support and extend these conquests while the production of
wealth has been considerably augmented since the Liberation period when
Europe was in ruins? On the contrary, the problem is the power of money,
so much opposed by the Resistance, and of the big, boldfaced, selfish
man, with his own servants in the highest spheres of the State.

Banks, since privatized again, have proved to be concerned foremost for
their dividends and for the very high salaries of their leaders, not the
general interest. The disparity between the poorest and the richest has
never been so great, and amassing money, competition, so encouraged.

The basic motive of the Resistance was indignation!

We, the veterans of the resistance movements and combat forces of Free
France, we call on the young generation to live by, to transmit, the
legacy of the Resistance and its ideals. We say to them: Take our place,
"Indignez-vous!" [Get angry! or Cry out!].

The political, economic, intellectual leaders, and the whole society do
not have to give in, nor allow oppression by an actual international
dictatorship of the financial markets, which threatens peace and
democracy.

I wish for you all, each of you, to have your own motive for
indignation. It is precious. When something outrages you as I was
outraged by Nazism, then people become militant, strong, and engaged.
They join this current of history, and the great current of history must
continue thanks to each individual. And this current goes towards more
justice, more freedom, but not this unbridled freedom of the fox in the
henhouse. The rights contained in the UN Universal Declaration of Human
Rights of 1948 are just that, universal.

If you meet somebody who does not benefit from it, feel sorry for them
but help them to win their rights.

Two visions of history

When I try to understand what caused fascism, what made it so we were
overcome by Hitler and the Vichy [French government that collaborated
with Hitler], I tell myself that the propertied, with their selfishness,
were terrifically afraid of Bolshevik revolution. They were allowed to
lead with their fear.

But if, today as then, an active minority stands up, it will be enough;
we shall be the leavening that makes the bread rise. Certainly, the
experience of a very old person like me, born in 1917, is different from
the experience of the today's young persons. I often ask professors for
the opportunity to interact with their students, and I say to them: You
don't have the same obvious reasons to engage you. For us, to resist was
not to accept German occupation, defeat. It was comparatively simple.
Simple as what followed, decolonization. Then the war in Algeria.

It was necessary that Algeria become independent, it was obvious. As for
Stalin, we all applauded the victory of the Red Army against the Nazis
in 1943. But already we had known about the big Stalinist trials of
1935, and even if it was necessary to keep an ear open towards communism
to compensate against American capitalism, the necessity to oppose this
unbearable form of totalitarianism had established itself as an
obviousness. My long life presented a succession of reasons to outrage
me.

These reasons were born less from an emotion than a deliberate
commitment. As a young student at normal school [teachers college] I was
very influenced by Sartre, a fellow student. His "Nausea" [a novel],
"The Wall," [play], and "The Being and Nothingness" [essay] were very
important in the training of my thought. Sartre taught us, "You are
responsible as individuals." It was a libertarian message. The
responsibility of a person can not be assigned by a power or an
authority. On the contrary, it is necessary to get involved in the name
of one's responsibility as a human being.

When I entered the French Ecole Normale Superieure, Ulm Street, in Paris
in 1939, I entered it as a fervent adherent of the philosopher Hegel,
and I adhered to the thought of Maurice Merleau-Ponty. His teaching
explored concrete experience, that of the body and of its relations with
the senses, one big singular sense faced with a plurality of senses. But
my natural optimism, which wants all that is desirable to be possible,
carried me rather towards Hegel. Hegelism interprets the long history of
humanity as having a meaning: It is the freedom of man progressing step
by step. History is made of successive shocks, and the taking into
account of challenges. The history of societies thus advances; and in
the end, man having attained his full freedom, we have the democratic
state in its ideal form.

There is certainly another understanding of history. It says progress is
made by "freedom" of competition, striving for "always more"; it can be
as if living in a devastating hurricane. That's what it represented to a
friend of my father, the man who shared with him an effort to translate
into German "The Search for Time Lost" [novel] by Marcel Proust.

That was the German philosopher Walter Benjamin. He had drawn a
pessimistic view from a painting by the Swiss painter Paul Klee,
"Angelus Novus," where the face of the angel opens arms as if to contain
and push back a tempest, which he identifies with progress. For
Benjamin, who would commit suicide in September 1940 to escape Nazism,
the sense of history is the overpowering progression of disaster upon
disaster.

Indifference: the worst of attitudes

It is true the reasons to be indignant can seem today less clearly
related or the world too complex. Who's doing the ordering, who decides?
It is not always easy to differentiate between all the currents that
govern us. We are not any more dealing with a small elite whose joint
activities can be clearly seen. It is a vast world, of which we have a
feeling of interdependence.

We live in an interconnectivity as never before. But in this world there
still are intolerable things. To see them, it is well and necessary to
look, to search. I say to the young people, Search little, and that is
what you are going to find. The worst of attitudes is indifference, to
say "I can do nothing there, I'll just manage to get by." By including
yourself in that, you lose one of the essential elements that makes the
human being: the faculty of indignation and the commitment that is a
consequence of it.

They [young people] can already identify two big new challenges:

1. The huge gap which exists between the very poor and the very rich and
that does not cease increasing. It is an innovation of the 20th and 21st
centuries. The very poor in the today's world earn barely two dollars a
day. The new generation cannot let this gap become even greater. The
official reports alone should provoke a commitment.

2. Human rights and state of the planet: I had the chance after the
Liberation to join in the writing of the Universal Declaration of Human
Rights, adopted by the United Nations organization, on December 10th,
1948, in Paris at the palace of Chaillot. It was as principal private
secretary of Henry Laugier, the adjunct general-secretary of the UN, and
as and secretary of the Commission on Human Rights that I with others
was led to participate in the writing of this statement. I wouldn't know
how to forget the role in its elaboration of Ren Cassin, who was
national commissioner of justice and education in the government of Free
France in London in 1941 and won the Nobel peace prize in 1968, nor that
of Pierre Mend s-France in the Economic and Social Council, to whom the
text drafts we worked out were submitted before being considered by the
Third Committee (Social, Humanitarian and Cultural) of the General
Assembly. It was ratified by the 54 member states in session of the
United Nations, and I certified it as secretary.

It is to Ren Cassin that we owe the term "universal rights" instead of
"international rights" as offered by our American and British friends.
This [universal versus international] was key because, at the end of the
Second World War, what was at stake was to becomeereignty," which a
nation can emphasize while it devotes itself to crimes against humanity
on its own soil. Such was the case of Hitler, who felt himself supreme
and authorized to carry out a genocide. This universal statement owed
much to universal revulsion towards Nazism, fascism, and totalitarianism
-- and owes a lot, in our minds, to the spirit of the Resistance.

I had a feeling that it was necessary to move quickly so as not to be
dupes of the hypocrisy that there was in the UN membership, some whom
claimed these values already won but had no intention at all to promote
them faithfully -- claimed that we were trying to impose values on them.

I can not resist the desire to quote Article 15 of the Universal
Declaration of Human Rights (1948): "Everyone has the right to a
nationality." Article 22 says, "Everyone, as a member of society, has
the right to social security and is entitled to realization, through
national effort and international cooperation and in accordance with the
organization and resources of each State, of the economic, social and
cultural rights indispensable for his dignity and the free development
of his personality." And if this statement has a declarative scope, and
not statutory, the Declaration nevertheless has played a powerful role
since 1948. It saw colonized people take it up in their fight for
independence; it sowed minds in a battle for freedom.

I note with pleasure that in the course of last decades there has been
an increase in nongovernmental organizations (NGOs) and social movements
such as ATTAC (Association for the Taxation of Financial Transactions);

also FIDH (International Federation for Human Rights) and Amnesty
International, which are active and competitive. It is obvious that to
be effective today it is necessary to act in a network, to use all
modern means of communication.

To the young people, I say: Look around you, you will find topics that
justify your indignation facts about treatment of immigrants, of
"illegal" immigrants, of the Roma [aka Gypsies]. You will find concrete
situations that lead you to strong citizen action. Search and you shall
find!

My indignation regarding Palestine outrages by Israel [Indignez-vous!]

Today, my main indignation concerns Palestine, the Gaza Strip, and the
West Bank of Jordan. This conflict is outrageous. It is absolutely
essential to read the report by Richard Goldstone, of September 2009, on
Gaza, in which this South African, Jewish judge, who claims even to be a
Zionist, accuses the Israeli army of having committed "acts comparable
to war crimes and perhaps, in certain circumstances, crimes against
humanity" during its "Operation Cast Lead," which lasted three weeks.

I went back to Gaza in 2009 myself, when I was able to enter with my
wife thanks to our diplomatic passports, to study first-hand what this
report said. People who accompanied us were not authorized to enter the
Gaza Strip. There and in the West Bank of Jordan. We also visited the
Palestinian refugee camps set up from 1948 by the United Nations agency
UNRWA, where more than three million Palestinians expelled off their
lands by Israel wait even yet for a more and more problematical return.

As for Gaza, it is a roofless prison for one and a half million
Palestinians. A prison where people get organized just to survive.
Despite material destruction such as that of the Red Crescent hospital
by Operation Cast Lead, it is the behavior of the Gazans, their
patriotism, their love of the sea and beaches, their constant
preoccupation for the welfare of their children, who are innumerable and
cheerful, that haunt our memory. We were impressed by how ingeniously
they face up to all the scarcities that are imposed on them. We saw them
making bricks, for lack of cement, to rebuild the thousands of houses
destroyed by tanks. They confirmed to us that there had been 1400 deaths
including women, children, and oldsters in the Palestinian camp
during this Operation Cast Lead led by the Israeli army, compared to
only 50 injured men on the Israeli side. I share conclusions of the
South African judge. That Jews can, themselves, perpetrate war crimes is
unbearable. Alas, history does not give enough examples of people who
draw lessons from their own history. [The author, St phane Hessel, had
a Jewish father.]

Terrorism, or exasperation?

I know that Hamas [party of Palestine freedom fighters], which had won
the last legislative elections, could not help it that rockets were
launched on Israeli cities in response to the situation of isolation and
blockade in which Gazans exist. I think, naturally, that terrorism is
unacceptable; but it is necessary to acknowledge (from experience in
France) that when people are occupied by forces immensely superior to
their own, popular reaction cannot be altogether bloodless.

Does it serve Hamas to send rockets onto the town of Sd rot [Israeli
town across the border from Gaza]?

The answer is no. This does not serve their purpose, but they can
explain this gesture by the exasperation of Gazans. In the notion of
exasperation, it is necessary to understand violence as the regrettable
conclusion of situations not acceptable to those who are subjected them.

Thus, they can tell themselves, terrorism is a form of exasperation. And
that this "terrorism" is a misnomer. One should not have to resort to
this exasperation, but it is necessary to have hope. Exasperation is a
denial of hope. It is comprehensible, I would say almost natural, but it
still is not acceptable. Because it does not allow one to acquire
results that hope can possibly, eventually produce.

Nonviolence: the way we must learn to follow

I am persuaded that the future belongs to nonviolence, to reconciliation
of different cultures. It is by this way that humanity will have to
enter its next stage. But on this I agree with Sartre: We cannot excuse
the terrorists who throw bombs, but we can understand them. Sartre wrote
in 1947: "I recognize that violence in whatever form it may manifest
itself is a setback. But it is an inevitable setback because we are in a
world of violence. And if it is true that recourse to violence risks
perpetuating it, it is also true it is the sure means to make it stop."

To that I would add that nonviolence is a surer means of making violence
stop. One can not condone the terrorism, using Sartre or in the name of
this principle, during the war of Algeria, nor during the Munich Games
of 1972 the murder attempt made against Israeli athletes. Terrorism is
not productive, and Sartre himself would end up wondering at the end of
his life about the sense of violence and doubt its reason for being.

However, to proclaim "violence is not effective" is more important than
to know whether one must condemn or not those who devote themselves to
it. Terrorism is not effective. In the notion of effectiveness, a
bloodless hope is needed. If there is a violent hope, it is in the poem
of William Apollinaire "that hope is violent," and not in policy.

Sartre, in March 1980, within three weeks of his death, declared: "It is
necessary to try to explain why the world of today, which is horrible,
is only an instant in a long historical development, that hope always
has been one of the dominant forces in revolutions and insurrections,
and how I still feel hope as my conception of the future." [Note 5]

It is necessary to understand that violence turns its back on hope. It
is necessary to prefer to it hope, hope over violence. Nonviolence is
the way that we must learn to follow. So must the oppressors.
10

It is necessary to arrive at negotiations to remove oppression; it is
what will allow you to have no more terrorist violence. That's why you
should not let too much hate pile up.

The message of Mandela and Martin Luther King finds all its pertinence
in the world that overcame the confrontation of ideologies [e.g.,
Nazism] and conquered totalitarianism [e.g.,Hitler]. It is also a
message of hope in the capacity of modern societies to overcome
conflicts by a mutual understanding and a vigilant patience. To reach
that point is necessarily based on rights, against es, such as the
military intervention in Iraq.

We had this economic crisis, but we still did not initiate a new policy
of development. Also, the summit of Copenhagen against climatic warming
did not bring about a true policy for the preservation of the planet.

We are on a threshold between the terror of the first decade and the
possibilities of following decades. But it is necessary to hope, it is
always necessary to hope. The previous decade, that of 1990s, had been a
time of great progress. The United Nations had enough wisdom to call
conferences such as those of Rio on environment, in 1992, and that of
Beijing on women, in 1995. In September 2000, on the initiative of the
general secretary of United Nations, Kofi Annan, the 191 member
countries adopted a statement on the "eight objectives of the millennium
for development," by which they notably promised to reduce poverty in
the world by half before 2015.

My big regret is that neither Obama nor the European Union has yet
committed themselves to what should be the provision for a useful forum
bearing on the fundamental values.

Conclusion

How to conclude this call to be indignant? By saying still what, on the
occasion of the sixtieth anniversary of the program of the National
Council of the Resistance, we said on March 8th, 2004 -- we veterans of
the resistance movements and combat forces of Free France (1940-1945) --
that certainly "Nazism was conquered, thanks to the sacrifice of our
brothers and sisters of the Resistance and United Nations against
fascist barbarism. But this threat did not completely disappear, and our
anger against injustice is ever intact." [Note 6] Also, let us always be
called in "a truly peaceful insurrection against means of mass
communication that offer as a vista for our youth only the consumption
of mass trivia, contempt of the weakest and the culture, a generalized
amnesia, and the hard competition of all against all."

To those who will make the 21st century, we say with our affection:

TO CREATE IS TO RESIST; TO RESIST IS TO CREATE.

Wouter Verhelst: Good news and bad news

... in the world of Belgian Tennis. I learned yesterday that Justine Henin, ex-number 1 tennis player and still one of the great talents amongst women's tennis, is forced to stop her come-back attempt due to an elbow injury which she had been dealing with since the last Wimbledon. I don't want to guess what's going through her mind at this point in time, but it must be terrible. She'd been working so hard for tennis for all her life, but took a break from the sport because she realized that she needed to give her life apart from the sport a serious boost. And when she'd done that and realized that she still loves tennis and wants to do a come-back, this happens before she'd even been able to accomplish even one of her goals that she'd set for her come-back. Terrible. Justine, on the off chance that you read this: you have my greatest possible sympathies. The slightly better news also involves an injury; but this time it's not a Belgian player, but it involves Venus Williams. Now I'm not one to gloat at sportspeople's problems, and I do wish Venus all the best and a speedy recovery; but since her injury means she won't be able to make it for the Fed Cup tie against Belgium, slightly over a week from now, the result is that Belgium is now favourite for winning this tie. Of course, the team can't claim having won until the tie is over, but hey. Too bad I can't be there. But there's this other thing that I'm involved with, which just happens to be in the same weekend. Oh well.

25 January 2011

Benjamin Mako Hill: Annual Free Software Foundation Fundraiser

<script type="text/javascript"> var fsf_widget_text = "Help protect your freedom!"; var fsf_widget_d_btn = "Donate"; var fsf_widget_share = "Share this widget."; var fsf_widget_size = "normal"; var fsf_associate_id = "3427"; </script> <script src="http://gplv3.fsf.org/static/widget/widget.js" type="text/javascript"></script>
The Free Software Foundation is in the last week of its annual fundraiser and has still has a bit of ground to make up. The FSF needs members and donations to merely sustain its basic activity protecting free software and engaging in minimal outreach. So as I've done in the last couple years, I've written a fundraising appeal for the organization. That why today my face is plastered, Jimmy Wales style, all over the FSF website. (For the record, the last bit was not my idea and I find it a little embarrassing.) My appeal this year begins with my feeling that as our lives become increasingly mediated by technology, the question of who controls technology becomes another way of asking "Who controls us?" Read the appeal for more. That statement, like much of the FSF's work, is not going to get very far with most users today. Most people don't think of technology as the source of control, power, or autonomy. They don't understand why software should be free or even what that might mean. Making this argument widely is something the FSF works hard to do. But supporting this work is a bit of a chicken-egg problem. The users we most need to reach have never heard our message and it can take some time to turn even the most receptive into supporters. For those of us that do care about technology and the power it has over its users, it's hugely important that we take action to support software freedom and organizations like the FSF that play a key place in this struggle. If we don't, nobody else will. The pool of people who might support the FSF today is small -- there are currently about 3,000 members -- but the stakes are huge: A choice between a world of DRM, locked down mobile phones, and non-free network services on one hand, and the possibility of a world where there are free alternatives on the other. The FSF won't win this fight alone, but it plays a critical role in it. The FSF is a small, humble, and responsible non-profit and a little money can have a big impact. If you are not an FSF associate member, now would be a great time to join. Membership is $120 per year ($60 for students) and payable monthly. In preparing this appeal, I realized that my membership had lapsed a couple months ago and that my partner Mika was still donating at the reduced student price. We've personally fixed both these things and made an addition donation. If you are a member, you can check your status or make a donation. If you are not yet, Please consider joining us.

22 January 2011

Lars Wirzenius: Obnam and btree journal dump

For some years now I have kept a private journal, sort of a private blog. It is mainly related to my work and hobby development activities, though some private life gets poured into it as well. It's a private journal. It is not meant for others to read. It has a dual purpose in life: I find this journalling activity to be beneficial to software development. Since I mainly work alone, at home, it is a bit like having someone to talk to across the desk. However, since it is recorded, it is more practical to grep things for useful stuff later on. Re-reading entries also gives me a wonderful sense of how fast or slow things happen. For example, on January 23 last year, exactly one year ago tomorrow, I wrote:
That concludes my list of next actions that should be done before obnam is theoretically able to do normal backups.
I then spent most of last year actually making Obnam be capable of doing backups for real. Recently, I thought that it might be interesting for some to read another programmer's thought processes when designing, writing, and debugging code. For this reason, I've collected some of my journal entries from the past year or so, those related to Obnam and my Python B-tree library (which Obnam uses). I have lightly sanitized them, to protect other people's privacy. Other than that, it's me as raw as you could wish. The dump is on my website. Please be gentle and laugh at me behind my back.

20 January 2011

John Goerzen: Research on deduplicating disk-based and cloud backups

Yesterday, I wrote about backing up to the cloud. I specifically was looking at cloud backup services. I ve been looking into various options there, but also various options for disk-based backups. I d like to have both onsite and offsite backups, so both types of backup are needed. Also, it is useful to think about how the two types of backups can be combined with minimal overhead. For the onsite backups, I d want to see:
  1. Preservation of ownership, permissions, etc.
  2. Preservation of symlinks and hardlinks
  3. Space-efficient representation of changes ideally binary deltas or block-level deduplication
  4. Ease of restoring
  5. Support for backing up Linux and Windows machines
Deduplicating Filesystems for Local Storage Although I initially thought of block-level deduplicating file systems as something to use for offsite backups, they could also make an excellent choice for onsite disk-based backups. rsync-based dedup backups One way to use them would be to simply rsync data to them each night. Since copies are essentially free, we could do (or use some optimized version of) cp -r current snapshot/2011-01-20 or some such to save off historic backups. Moreover, we d get dedup both across and within machines. And, many of these can use filesystem-level compression. The real upshot of this is that the entire history of the backups can be browsed as a mounted filesystem. It would be fast and easy to find files, especially when users call about that file that they deleted at some point in the past but they don t remember when, exactly what it was called, or exactly where it was stored. We can do a lot more with find and grep to locate these things than we could do with tools in Bacula (or any other backup program) restore console. Since it is a real mounted filesystem, we could also do fun things like make tarballs of it at will, zip parts up, scp them back to the file server, whatever. We could potentially even give users direct access to their files to restore things they need for themselves. The downside of this approach is that rsync can t store all the permissions unless it s running as root on the system. Wrappers such as rdup around rsync could help with that. Another downside is that there isn t a central scheduling/statistics service. We wouldn t want the backup system to be hammered by 20 servers trying to send it data at once. So there d be an element of rolling our own scripts, though not too bad. I d have preferred not to authorize a backup server with root-level access to dozens of machines, but may be inescapable in this instance. Bacula and dedup The other alternative I thought of system such as Bacula with disk-based volumes . A Bacula volume is normally a tape, but Bacula can just write them to disk files. This lets us use the powerful Bacula scheduling engine, logging service, pre-backup and post-backup jobs, etc. Normally this would be an egregious waste of disk space. Bacula, like most tape-heritage programs, will write out an entire new copy of a file if even one byte changes. I had thought that I could let block-level dedupe reduce the storage size of Bacula volumes, but after looking at the Bacula block format spec, this won t be possible as each block will have timestamps and such in it. The good things about this setup revolve around using the central Bacula director. We need only install bacula-fd on each server to be backed up, and it has a fairly limited set of things it can do. Bacula already has built-in support for defining simple or complicated retention policies. Its director will email us if there is a problem with anything. And its logs and catalog are already extensive and enable us to easily find out things such as how long backups take, how much space they consume, etc. And it backs up Windows machines intelligently and comprehensively in addition to POSIX ones. The downsides are, of course, that we don t have all the features we d get from having the entire history on the filesystem all at once, and far less efficient use of space. Not only that, but recovering from a disaster would require a more extensive bootstrapping process. A hybrid option may be possible: automatically unpacking bacula backups after they ve run onto the local filesystem. Dedupe should ensure this doesn t take additional space if the Bacula blocksize aligns with the filesystem blocksize. This is certainly not a given however. It may also make sense to use Bacula for Windows and rsync/rdup for Linux systems. This seems, however, rather wasteful and useless. Evaluation of deduplicating filesystems I set up and tested three deduplicating filesystems available for Linux: S3QL, SDFS, and zfs-fuse. I did not examine lessfs. I ran a similar set of tests for each:
  1. Copy /usr/bin into the fs with tar -cpf - /usr/bin tar -xvpf - -C /mnt/testfs
  2. Run commands to sync/flush the disk cache. Evaluate time and disk used at this point.
  3. Rerun the tar command, putting the contents into a slightly different path in the test filesystem. This should consume very little additional space since the files will have already been there. This will validate that dedupe works as expected, and provide a hint about its efficiency.
  4. Make a tarball of both directories from the dedup filesystem, writing it to /dev/zero (to test read performance)
I did not attempt to flush read caches during this, but I did flush write caches. The test system has 8GB RAM, 5GB of which was free or in use by a cache. The CPU is a Core2 6420 at 2.13GHz. The filesystems which created files atop an existing filesystem had ext4 mounted noatime beneath them. ZFS was mounted on an LVM LV. I also benchmarked native performance on ext4 as a baseline. The data set consists of 3232 files and 516MB. It contains hardlinks and symlinks. Here are my results. Please note the comments below as SDFS could not accurately complete the test.
Test ext4 S3QL SDFS zfs-fuse
First copy 1.59s 6m20s 2m2s 0m25s
Sync/Flush 8.0s 1m1s 0s 0s
Second copy+sync N/A 0m48s 1m48s 0m24s
Disk usage after 1st copy 516MB 156MB 791MB 201MB
Disk usage after 2nd copy N/A 157MB 823MB 208MB
Make tarball 0.2s 1m1s 2m22s 0m54s
Max RAM usage N/A 150MB 350MB 153MB
Compression none lzma none gzip-2
It should be mentioned that these tests pretty much ruled out SDFS. SDFS doesn t appear to support local compression, and it severely bloated the data store, which was much larger than the original data. Moreover, it permitted any user to create and modify files, even if the permissions bits said that the user couldn t. tar gave many errors unpacking symlinks onto the SDFS filesystem, and du -s on the result threw up errors as well. Besides that, I noted that find found 10 fewer files than in my source data. Between the huge memory consumption, the data integrity concerns, and inefficient disk storage, SDFS is out of the running for this project. S3QL is optimized for storage to S3, though it can also store its files locally or on an sftp server a nice touch. I suspect part of its performance problem stems from being designed for network backends, and using slow compression algorithms. S3QL worked fine, however, and produced no problems. Creating a checkpoint using s3qlcp (faster than cp since it doesn t have to read the data from the store) took 16s. zfs-fuse appears to be the most-used ZFS implementation on Linux at the moment. I set up a 2GB ZFS pool for this test, and set dedupe=on and compress=gzip-2. When I evaluated compression in the past, I hadn t looked at lzjb. I found a blog post comparing lzjb to the gzip options supported by zfs and wound up using gzip-2 for this test. ZFS really shone here. Compared to S3QL, it took 25s instead of over 6 minutes to copy the data over and took only 28% more space. I suspect that if I selected gzip -9 compression it would have been closer both in time and space to S3QL. But creating a ZFS snapshot was nearly instantaneous. Although ZFS-fuse probably doesn t have as many users as ZFS on Solaris, still it is available in Debian, and has a good backing behind it. I feel safer using it than I do using S3QL. So I think ZFS wins this comparison. I spent quite some time testing ZFS snapshots, which are instantaneous. (Incidentally, ZFS-fuse can t mount them directly as documented, so you create a clone of the snapshot and mount that.) They worked out as well as could be hoped. Due to dedupe, even deleting and recreating the entire content of the original filesystem resulted in less than 1MB additional storage used. I also tested creating multiple filesystems in the zpool, and confirmed that dedupe even works between filesystems. Incidentally wow, ZFS has a ton of awesome features. I see why you OpenSolaris people kept looking at us Linux folks with a sneer now. Only our project hasn t been killed by a new corporate overlord, so guess that maybe didn t work out so well for you <grin>. The Cloud Tie-In This discussion leaves another discussion: what to do about offsite backups? Assuming for the moment that I want to back them up over the Internet to some sort of cloud storage facility, there are about 3 options:
  1. Get an Amazon EC2 instance with EBS storage and rsync files to it. Perhaps run ZFS on that thing.
  2. Use a filesystem that can efficiently store data in S3 or Cloud Files (S3QL is the only contender here)
  3. Use a third-party backup product (JungleDisk appears to be the leading option)
There is something to be said for using a different tool for offsite backups if there is some tool-level issue, that could be helpful. One of the nice things about JungleDisk is that bandwidth is free, and disk is the same $0.15/GB-mo that RackSpace normally charges. JungleDisk also does block-level dedup, and has a central management interface. This all spells nice for us. The only remaining question would be whether to just use JungleDisk to back up the backup server, or to put it on each individual machine as well. If it just backs up the backup server, then administrative burdens are lower; we can back everything there up by default and just not worry about it. On the other hand, if there is a problem with our main backups, we could be really stuck. So I d say I m leaning towards ZFS plus some sort of rsync solution and JungleDisk for offsite. I had two people suggest CrashPlan Pro on my blog. It looks interesting, but is a very closed product which makes me nervous. I like using standard tools and formats gives me more peace of mind, control, and recovery options. CrashPlan Pro supports multiple destinations and says that they do cloud hosting, but don t list pricing anywhere. So I ll probably not mess with it. I m still very interested in what comments people may have on all this. Let me know!

Alexander Reichle-Schmehl: Myths and Facts about Firmwares and their non-removal from Debian

Debian's announcement to release Squeeze with a completely free Linux kernel caused quite some attention, which is actually a good thing. However, it also seems to have caused quite some uncertainty and was often partially misunderstood and miss quoted. I'll try to summarize and answer some of them in this blog post: So, one question remains: What is to bad about non-free firmware files? Aren't they just some tiny programs executed in the CPU of the device? Why care about them Good question! Let's take possible legal issues aside, and just look the practical side. The core problem is: Without source (and tool chain to use the source) firmware files are just some random numbers for us. We don't know what they are doing, we can't analyse and improve them. We can't change them, we can't support them. Maybe you already followed the link above to comment of the Fedora developer. I cite him here again, because I think he summarized the problem so well:
Updated qlogic 2400 and 2500 firmware to 5.03.13. What does 5.03.13 do? No one knows, except for QLogic, and they're not telling. I asked, and they told me that information was only available under NDA. So, I encourage you to imagine what this firmware does, and the bugs it fixes. While you're at it, imagine a world where vendors release source code for their firmware.
So, now that we established the fact, that we can't support firmware files, one could wonder, if we actually need to do so. What harm can a simple, tiny program in a peripheral device do to your computer? Well, scientist already managed to created trojaned firmwares for some network cards. So it is a problem, and can even under some circumstanced lead to security problems! So, to summarise: Yes, Debian changed something in the Kernels. No, it will continue to work as well as usual. Some users might need to enable the non-free repository in their sources list, but those who won't, don't have to. Firmware files needed during installation are also available, and can be loaded by the installation system. So, what's all the fuss about? BTW: Those of you, who can't remember links very well, but fear, they still might need the links to the non-free images and tarballs. just remember two things: wiki and Firmware, as you'll find all you need on the Firmware page of Debian's wiki.

Next.

Previous.