Search Results: "spa"

5 February 2011

Craig Small: A few minor updates

No real terribly exciting updates lately but all important in their own way. ncurses A new version of ncurses got uploaded into sid. It should hopefully fix the backspace versus delete problem some of the FreeBSD people have been having with some of the terminal emulators. Oh, and if you are pining for the good old days when 8 colours and 64kB of memory was good enough for everyone, checkout ncurses-examples which have a bunch of interesting things you can do with curses, including some games. Anyone remeber the tower of Hanoi? dh-make dh-make, the tool to take a raw tar (or zip) archive and make a Debian package has been updated too. One of the interesting side-effects of maintaining dh-make is you find out (usually through bug reports) the little changes that happen to the Debian package build process. Most people, including myself, though the main big thing between versions 1 and 2 and version 3 of the source formats was the inclusion of quilt patches right inside the source files. And probably for the sake of "main things" this is true. What I didn't realise is that dpkg-source has changed in other ways too. Bug 580804 tripped this one up. For most people, you run dh_make, fix some things up and use debuild. If you didn't have the original tar file (perhaps you got the source from git or svn) you just use --creatorig and it copies the current directory into ../whatever-1.2.3.orig Source format 3 doesn't like this. Previously it would find that there was no orig.tar.gz archive but there was a orig directory and make the archive from that. Now you have to do it yourself and dh_make does it like that too. pidgin-musictracker Now this should of been a pretty simple stock-standard upstream upgrade. Download the file, uupdate, debuild and release. The problem is that upstream didn't fix their po files. These are the translation files that convert the raw english texts in a program into whatever language. This means the first time you run the build process, you have no diff file. However when you run it the second time, part of the install target rebuilds the po files (mainly the line references), you have differences and then a big diff file of these po updates. The real fix is to make sure your po files are up to date before you release! The trick is that they don't automatically and I have been caught myself; though it happens in the release testing phase (because my release testing involves usually building the debian archive so I see my own problem). irc came to the rescue again. Someone there suggested getting dpkg-source to ignore the changes. The way you do that is put parameters into debian/source/options. I wanted to ignore the changes to the po and pot files so my file has:
extend-diff-ignore=po/.*\.pot?
I put that in and it works!

4 February 2011

Richard Darst: DebianNYC Squeeze release party

DebianNYC is having a Squeeze release/super bowl party this Sunday. It is in Park Slope, Brooklyn, and there will be both super bowl watchers and Debian installers/hackers who didn't know the super bowl was this weekend. For more information and to RSVP, see the wiki.Someone in IRC had the bright idea to have a "upgrade to half-broken testing" party in a few weeks. This sounds a lot more fun - and educational. We have been having so many events lately that I am a bit worried about having things too close together, so I guess we'll take it as it comes and people want to organize things. In the near future, we could have a Novice Night in Brooklyn, a Novice Night in the Bronx, a workshop on configuration management of clusters, and now this testing upgrade party.

3 February 2011

Raphaël Hertzog: People behind Debian: Mike Hommey, Firefox/Iceweasel maintainer

Mike Hommey, our Iceweasel maintainer

For as long as I have known Mike, he s been maintaining one of the largest (and most widely used) package: Iceweasel, Debian s default web browser. It s effectively Mozilla s Firefox although it has been renamed to avoid some rules enforced by Mozilla s trademark policy. But we might see Firefox back in Debian read on to know Mike s plans. His story is also very interesting in that he went from simple user to Iceweasel maintainer, and now he s working for Mozilla Corporation. But you already knew that contributing to free software is a good way to grow skills and gain experience that are valuable on the job market, right? ;-) My questions are in bold, the rest is by Mike. Who are you? I am Mike Hommey. I am French, even if my name doesn t sound so. I have been a GNU/Linux user since 1995, using Debian since 1999, and started packaging in 2003. After a long time in the NM queue, I became a DD in 2005. The main reason I started being actively involved is quite selfish: I wanted some things done, and I just did them. My first packages were extensions for the Mozilla Suite, the old one that directly inherited from Netscape Communicator. Later on, I wanted some of these extensions to work with the new kid on the block: Phoenix; or maybe it was already Firebird. By then, integrating extensions was a hassle, and the mozilla-browser maintainer had come up with a script that would make it easier for extension packages. So I ported that script for use with Phoenix/Firebird and sent it to the BTS. Then Phoenix/Firebird became Firefox, and it grew a real extension manager. Except that is was entirely not suited for extensions installed by the packaging system. So I made it work better. And made a few more changes to make Firefox work better. All through patches or NMUs. Then I became co-maintainer, then Firefox couldn t keep its name and became Iceweasel, and, fast-forward until today, I m now taking care of Iceweasel/Xulrunner and Iceape in stable, unstable, experimental and an external repository. This 8 years-long story (wow, I hadn t even realized it had been that long until I wrote it) tells you something: newcomers shouldn t be afraid to stick their hands anywhere, even in big packages. I was only a Mozilla and Phoenix end-user when I started. Apart from Mozilla-related packaging, I also happen to (co-)maintain some other packages in Debian, most notably webkit, though I haven t been very active in the pkg-webkit team recently. There are several reasons for that, the main one being that there are other team members getting the job done. And I can t say that much about the pkg-mozilla team, since there are effectively no other team members. What s your biggest achievement within Debian? I am quite proud that Squeeze will be the first Debian release where Iceweasel works on all supported architectures. Previously, we only knew the package compiled on all these architectures. Starting with Squeeze, we know it passes several of the test suites coming with the source code. This ensures most of the browser is functional. It was actually far from being the case before, as it appeared when the test suites were first enabled. I think it is very important to have properly working packages on all our supported architectures. The current evolution of hardware shows how important it still is. Who would have thought a few years back that we would see ARM or MIPS based netbooks today ? What are your plans for Debian Wheezy? For Debian Wheezy, I would like to improve Iceweasel/Icedove/Iceape integration with the packaging system. Something that would allow to use the apt database to find extensions or plugins, on top of the current behaviour. I also want to improve desktop integration, most notably for KDE. Another thing I d like to see happen is to package google breakpad independently (its source code is currently embedded in chromium and iceweasel, though not built, as far as I know), and have our Mozilla products builds be able to send their crash reports upstream. It is very important that upstream gets direct visibility over what crashes on GNU/Linux systems. It helps making them aware of issues that would otherwise probably remain unnoticed, as there are much more users using distributions packages than upstream tarballs. And not all Debian users report crashes to the BTS. I know at least Fedora and SuSE are already doing it. More importantly, I want to avoid the sad situation we got with Squeeze, where we re going to release a 18 months old Iceweasel. We got there for a combination of reasons, one being that I was preparing for a freeze in December 2009 and focusing on getting 3.5 in shape instead of the then upcoming 3.6. The result was that we didn t get 3.6 packages before upstream released 3.6.3 in April 2010. By then it was too late, considering the uncertainty of the release schedule, to update reverse dependencies and get 3.6 ready for Squeeze. Things are different for 4.0. I m already getting ahead of things, and while reverse dependencies haven t been taken care of, all beta releases have been made available through http://mozilla.debian.net/, usually within hours after upstream release (sometimes even a few hours before official announcement, but always after availability on ftp.mozilla.org). A great share of our patches were also pushed upstream. Several things made that possible: There will be a couple major upstream releases until Wheezy comes along, and I want to make them available early in unstable, even if that means breaking reverse dependencies until they are fixed. Finally, I would like to get other people involved, starting with the Icedove maintainer. Volunteers are very much welcome, for any kind of involvement: bug triaging, documentation (which we are deeply lacking), testing, code People interested can contact the pkg-mozilla team list. As the Debian packaging gets less time consuming, I should be able to spend more time mentoring, which should help. If you could spend all your time on Debian, what would you work on? I actually have been working almost full time on Debian for a few months during a more-or-less purposeful unemployment period. That s when I attempted to add an alert popup when Iceweasel or extensions are upgraded (which was eventually removed because not working very effectively). That s when I worked on Iceweasel 3.6 and the first 4.0 betas. That s when I pushed more than 70 patches upstream, and got upstream commit access. I know for a fact that keeping up with upstream and answering bug reports is already time consuming, and time comes in finite quantity. Working full time on Debian allows to do more than that, as I ve been able to experience first hand. Unfortunately, that doesn t help pay the bills, so it would be best if time could be shared with other persons. Like, actual members in the pkg-mozilla team. ;) Anyways, if I were to work full time on Debian again, I d like to do more than packaging, and help improve some of our infrastructure, most notably the buildd network, to grow some automated way to get packages built on all or some of our architectures. Call that PPA if you want, though I don t think we need something exactly like PPA. We sure do have porter boxes, but there is a huge overhead to get packages built there: you need to first try, then see you lack build dependencies, then wait for admins to install them, then come back and launch your build, and finally come back again much later to see how it went. And that s when it goes smoothly. Repeat for each architecture. It s already cumbersome when it s a small package, so imagine when it s Iceweasel. I d like to be able to push a (signed) .dsc somewhere, say I want it built on that and that architecture, and get binary packages and logs back. It could then be up to developers to distribute them, or not. Sometimes, you re just interested in test suite results. Another related wish would be a way to access build trees of failed builds, possibly on the buildd where it failed. What s the biggest problem of Debian? Lack of contributors, lack of contributors, and lack of contributors. It can sound weird to say that in a community of 1000+ people. But the sad reality is that we ve seen, repeatedly, calls for help in various places from various understaffed teams (most often of effectively one member) for years. So there is obviously a problem getting people to invest time in these teams either on the long term, or at all. I know this is not a very significant metric, but it already should say a lot: I wouldn t be surprised if more than 2/3 of the number of source lines of code in Debian come from less than 5% of the number of packages in the archive, and see less than 5% of the DD/DM population involved. If someone feels like doing the actual math, I m curious to know how far I m from reality, and in what direction. We also lack manpower to be able to make our release development smoother. In a perfect world where manpower is not a scarce resource, we should work on $release+1 when $release is being frozen. Said otherwise, we should have been able to start working on Wheezy in August 2010, or even earlier. You ve been working for Mozilla Corporation for a few months. How does the work look like and what are you working on? I am actually not working full time for MoCo; I am only contracting for 4 days a week. That was a request I made to keep having enough spare time for Debian or other activities. I am working from home, and am pretty much free about when I work, as long as I work long enough. I take advantage of this freedom and the saved commuting time to live a healthier life by doing some sports in the daytime (which also happens to be cheaper). I mostly work with Taras Glek and other people on various aspects of Firefox startup performance. This involves pretty interesting low-level stuff, in which I actually got involved before being contracted. This work has nothing to do with packaging Iceweasel, though I can have a few opportunities to do things somehow related. You told in your blog that you were discussing with Mozilla how Debian could use the Firefox trademark. Can you tell us a bit about the progress you made? I have got an informal approval of the patches we are applying to the Iceweasel 4.0 beta packages. There are a few concerns about using the system cairo library, and the embedded copy of libpng needs to be used to have support for the animated PNGs that are used in the user interface and that are required in Firefox feature-set. Other than that, there are pretty much no objection on the technical side. We managed to get to some common ground as to how to manage stable updates and more generally, changes to our packages, but I m still waiting for an actual agreement draft. While I, as the maintainer, am fine with the way it should look according to my discussions with the people involved at Mozilla, I ll seek review from my fellow Debian Developers when I ll have something to show. I haven t decided yet how this will reflect on the Debian packages, though. For example, would Iceweasel stay? But I do hope Wheezy will get Firefox back. Is there someone in Debian that you admire for their contributions? Not someone, but all past, present and future release team members. This is a team that receives a lot of critics (and I indirectly gave one a few paragraphs ago), yet manages to keep doing an amazing job and its members are very committed to their task. I admire their dedication.
Thank you to Mike for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

8 comments Liked this article? Click here. My blog is Flattr-enabled.

1 February 2011

Ana Beatriz Guerrero Lopez: Debian at the Google Code-in 2010-11

As you might know, Debian was one of the 20 organizations selected to participate in the first Google Code-in. We got a moderate success since we started preparing the contest very late (I joined as mentor/admin the very first day of the contest!). Because of this, unlike other organizations, we did not manage to engage interested students from the very beginning of the Code-in. Since we did not have too many Debian tasks the first days, we did not attract many students, and thus did not have very many throughout the contest. Given Debian tasks were mostly about improving and contributing to Debian, it was quite hard having tasks that did not require having a Debian installation and that made it difficult for many students to get involved. Also, until the last two weeks we did not have any translation task that could have attracted earlier students without too much confidence in their technical and English skills. Nonetheless, Debian got 39 tasks done: With about 30 students participating and at least one of the students has keep contributing to Debian after the contest! The contest was sometimes stressful with several students wanting their tasks to be reviewed at the same time, and the date was problematic due to the winter (or summer!) holidays but still I am quite happy with the final result :D Debian will be also participating again this year in the Google Summer of Code 2011. Join the soc-coordination mailing list and stay tunned if you want to participate as mentor or student.

Cyril Brulebois: Debian XSF News #3

Time for a third Debian XSF News issue! Debian XSF News #3 It s been a year On a personal note, it s been a year to the day since I first looked into X. After having hacked on the Debian Installer to make it use X.Org instead of DirectFB, I did some heavy bug triaging, resulting in a drop in the xorg-server bug count in March. The same happened past week as written above, resulting in a second drop. In the meanwhile, the bug count remained more or less stable, since we try to reply quickly to new bugs, and since Julien Viard de Galbert does bug triaging on a weekly basis: BTS graph for xorg-server One might ask: what does maintaining X mean?

31 January 2011

Tiago Bortoletto Vaz: TXT2TAGS SAMPLE - Aurelio Jargas

Before you start reading: this post is only intended to test a plugin that I've just written to give PyBlosxom a txt2tags formatting support. The original txt2tag sample used here is available at txt2tag website. Mine with few changes is here.IntroductionWelcome to the txt2tags sample file.Here you have examples and a brief explanation of allmarks.The first 3 lines of the this file are used as headers,on the following format:
  line1: document title  line2: author name, email  line3: date, version
Lines with balanced equal signs = around are titles.Fonts and BeautifiersWe have two sets of fonts:The NORMAL type that can be improved with beautifiers.The TYPEWRITER type that uses monospaced font forpre-formatted text.We will now enter on a subtitle...BeautifiersThe text marks for beautifiers are simple, just as youtype on a plain text email message.We use double *, /, - and _ to represent bold,italic, strike and underline.The bold italic style is also supported as acombination.Pre-Formatted TextWe can put a code sample or other pre-formatted text:
    here    is     pre-formatted  //marks// are  **not**   interpreted 
And also, it's easy to put a one line pre-formattedtext:
  prompt$ ls /etc
Or use pre-formatted inside sentences.More CosmeticsSpecial entities like email (duh@somewhere.com) and URL (http://www.duh.com)are detected automagically, as long as the horizontal line:
^ thin or large v
You can also specify an explicit link with label.And remember,
A TAB in front of the line does a quotation.
More TABs, more depth (if allowed).
Nice.ListsA list of items is natural, just putting a dash ora plus at the beginning of the line.Plain ListThe dash is the default list identifier. For sublists,just add spaces at the beginning of the line. Morespaces, more sublists.The list ends with two consecutive blank lines.Numbered ListThe same rules as the plain list, just a differentidentifier (plus).
  1. one
  2. two
  3. three
    • mixed lists!
    • what a mess
      1. counting again
      2. ...
  4. four
Definition ListThe definition list identifier is a colon, followed bythe term. The term contents is placed on the next line.
orange
a yellow fruit
apple
a green or red fruit
other fruits
  • wee!
  • mixing lists
    1. again!
    2. and again!
TablesUse pipes to compose table rows and cells.Double pipe at the line beginning starts a heading row.Natural spaces specify each cell alignment.
heading 1heading 2heading 3
cell 1.1cell 1.2cell 1.3
cell 2.1cell 2.2cell 2.3
Without the last pipe, no border:
heading 1heading 2heading 3
cell 1.1cell 1.2cell 1.3
cell 2.1cell 2.2cell 2.3
Special EntitiesBecause things were too simple.ImagesThe image mark is as simple as it can be: [filename]. OtherThe handy %%date macro expands to the current date.So today is 20110206 on the ISO YYYYMMDD format.You can also specify the date format with the %? flags,as %%date(%m-%d-%Y) which gives: 02-06-2011.That's all for now.

John Goerzen: Unix Password and Authority Management

One of the things that everyone seems to do different is managing passwords. We haven t looked at that in quite some time, despite growth both of the company and the IT department. As I look to us moving some things to the cloud, and shifting offsite backups from carrying tapes to a bank to backups via the Internet, I m aware that the potential for mischief whether intentional or not is magnified. With cloud hosting, a person could, with the press of a button, wipe out the equivalent of racks of machines in a few seconds. With disk-based local and Internet-based offsite backups, the potential for malicious behavior may be magnified; someone could pretty quickly wipe out local and remote backups. Add to that the mysterious fact that many enterprise-targeted services allow only a single username/password for an account, and make no provision for ACLs to delegate permissions to others. Even Rackspace Cloud has this problem, as do their JungleDisk backup product, and many, many other offsite backup products. Amazon AWS seems to be the only real exception to this rule, and their ACL support is more than a little complicated. So one of the questions we will have to address is the balance of who has these passwords. Too many people and the probability of trouble, intentional or not, rises. Too few and productivity is harmed, and potentially also the ability to restore. (If only one person has the password, and that person is unavailable, company data may be as well.) The company does have some storage locations, including locked vaults and safe deposit boxes, that no IT people have access to. I am thinking that putting a record of passwords in those locations may be a good first step, as putting the passwords in the control of those that can t use them seems a reasonable step. But we ve been thinking of this as it pertains to our local systems as well. We have, for a number of years now, assigned a unique root password to every server. These passwords are then stored in a password-management tool, encrypted with a master password, and stored on a shared filesystem. Everyone in the department therefore can access every password. Many places where I worked used this scheme, or some variant of it. The general idea was that if root on one machine was compromised and the attacker got root s password, it would prevent the person from being able to just try that password on the other servers on the network and achieve a greater level of intrusion. However, the drawback is that we now have more servers than anyone can really remember the passwords for. So many people are just leaving the password tool running. Moreover, while the attack described above is still possible, these days I worry more about automated intrusion attempts that most likely won t try that attack vector. A couple of ways we could go may include using a single root password everywhere, or a small set of root passwords. Another option may be to not log in to root accounts at all possibly even disabling their password and requiring the use of user accounts plus sudo. This hasn t been practical to date. We don t want to make a dependency on LDAP from a bunch of machines just to be able to use root, and we haven t been using a tool such as puppet or cfengine to manage this stuff. Using such a tool is on our roadmap and could let us manage that approach more easily. But this approach has risks too. One is that if user accounts can get to root on many machines, then we re not really more secure than a standard root password. Second is that it makes it more difficult to detect and enforce password expiration and systematic password changes. I m curious what approaches other people are taking on this.

28 January 2011

John Goerzen: Slices of 2010

A lot of people look back on their year in December. I think it doesn t make sense to look back on a year until it s done. And then take a little time to let it simmer. That s my excuse for not doing this until almost February, anyhow. 2010 started with a 3-day ordeal of taking out the trash, nearly snapping the hood off my pickup while closing it, and Greek poetry. I then dived right into reading the Odyssey, and my post suggesting we read the Old Testament of the Bible like we read Greek mythology generated almost 100 comments. I guess I said something controversial I also read a number of other books last year. Our family had quite the year. In March, Terah and I got to visit Europe for the first time. A few months later, we visited New York City for the first time and I attended Debconf10. Jacob predictably loved our train trip to NYC, calling our room in the sleeper the best place of ever. Terah had thyroid surgery, and Jacob survived a tractor accident with amazingly minor injuries. But as I look back at my posts from last year, I m struck that most of them were about more everyday, ordinary events of life. Jacob loved the switchbox he and I built, Oliver somtimes acts like Yoda, and they both loved camping. I have made probably way too many posts with photos of them last year, but I m making no promises to slow down this year! Jacob and I built a computer together, and he loves bashing random words at the Linux shell. He also developed an interest in some half-broken FRS radios, and had tons of fun with them. Eventually this kindled in interest in amatuer radio in me. At the end of 2009, I finally completed my computer science degree and mused about what I might do next. I m a curious, inquisitive person that enjoys both challenge and knowledge. I thought that perhaps I would dive into reading more, or perhaps take some more college classes. Turns out I instead picked up amateur radio one of the big unexpected twists of the year. Jacob really did kindle that interest in me, and by July I had passed my technician and general exams and was on the air. I continued to study, and passed my extra class exam in October. Since then, I ve rigged up my bicycle for amateur radio operation, made some fun contacts, set up outdoor antennas, and got going with digital operation. It has been a lot of fun, and will keep providing opportunities for a challenge for me for a very long time. Also on the technical side, I made my archive of Gopherspace available for download. It s hard to cut off the list I have dozens of other things I could mention but I suppose I should end it with the Christmas gathering / piano moving episode.

Richard Darst: DebConf communication channels and decisions

This post is designed to describe communications channels and decision processes of DebConf. This is mostly designed to help new team members integrate into the team better and be able to contribute substantially. This only describes internal communication channels, not between DebConf and attendees.I should emphasize that this is the ideal process. Since time is so limited, things get rushed. You shouldn't worry about the steps above slowing you down, they are mostly what you'd do anyway if you are working transparently. Furthermore, once something has been delegated to you, just try to be public in your discussions and document what you will do. This is supposed to be a guide to help you take initiative, not slow you down.
#debconf-team (on OFTC) is the primary IRC channel. Most people idle here. Discussion and decisions often happen here, but saying things only in IRC isn't enough. IRC is ephemeral, so important information and decisions should be on the website, wiki, or mailing lists. IRC is good for getting advice from past members and keeping people up to date, and real-time communication among teams. Groups actively talking among each other usually use this channel. Not everyone can be expected to watch IRC all the time.The mailing lists, in particular debconf-team@ are for permanent records of meetings, and for getting feedback from all team members. For a decision to have weight, there should be a chance given for this list to provide feedback.The wiki is the place for permanent and organized documentation. Ideally, the wiki is a place for team members and others to go to find out just what we are doing. (Official information for attendees goes on the website, not the wiki.)Decisions in DebConf are hopefully made by delegation or rough consensus, but often time gets short and things have got to get done. Tradition and historical precedent play a big part: not everything can be re-designed every year. What follows below is a rough best-practice for a decision process, but I'm not saying that everything always is or has to be like this. Things get faster once you get more familiar with people.People first should read historical documents or chat with people and learn how things have worked in the past. People should know what you are working on, either through IRC or a mail to debconf-team@. This will get you feedback early and perhaps some additional advice and help.The group discusses and decides on some sort of proposal or course of action. This could be by IRC, lists, or whatever they need. It should be some public forum so that other people can watch and give feedback.The group's proposal is sent to list for feedback. Hopefully, they would have a preliminary write-up on the wiki so others can add to it there. Perhaps there is more than one proposal and they can be given side by side. Sending a proposal isn't just so that people can object: even if everyone thinks it looks good, it is important that there is some record of the process. You can get other opinions on the proposal here, and if it seems there is rough consensus, then you could consider it decided - just let someone know that you are implementing it.If it's not clear that there is agreement on the proposal, you should bring it up in a IRC meeting. Add it to an agenda, and we can discuss it. Discussing it at a meeting has the advantage that most people are there, so we can make sure that the idea isn't lost due to indecision. We want meetings to be efficient, so try to have at least a description on the lists first.Finally, things should be documented on the wiki or website. This is important to maintain transparency, and since this will be the first place people look for reference in future years.

Amaya Rodrigo: Indignez Vous!

If There is any sort of food for thought worth reading, this is it.
INDIGNEZ-VOUS! GET ANGRY! CRY OUT by St phane Hessel


St phane Hessel, author of Indignez-vous!
After 93 years, it is almost the final act. The end for me is not very far off any more. But it still leaves me a chance to be able to remind others of what acted as the basis of my political engagement. It was the years of resistance to the Nazi occupation -- and the program of social rights worked out 66 years ago by the National Council of the Resistance!

It is to Jean Moulin [murdered founder of the Council] that we owe, as part of this Council, the uniting of all elements of occupied France -- the movements, the parties, the labor unions -- to proclaim their membership in Fighting France, and we owe this to the only leader that it acknowledged, General de Gaulle. From London, where I had joined de Gaulle in March 1941, I learned that this Council had completed a program and adopted it on March 15th, 1944, that offered for liberated France a group of principles and values on which would rest the modern democracy of our country.

These principles and these values, we need today more than ever. It is up to us to see to it, all together, that our society becomes a society of which we are proud, not this society of immigrants without papers -- expulsions, suspicion regarding the immigrants. Not this society where they call into question social security and national retirement and health plans. Not this society where mass media are in the hands of the rich. These are things that we would have refused to give in to if we had been the true heirs of the National Council of the Resistance.









From 1945, after a dreadful drama [WWII], it was an ambitious
resurrection of society to which the remaining contingent of the Council
of the Resistance devoted itself. Let us remember them while creating
national health and pensions plans such as the Resistance wished, as its
program stipulated, "a full plan of French national health and social
security, aimed at assuring all citizens the means of existence whenever
they are unable to obtain them by a job; a retirement allowing the old
workers to finish their days with dignity."

The sources of energy, electricity, and gas, mines, the big banks, were
nationalized. Now this was as the program recommended: "... the return
to the nation of big monopolized means of production, fruits of common
labor, sources of energy, wealth from the mines, from insurance
companies and from big banks; the institution of a true economic and
social democracy involving the ousting of the big economic and financial
fiefdoms from the direction of the economy."

General interest must dominate over special interest. The just man
believes that wealth created in the realm of labor should dominate over
the power of money.

The Resistance proposed, "a rational organization of the economy
assuring the subordination of special interests to general interest, and
the emancipation of 'slaves' of the professional dictatorship that was
instituted just as in the fascist states," which had used the interim
[for two years after the war] government of the Republic as an agent.

A true democracy needs an independent press, and the Resistance
acknowledged it, demanded it, by defending "the freedom of the press,
its honor, and its independence from the State, the power of money and
foreign influence." This is what relieved restrictions on the press from
1944 on. And press freedom is definitely what is in danger today.

The Resistance called for a "real possibility for all French children to
benefit from the most advanced education," without discrimination.
Reforms offered in 2008 go contrary to this plan. Young teachers, whose
actions I support, went so far as refusing to apply them, and they saw
their salaries cut by way of punishment. They were indignant,
"disobeyed," judging these reforms too far from the ideal of the
democratic school, too much in the service of a society of commerce and
not developing the inventive and critical mind enough. 2

All the foundations of the social conquests of the Resistance are
threatened today.

The motive of the Resistance: indignation (Indignez-vous!)

Some dare to say to us that the State cannot afford the expenses of
these measures for citizens any more. But how can there be today a lack
of money to support and extend these conquests while the production of
wealth has been considerably augmented since the Liberation period when
Europe was in ruins? On the contrary, the problem is the power of money,
so much opposed by the Resistance, and of the big, boldfaced, selfish
man, with his own servants in the highest spheres of the State.

Banks, since privatized again, have proved to be concerned foremost for
their dividends and for the very high salaries of their leaders, not the
general interest. The disparity between the poorest and the richest has
never been so great, and amassing money, competition, so encouraged.

The basic motive of the Resistance was indignation!

We, the veterans of the resistance movements and combat forces of Free
France, we call on the young generation to live by, to transmit, the
legacy of the Resistance and its ideals. We say to them: Take our place,
"Indignez-vous!" [Get angry! or Cry out!].

The political, economic, intellectual leaders, and the whole society do
not have to give in, nor allow oppression by an actual international
dictatorship of the financial markets, which threatens peace and
democracy.

I wish for you all, each of you, to have your own motive for
indignation. It is precious. When something outrages you as I was
outraged by Nazism, then people become militant, strong, and engaged.
They join this current of history, and the great current of history must
continue thanks to each individual. And this current goes towards more
justice, more freedom, but not this unbridled freedom of the fox in the
henhouse. The rights contained in the UN Universal Declaration of Human
Rights of 1948 are just that, universal.

If you meet somebody who does not benefit from it, feel sorry for them
but help them to win their rights.

Two visions of history

When I try to understand what caused fascism, what made it so we were
overcome by Hitler and the Vichy [French government that collaborated
with Hitler], I tell myself that the propertied, with their selfishness,
were terrifically afraid of Bolshevik revolution. They were allowed to
lead with their fear.

But if, today as then, an active minority stands up, it will be enough;
we shall be the leavening that makes the bread rise. Certainly, the
experience of a very old person like me, born in 1917, is different from
the experience of the today's young persons. I often ask professors for
the opportunity to interact with their students, and I say to them: You
don't have the same obvious reasons to engage you. For us, to resist was
not to accept German occupation, defeat. It was comparatively simple.
Simple as what followed, decolonization. Then the war in Algeria.

It was necessary that Algeria become independent, it was obvious. As for
Stalin, we all applauded the victory of the Red Army against the Nazis
in 1943. But already we had known about the big Stalinist trials of
1935, and even if it was necessary to keep an ear open towards communism
to compensate against American capitalism, the necessity to oppose this
unbearable form of totalitarianism had established itself as an
obviousness. My long life presented a succession of reasons to outrage
me.

These reasons were born less from an emotion than a deliberate
commitment. As a young student at normal school [teachers college] I was
very influenced by Sartre, a fellow student. His "Nausea" [a novel],
"The Wall," [play], and "The Being and Nothingness" [essay] were very
important in the training of my thought. Sartre taught us, "You are
responsible as individuals." It was a libertarian message. The
responsibility of a person can not be assigned by a power or an
authority. On the contrary, it is necessary to get involved in the name
of one's responsibility as a human being.

When I entered the French Ecole Normale Superieure, Ulm Street, in Paris
in 1939, I entered it as a fervent adherent of the philosopher Hegel,
and I adhered to the thought of Maurice Merleau-Ponty. His teaching
explored concrete experience, that of the body and of its relations with
the senses, one big singular sense faced with a plurality of senses. But
my natural optimism, which wants all that is desirable to be possible,
carried me rather towards Hegel. Hegelism interprets the long history of
humanity as having a meaning: It is the freedom of man progressing step
by step. History is made of successive shocks, and the taking into
account of challenges. The history of societies thus advances; and in
the end, man having attained his full freedom, we have the democratic
state in its ideal form.

There is certainly another understanding of history. It says progress is
made by "freedom" of competition, striving for "always more"; it can be
as if living in a devastating hurricane. That's what it represented to a
friend of my father, the man who shared with him an effort to translate
into German "The Search for Time Lost" [novel] by Marcel Proust.

That was the German philosopher Walter Benjamin. He had drawn a
pessimistic view from a painting by the Swiss painter Paul Klee,
"Angelus Novus," where the face of the angel opens arms as if to contain
and push back a tempest, which he identifies with progress. For
Benjamin, who would commit suicide in September 1940 to escape Nazism,
the sense of history is the overpowering progression of disaster upon
disaster.

Indifference: the worst of attitudes

It is true the reasons to be indignant can seem today less clearly
related or the world too complex. Who's doing the ordering, who decides?
It is not always easy to differentiate between all the currents that
govern us. We are not any more dealing with a small elite whose joint
activities can be clearly seen. It is a vast world, of which we have a
feeling of interdependence.

We live in an interconnectivity as never before. But in this world there
still are intolerable things. To see them, it is well and necessary to
look, to search. I say to the young people, Search little, and that is
what you are going to find. The worst of attitudes is indifference, to
say "I can do nothing there, I'll just manage to get by." By including
yourself in that, you lose one of the essential elements that makes the
human being: the faculty of indignation and the commitment that is a
consequence of it.

They [young people] can already identify two big new challenges:

1. The huge gap which exists between the very poor and the very rich and
that does not cease increasing. It is an innovation of the 20th and 21st
centuries. The very poor in the today's world earn barely two dollars a
day. The new generation cannot let this gap become even greater. The
official reports alone should provoke a commitment.

2. Human rights and state of the planet: I had the chance after the
Liberation to join in the writing of the Universal Declaration of Human
Rights, adopted by the United Nations organization, on December 10th,
1948, in Paris at the palace of Chaillot. It was as principal private
secretary of Henry Laugier, the adjunct general-secretary of the UN, and
as and secretary of the Commission on Human Rights that I with others
was led to participate in the writing of this statement. I wouldn't know
how to forget the role in its elaboration of Ren Cassin, who was
national commissioner of justice and education in the government of Free
France in London in 1941 and won the Nobel peace prize in 1968, nor that
of Pierre Mend s-France in the Economic and Social Council, to whom the
text drafts we worked out were submitted before being considered by the
Third Committee (Social, Humanitarian and Cultural) of the General
Assembly. It was ratified by the 54 member states in session of the
United Nations, and I certified it as secretary.

It is to Ren Cassin that we owe the term "universal rights" instead of
"international rights" as offered by our American and British friends.
This [universal versus international] was key because, at the end of the
Second World War, what was at stake was to becomeereignty," which a
nation can emphasize while it devotes itself to crimes against humanity
on its own soil. Such was the case of Hitler, who felt himself supreme
and authorized to carry out a genocide. This universal statement owed
much to universal revulsion towards Nazism, fascism, and totalitarianism
-- and owes a lot, in our minds, to the spirit of the Resistance.

I had a feeling that it was necessary to move quickly so as not to be
dupes of the hypocrisy that there was in the UN membership, some whom
claimed these values already won but had no intention at all to promote
them faithfully -- claimed that we were trying to impose values on them.

I can not resist the desire to quote Article 15 of the Universal
Declaration of Human Rights (1948): "Everyone has the right to a
nationality." Article 22 says, "Everyone, as a member of society, has
the right to social security and is entitled to realization, through
national effort and international cooperation and in accordance with the
organization and resources of each State, of the economic, social and
cultural rights indispensable for his dignity and the free development
of his personality." And if this statement has a declarative scope, and
not statutory, the Declaration nevertheless has played a powerful role
since 1948. It saw colonized people take it up in their fight for
independence; it sowed minds in a battle for freedom.

I note with pleasure that in the course of last decades there has been
an increase in nongovernmental organizations (NGOs) and social movements
such as ATTAC (Association for the Taxation of Financial Transactions);

also FIDH (International Federation for Human Rights) and Amnesty
International, which are active and competitive. It is obvious that to
be effective today it is necessary to act in a network, to use all
modern means of communication.

To the young people, I say: Look around you, you will find topics that
justify your indignation facts about treatment of immigrants, of
"illegal" immigrants, of the Roma [aka Gypsies]. You will find concrete
situations that lead you to strong citizen action. Search and you shall
find!

My indignation regarding Palestine outrages by Israel [Indignez-vous!]

Today, my main indignation concerns Palestine, the Gaza Strip, and the
West Bank of Jordan. This conflict is outrageous. It is absolutely
essential to read the report by Richard Goldstone, of September 2009, on
Gaza, in which this South African, Jewish judge, who claims even to be a
Zionist, accuses the Israeli army of having committed "acts comparable
to war crimes and perhaps, in certain circumstances, crimes against
humanity" during its "Operation Cast Lead," which lasted three weeks.

I went back to Gaza in 2009 myself, when I was able to enter with my
wife thanks to our diplomatic passports, to study first-hand what this
report said. People who accompanied us were not authorized to enter the
Gaza Strip. There and in the West Bank of Jordan. We also visited the
Palestinian refugee camps set up from 1948 by the United Nations agency
UNRWA, where more than three million Palestinians expelled off their
lands by Israel wait even yet for a more and more problematical return.

As for Gaza, it is a roofless prison for one and a half million
Palestinians. A prison where people get organized just to survive.
Despite material destruction such as that of the Red Crescent hospital
by Operation Cast Lead, it is the behavior of the Gazans, their
patriotism, their love of the sea and beaches, their constant
preoccupation for the welfare of their children, who are innumerable and
cheerful, that haunt our memory. We were impressed by how ingeniously
they face up to all the scarcities that are imposed on them. We saw them
making bricks, for lack of cement, to rebuild the thousands of houses
destroyed by tanks. They confirmed to us that there had been 1400 deaths
including women, children, and oldsters in the Palestinian camp
during this Operation Cast Lead led by the Israeli army, compared to
only 50 injured men on the Israeli side. I share conclusions of the
South African judge. That Jews can, themselves, perpetrate war crimes is
unbearable. Alas, history does not give enough examples of people who
draw lessons from their own history. [The author, St phane Hessel, had
a Jewish father.]

Terrorism, or exasperation?

I know that Hamas [party of Palestine freedom fighters], which had won
the last legislative elections, could not help it that rockets were
launched on Israeli cities in response to the situation of isolation and
blockade in which Gazans exist. I think, naturally, that terrorism is
unacceptable; but it is necessary to acknowledge (from experience in
France) that when people are occupied by forces immensely superior to
their own, popular reaction cannot be altogether bloodless.

Does it serve Hamas to send rockets onto the town of Sd rot [Israeli
town across the border from Gaza]?

The answer is no. This does not serve their purpose, but they can
explain this gesture by the exasperation of Gazans. In the notion of
exasperation, it is necessary to understand violence as the regrettable
conclusion of situations not acceptable to those who are subjected them.

Thus, they can tell themselves, terrorism is a form of exasperation. And
that this "terrorism" is a misnomer. One should not have to resort to
this exasperation, but it is necessary to have hope. Exasperation is a
denial of hope. It is comprehensible, I would say almost natural, but it
still is not acceptable. Because it does not allow one to acquire
results that hope can possibly, eventually produce.

Nonviolence: the way we must learn to follow

I am persuaded that the future belongs to nonviolence, to reconciliation
of different cultures. It is by this way that humanity will have to
enter its next stage. But on this I agree with Sartre: We cannot excuse
the terrorists who throw bombs, but we can understand them. Sartre wrote
in 1947: "I recognize that violence in whatever form it may manifest
itself is a setback. But it is an inevitable setback because we are in a
world of violence. And if it is true that recourse to violence risks
perpetuating it, it is also true it is the sure means to make it stop."

To that I would add that nonviolence is a surer means of making violence
stop. One can not condone the terrorism, using Sartre or in the name of
this principle, during the war of Algeria, nor during the Munich Games
of 1972 the murder attempt made against Israeli athletes. Terrorism is
not productive, and Sartre himself would end up wondering at the end of
his life about the sense of violence and doubt its reason for being.

However, to proclaim "violence is not effective" is more important than
to know whether one must condemn or not those who devote themselves to
it. Terrorism is not effective. In the notion of effectiveness, a
bloodless hope is needed. If there is a violent hope, it is in the poem
of William Apollinaire "that hope is violent," and not in policy.

Sartre, in March 1980, within three weeks of his death, declared: "It is
necessary to try to explain why the world of today, which is horrible,
is only an instant in a long historical development, that hope always
has been one of the dominant forces in revolutions and insurrections,
and how I still feel hope as my conception of the future." [Note 5]

It is necessary to understand that violence turns its back on hope. It
is necessary to prefer to it hope, hope over violence. Nonviolence is
the way that we must learn to follow. So must the oppressors.
10

It is necessary to arrive at negotiations to remove oppression; it is
what will allow you to have no more terrorist violence. That's why you
should not let too much hate pile up.

The message of Mandela and Martin Luther King finds all its pertinence
in the world that overcame the confrontation of ideologies [e.g.,
Nazism] and conquered totalitarianism [e.g.,Hitler]. It is also a
message of hope in the capacity of modern societies to overcome
conflicts by a mutual understanding and a vigilant patience. To reach
that point is necessarily based on rights, against es, such as the
military intervention in Iraq.

We had this economic crisis, but we still did not initiate a new policy
of development. Also, the summit of Copenhagen against climatic warming
did not bring about a true policy for the preservation of the planet.

We are on a threshold between the terror of the first decade and the
possibilities of following decades. But it is necessary to hope, it is
always necessary to hope. The previous decade, that of 1990s, had been a
time of great progress. The United Nations had enough wisdom to call
conferences such as those of Rio on environment, in 1992, and that of
Beijing on women, in 1995. In September 2000, on the initiative of the
general secretary of United Nations, Kofi Annan, the 191 member
countries adopted a statement on the "eight objectives of the millennium
for development," by which they notably promised to reduce poverty in
the world by half before 2015.

My big regret is that neither Obama nor the European Union has yet
committed themselves to what should be the provision for a useful forum
bearing on the fundamental values.

Conclusion

How to conclude this call to be indignant? By saying still what, on the
occasion of the sixtieth anniversary of the program of the National
Council of the Resistance, we said on March 8th, 2004 -- we veterans of
the resistance movements and combat forces of Free France (1940-1945) --
that certainly "Nazism was conquered, thanks to the sacrifice of our
brothers and sisters of the Resistance and United Nations against
fascist barbarism. But this threat did not completely disappear, and our
anger against injustice is ever intact." [Note 6] Also, let us always be
called in "a truly peaceful insurrection against means of mass
communication that offer as a vista for our youth only the consumption
of mass trivia, contempt of the weakest and the culture, a generalized
amnesia, and the hard competition of all against all."

To those who will make the 21st century, we say with our affection:

TO CREATE IS TO RESIST; TO RESIST IS TO CREATE.

26 January 2011

Martin Pitt: Na zdrav PyGI!

(Update: Link to Tomeu s blog post, repost for planet.gnome.org) Last week I was in Prague to attend the GNOME/Python 2011 Hackfest for gobject-introspection, to which Tomeu Vizoso kindly invited me after I started working with PyGI some months ago. It happened at a place called brmlab which was quite the right environment for a bunch of 9 hackers: Some comfy couches and chairs, soldering irons, lots of old TV tubes, chips, and other electronics, a big Pirate flag, really good Wifi, plenty of Club Mate and Coke supplies, and not putting unnecessary effort into mundane things like wallpapers. It was really nice to get to know the upstream experts John (J5) Palmieri and Tomeu Vizoso (check out Tomeu s blog post for his summary and some really nice photos). When sitting together in a room, fully focussing on this area for a full week, it s so much easier to just ask them about something and getting things done and into upstream than on IRC or bugzilla, where you don t know each other personally. I certainly learned a lot this week (and not only how great Czech beer tastes :-) )! So what did I do? Application porting After already having ported four Ubuntu PyGTK applications to GI before (apport, jockey, aptdaemon, and language-selector),
my main goal and occupation during this week was to start porting a bigger PyGTK application. I picked system-config-printer, as it s two magnitudes bigger than the previous projects, exercises quite a lot more of the GTK GI bindings, and thus also exposes a lot more GTK annotation and pygobject bugs. This resulted in a new pygi s-c-p branch which has the first 100 rounds of test, break, fix iterations. It now at least starts, and you can do a number of things with it, but a lot of functionality is still broken. As a kind of finger exercise and also to check for how well pygi-convert works for small projects now, I also ported computer-janitor. This went really well (I had it working after about 30 minutes), and also led me to finally fixing the unicode vs. str mess for GtkTreeView that you got so far with Python 2.x. pygobject and GTK fixes Porting system-config-printer and computer-janitor uncovered a lot of opportunities to improve pygi-convert.sh, a big perl -e kind of script to do the mechanical grunt work of the porting process. It doesn t fix up changed signatures (such as adding missing arguments which were default arguments in PyGTK, or the ubiquitous user_data argument for signal handlers), but at least it gets a lot of namespaces, method, and constant names right. I also fixed three annotation fixes in GTK+. We also collaboratively reviewed and tested Pavel s annotation branch which helped to fix tons of problems, especially after Steve Fr cinaux s excellent reference leak fix, so if you play around with current pygobject git head, you really also have to use the current GTK+ git head. Speaking of which, if you want to port applications and always stay on top of the pygobject/GTK development without having to clutter your package system with make install s of those, it works very well to have this in your ~/.bashrc:
export GI_TYPELIB_PATH=$HOME/projects/gtk/gtk:$HOME/projects/gtk/gdk
export PYTHONPATH=$HOME/projects/pygobject
Better GVariant/GDBus support The GNOME world is moving from the old dbus-glib python bindings to GDBus, which is integrated into GLib. However, dbus-python exposed a really nice and convenient way of doing D-Bus calls, while using GDBus from Python was hideously complicated, especially for nontrivial arguments with empty or nested arrays:
from gi.repository import Gio, GLib
from gi._gi import variant_type_from_string
d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)
vb = GLib.VariantBuilder()
vb.init(variant_type_from_string('r'))
vb.add_value(GLib.Variant('s', 'test'))
vb.add_value(GLib.Variant('u', 1))
vb.add_value(GLib.Variant('s', 'gtk-ok'))
vb.add_value(GLib.Variant('s', 'Hello World!'))
vb.add_value(GLib.Variant('s', 'Subtext'))
# add an empty array
eavb = GLib.VariantBuilder()
eavb.init(variant_type_from_string('as'))
vb.add_value(eavb.end())
# add an empty dict
eavb = GLib.VariantBuilder()
eavb.init(variant_type_from_string('a sv '))
vb.add_value(eavb.end())
vb.add_value(GLib.Variant('i', 10000))
args = vb.end()
result = notify.call_sync('Notify', args, 0, -1, None)
id = result.get_child_value(0).get_uint32()
print id
So I went to making the GLib.Variant constructor work properly with nested types and boxed variants, adding Pythonic GVariant iterators and indexing (so that you can treat GVariant dictionaries/arrays/tuples just like their Python equivalents), and finally a Variant.unpack() method for converting the return value of a D-Bus call back into a native Python data type. This looks a lot friendlier now:
from gi.repository import Gio, GLib
d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)
args = GLib.Variant('(susssasa sv i)', ('test', 1, 'gtk-ok', 'Hello World!',
    'Subtext', [],  , 10000))
result = notify.call_sync('Notify', args, 0, -1, None)
id = result.unpack()[0]
print id
I also prepared another patch in GNOME#640181 which will provide the icing on the cake, i. e. handle the variant building/unpacking transparently and make the explicit call_sync() unnecessary:
from gi.repository import Gio, GLib
d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)
result = notify.Notify('(susssasa sv i)', 'test', 1, 'gtk-ok', 'Hello World!',
            'Subtext', [],  , 10000)
print result[0]
I hope that I can get this reviewed and land this soon. Thanks to our sponsors! Many thanks to the GNOME Foundation and Collabora for sponsoring this event!

Raphaël Hertzog: My Debian related goals for 2011

Like last year, here s a list of Debian related projects that I d like to tackle this year. I might not do them all, but I like to write them down, I am more productive when I have concrete objectives.
  1. Translate my Debian book into English.
    I will run a fundraising campaign with Ulule.com and if enough people are interested, I will spend a few months with Roland Mas to translate the book. Hopefully this project can be completed until the summer.
  2. Finish multiarch support in dpkg.
    I m working on this with Guillem Jover right now, thanks to Linaro who is sponsoring my work.
  3. Make deb files use XZ compression by default.
    It s a simple change in dpkg-deb and it would literally save gigabytes of space on mirrors. It s particularly important so that single CD/DVD can have a more complete set of software. #556407 (on DAK) needs to be fixed first though and a project-wide discussion is in order. Some archs might want to have a different default.
  4. Be more reactive to review/merge dpkg patches.
    While we welcome help, we have a backlog of patches sitting in the BTS and it happened several times that we failed to review/merge submitted branches in a decent time. It s very frustrating for the contributor. I already tried to improve the situation by creating the Review/Merge Queue but nobody stepped up to help review and update the patches.
    As I am getting more familiar with the C part of dpkg, I hope to be able to jump in when Guillem doesn t have the time or the interest.
  5. Implement the rolling distribution proposed as part of the CUT project and try to improve the release process.
    I really want the new rolling distribution but it will inevitably have an impact on the release process. It can t be done as a standalone project. I would also like to see progress in the way we deal with transitions (see discussion here).
  6. Work more regularly on the developers-reference.
    Hopefully I will be able to combine this with my blog writing activities, i.e. write blog articles on the topics that the developers-reference shall cover and then adapt my articles with some docbook markup.
To the above list, I shall add a few supplementary goals related to funding my Debian work:
  1. Write a 10-lesson course called Smart Package Management .
    It will delivered by email to my newsletter subscribers.
  2. Create an information product (most likely an ebook or an online training) and sell it on my blog.
    The precise topic has not yet been defined although I have a few ideas. Is there something that you would like me to teach you? Leave your suggestions in a comment.
  3. By the end of the year, have at least 1/3 of my time funded by donations and/or earnings of my information products.
    More concretely it means 700 each month or a 9-fold increase compared to the current income level (around 80 /month mostly via Flattr).
That makes up lots of challenges for this year. You can help me reach those goals, join my Debian Supporters Guild and you ll be informed every time that I start something new or when you can help in specific ways.

6 comments Liked this article? Click here. My blog is Flattr-enabled.

Francois Marier: Serving pre-compressed files using Apache

The easiest way to compress the data that is being served to the visitors of your web application is to make use of mod_deflate. Once you have enabled that module and provided it with a suitable configuration file, it will compress all releant files on the fly as it is serving them.

Given that I was already going to minify my Javascript and CSS files ahead of time (i.e. not using mod_pagespeed), I figured that there must be a way for me to serve gzipped files directly.

"Compiling" Static FilesI decided to treat my web application like a c program. After all, it starts as readable source code and ends up as an unreadable binary file.

So I created a Makefile to minify and compress all CSS and Javascript files using YUI Compressor and gzip:

all: build

build:
find static/css -type f -name "[^.]*.css" -execdir yui-compressor -o .css \;
find static/js -type f -name "[^.]*.js" -execdir yui-compressor -o .js \;
cd static/css && for f in *.css.css ; do gzip -c $$f > basename $$f .css .gz ; done
cd static/js && for f in *.js.js ; do gzip -c $$f > basename $$f .js .gz ; done

clean:
find static/css -name "*.css.css" -delete
find static/js -name "*.js.js" -delete
find static/css -name "*.css.gz" -delete
find static/js -name "*.js.gz" -delete
find -name "*.pyc" -delete

This leaves the original files intact and adds minified .css.css and .js.js files as well as minified and compressed .css.gz and .js.gz files.

How browsers advertise gzip supportThe nice thing about serving compressed content to browsers is that browsers that support receiving gzipped content (almost all of them nowadays) include the following HTTP header in their requests:
Accept-Encoding = gzip,deflate
(Incidently, if you want to test what non-gzipped enable browsers see, just browse to about:config and remove what's in the network.http.accept-encoding variable.)

Serving compressed files to clientsTo serve different files to different browsers, all that's needed is to enable Multiviews in our Apache configuration (as suggested on the Apache mailing list):

<Directory /var/www/static/css>
AddEncoding gzip gz
ForceType text/css
Options +Multiviews
SetEnv force-no-vary
Header set Cache-Control "private"
</Directory>

<Directory /var/www/static/js>
AddEncoding gzip gz
ForceType text/javascript
Options +Multiviews
SetEnv force-no-vary
Header set Cache-Control "private"
</Directory>

The ForceType directive is there to force the mimetype (as described in this solution) and to make sure that browsers (including Firefox) don't download the files to disk.

As for the SetEnv directive, it turns out that on Internet Explorer, most files with a Vary header (added by Apache) are not cached and so we must make sure it gets stripped out before the response goes out.

Finally, the Cache-Control headers are set to private to prevent intermediate/transparent proxies from caching our CSS and Javascript files, while allowing browsers to do so. If intermediate proxies start caching compressed content, they may incorrectly serve it to clients without gzip support.

25 January 2011

Paul van Tilburg: Also Giving up on Ruby Packaging

I have just sent a mail to the Debian/Ruby Extras mailing list stating that I will also give up most of my Ruby packaging efforts and reduce my involvement in the team. I am following Lucas Nussbaum in his decision. Although our reasons are slightly different, I subscribe most of what he says in his blog post. Besides that, I haven t done much Ruby programming in the past years, making me lose the motivation for the huge load of packages I am responsible for and thus subconsciously perceive that as a burden, as I feel guilty not having/making time to work on them. So, what does this mean? I will handover (co)maintainership of over 50 libraries. Many of which have dead upstream, unfortunately. Also will I be looking for someone to take over the administrative tasks of heading the team such as mailing list moderation and Alioth repository access. I will keep maintaining some programs that I still use, such as Camping, and am approachable for (mentor) uploads. Meanwhile I will continue my work on my PhD thesis, which is progressing well and should lead to my defence (hopefully) somewhere around September this year. That said, I will try to keep tinkering with Ruby and other F/OSS projects. Oh, and also to get more motivated and some inspiration

Christoph Berg: HP 2605dn not recognizing new cartridge

When my wife bought her HP 2605dn color laser, she had also ordered a set of replacement cartridges because the originals would supposedly not last very long. It turned out that they would last over two years, though. Now, the black cartridge was empty. The replacements are original HP components, but recycled/refilled second hand from a different source. I put in the black one, and the trouble started with the new drum having a big scratch that would appear in every print. After two years, returning the cartridge was not an option. As the old drum was still good, the idea was to put the new toner into the old cartridge. A bit of googling (and visual inspection) quickly revealed that you are not supposed to do this, the device doesn't have any facilities to access anything. Some post did indicate though that the cartridge actually consists of two parts that are only held together by two pivot pins and two small springs. If you pull the two parts apart a bit, you can feel the axis the parts are moving around. So, I would try to use the old drum part with the new toner container part. Removing the springs is easy (also removing the flap covering the drum, along with a tiny third spring). The two pivot pins are pretty inaccessible, but I managed to pull out both using a pointy tong. One pin is made of metal, if you need more space you can drill a small hole right next to it. The other pin is made of plastic with a small hole in it, in one case I succeeded by screwing in a little screw and pulling at it. Reassembling the parts is not so hard. I put the "new" cartridge into the printer, waited for the initialization to finish, only to find out the printer would still think the cartridge was empty and should be replaced. Duh. I then discovered a small chip attached to one corner of the drum unit, probably carrying product ID and some serial number. Luckily, it was easy to be removed and exchanged with the other drum unit. Still, the printer thought the cartridge was empty. Some (windows) driver re-installations later it was clear that there is no "ignore fill level and print anyway" option. (Cups didn't print either.) The HP Support Forum has a long Ink cartridge empty but it's not thread with lots of complaints but few answers. (Their problem is mostly with new HP cartridges becoming "empty" after a few days.) Someone suggests trying to cold reset the printer. For the 2605dn, that's holding the left and right buttons while powering on. I removed the cartridges for doing so. When I had put them back in, the printer asked me to confirm that non-HP material was installed, and subsequently didn't display any fill level for the black cartridge anymore, just " ? " in the display. Printing works now. Yay. (Lesson learned: I still hate dealing with printers.)

John Goerzen: rdiff-backup, ZFS, and rsync scripts

rdiff-backup vs. ZFS As I ve been writing about backups, I ve gone ahead and run some tests with rdiff-backup. I have been using rdiff-backup personally for many years now probably since 2002, when I packaged it up for Debian. It s a nice, stable system, but I always like to look at other options for things every so often. rdiff-backup stores an uncompressed current mirror of the filesystem, similar to rsync. History is achieved by the use of compressed backwards binary deltas generated by rdiff (using the rsync algorithm). So, you can restore the current copy very easily a simple cp will do if you don t need to preserve permissions. rdiff-backup restores previous copies by applying all necessary binary deltas to generate the previous version. Things I like about rdiff-backup:
  1. Bandwidth-efficient
  2. Reasonably space-efficient, especially where history is concerned
  3. Easily scriptable and nice CLI
  4. Unlike tools such as duplicity, there is no need to periodically run full backups old backups can be deleted without impacting the ability to restore more current backups
Things I don t like about it:
  1. Speed. It can be really slow. Deleting 3 months worth of old history takes hours. It has to unlink vast numbers of files and that s pretty much it, but it does it really slowly. Restores, backups, etc. are all slow as well. Even just getting a list of your increment sizes so you d know how much space would be saved can take a very long time.
  2. The current backup copy is stored without any kind of compression, which is not at all space-efficient
  3. It creates vast numbers of little files that take forever to delete or summarize
So I thought I would examine how efficient ZFS would be. I wrote a script that would replay the rdiff-backup history first it would rsync the current copy onto the ZFS filesystem and make a ZFS snapshot. Then each previous version was processed by my script (rdiff-backup s files are sufficiently standard that a shell script can process them), and a ZFS snapshot created after each. This lets me directly compare the space used by rdiff-backup to that used by ZFS using actual history. I enabled gzip-3 compression and block dedup in ZFS. My backups were nearly 1TB in size and the amount of space I had available for ZFS was roughly 600GB, so I couldn t test all of them. As it happened, I tested the ones that were the worst-case scenario for ZFS: my photos, music collection, etc. These files had very little duplication and very little compressibility. Plus a backup of my regular server that was reasonably compressible. The total size of the data backed up with rdiff-backup was 583 GB. With ZFS, this came to 498GB. My dedup ratio on this was only 1.05 (meaning 5% or 25GB saved). The compression ratio was 1.12 (60GB saved). The combined ratio was 1.17 (85GB saved). Interestingly 498 + 85 = 583. Remember that the data under test here was mostly a worst-case scenario for ZFS. It would probably have done better had I had the time to throw the rest of my dataset at it (such as the 60GB backup of my iPod, which would have mostly deduplicated with the backup of my music server). One problem with ZFS is that dedup is very memory-hungry. This is common knowledge and it is advertised that you need to use roughly 2GB of RAM per TB of disk when using dedup. I don t have quite that much to dedicate to it, so ZFS got VERY slow and thrashed the disk a lot after the ARC grew to about 300MB. I found some tweakables in zfsrc and the zfs command that let me tweak the ARC cache to grow bigger. But the machine in question only has 2GB RAM, and is doing lots of other things as well, so this barely improved anything. Note that this dedup RAM requirement is not out of line with what is expected from these sorts of solutions. Even if I got absolutely stellar dedup ratio of 2:1, that would get me at most 1TB. The cost of buying a 1TB disk is less than the cost of upgrading my system to 4GB RAM, so dedup isn t worth it here. I think the lesson is: think carefully about where dedup makes sense. If you re storing a bunch of nearly-identical virtual machine images the sort of canonical use case for this go for it. A general fileserver well, maybe you should just add more disk instead of more RAM. Then that raises the question: if I don t need dedup from ZFS, do I bother with it at all, or just use ext4 and LVM snapshots? I think ZFS still makes sense, given its built-in support for compression and very fast snapshots LVM snapshots are known to cause serious degradation to write performance once enabled, which ZFS doesn t. So I plan to switch my backups to use ZFS. A few observations on this:
  1. Some testing suggests that the time to delete a few months of old snapshots will be a minute or two with ZFS compared to hours with rdiff-backup.
  2. ZFS has shown itself to be more space-efficient than rdiff-backup, even without dedup enabled.
  3. There are clear performance and convenience wins with ZFS.
  4. Backup Scripts So now comes the question of backup scripts. rsync is obviously a pretty nice choice here and if used with inplace perhaps even will play friendly with ZFS snapshots even if dedup is off. But let s say I m backing up a few machines at home, or perhaps dozens at work. There is a need to automate all of this. Specifically, there s a need to:
    1. Provide scheduling, making sure that we don t hammer the server with 30 clients all at once
    2. Provide for run before jobs to do things like snapshot databases
    3. Be silent on success and scream loudly via emails to administrators on any kind of error and keep backing up other systems when there is an error
    4. Create snapshots and provide an automated way to remove old snapshots (or mount them for reading, as ZFS-fuse doesn t support the .zfs snapshot directory yet)
    To date I haven t found anything that looks suitable. I found a shell script system called rsbackup that does a large part of this, but something about using a script whose homepage is a forum makes me less than 100% confident. On the securing the backups front, rsync comes with a good-looking rrsync script (inexplicably installed under /usr/share/doc/rsync/scripts instead of /usr/bin on Debian) that can help secure the SSH authorization. GNU rush also looks like a useful restricted shell.

23 January 2011

Dirk Eddelbuettel: CRANberries is now tweeting

The CRANberries service (which reports on new and updated CRAN packages for the R language and environment) is now tweeting about new packages. Simply follow @CRANberriesFeed to receive theses messages. For the technically minded, adding this to the existing 200-line program which runs all of CRANberries was very easy. CRANberries relies only on R itself and a few Unix tools like diffstat as well as the simple blosxom txt-to-html/rss 'blog compiler'. The tweeting itself is now done by this new function
tweetNewBlogEntry <- function(curPkg, curVer, reposurl)  
    ## tests reveal that pipe(), cat(), close() is easiest
    ## to send multiple messages, may need --background option
    con <- pipe("bti --config bti.conf", "w")
    cat("New CRAN package", curPkg, "with initial version", curVer, " http://goo.gl/pgljT\n", file=con)
    close(con)
 
which simply pipes the message in Greg KH's bti program (which is now a new dependency). Special thanks to its Debian maintainer Gregor Herrmann for some helpful emails; I am using the newest release 0.29 which itself needs liboauth0. Once OAuth tokens are set-up (and see here for how to do that) all we need is the three-liner above. At this point I am not too sure what to do about updated packages. One message per updated package seems too noisy. To be seen---comments or suggestions welcome.

22 January 2011

Pau Garcia i Quiles: Request to Linux distributions

Do not waste my hard disk! I recently acquired an 2 TB hard disk drive, which I immediately formatted with ext4. Given that mkfs.ext4 defaults to 5% reserved blocks for too, that amounts to 100 GB of lost* space. Wow. 100 GB. On a desktop machine. For root. While mkfs belongs to e2fsprogs, I think this requires action on the distributions side. Let me explain. If you are installing a server, 5% makes perfect sense: there are a lot of logfiles, updates, potential break-ins and other disasters-to-happen for which root-reserved space will be preciously required. 5% is a good choice here. If 5% is too much (or too little), it will not matter: any decent sysadmin knows mkfs.ext4 -m and tune2fs -m (if you don t, Martin will teach you). On the other hand, if you are installing Ubuntu on your laptop, you probably don t know shit about mkfs, tune2fs, 5%, etc. You only know by removing Windows and installing Linux you ve magically lost 100GB of space. Vanished. So here is my request to Linux distributions: when installing a distribution in desktop profile (i. e. not Debian in server profile, RHEL, SLES, Ubuntu Server, etc), DO NOT RESERVE 5% to root. 100 or 200 MEGABYTES should be more than enough. * Yes, I know it s not really lost but just unusable by normal users but that technicality doesn t matter to 99% of the people out there.

Joey Hess: command composition

Traditionally, Unix has operated by piping between commands ("find grep sort uniq"). I wonder if we are experiencing a shift to command composition being a more common model? I started thinking about this when I considered the following command line:
chronic mr run git annex get
Unpacking, that uses chronic, which arranges for quiet running of mr (unless it fails, then it shows the full output), which in turn runs git in every repository it knows about, and git is asked to run the git-annex subcommand, which in turn has a "get" subcommand of its own. So, in other words, this is a cron job that syncs large files in a set of git repos, quietly. You might want to make it run niced also, which adds 3 more commands:
chronic nice ionice -c3 trickle -d100 mr run git annex get
This is the most extreme example I know of (unless you tack in a "sudo"), but there does seem to have been a gradual shift to using more composition, and more subcommands. In git in particular, two-levels of subcommands is not unknown, for example "git remote add". Subcommands, especially as implemented in git (or earlier, in arch), are kind of a special case of command composition that adds a namespace. I'm sure that a count of commands with subcommands would show it exploded in the past 10-12 years. I'll bet a count of composable commands would also show a significant increase. Anyone have other examples of long compositions of commands?

Debian News: Join us in the countdown to Squeeze

The release team announced Squeeze will be released the weekend of the 5th-6th of February.

Join us in the final countdown! Squeeze Countdown

The banner was made by mvd based on Valessio s theme Spacefun.You are welcome to link it from your website.

While you are waiting, you can also update your Lenny systems to 5.0.8.

Next.

Previous.