Search Results: "beh"

5 February 2011

Craig Small: JFFNMS and IPv6

ipv6-google-rrt.png One of the many Free Software projects I work on is JFFNMS, which is a network management system written in PHP. In light that the last IPv4 address blocks have now been allocated to APNIC it s probably timely to look at how to manage network devices in a new IPv6 world. First you need to get the basics sorted out and for that it is best to use the net-snmp command line utilities to check all is well. Then its onto what to do in JFFNMS itself. Now fixed with proper markup, I hope.
    strlcpy(hostname, a1, sizeof(hostname));
        if ((pptr = strchr (hostname, ':')))  
            remote_port = strtol (pptr + 1, NULL, 0);
         
Special Agent 6 All devices that are capable of being managed with SNMP have an agent. It s basically the SNMP server on the router, switch or server. The first step is to make sure it can accept and send traffic using IPv6. Depending on what the device is, it can be real simple. For example, my little Juniper router its a matter of setting the community and access control, just like IPv4 settings:
csmall@juniper1# set snmp community public clients 2001:db8:62:d0::/64
And then run snmpget on the command line to check
csmall@elmo$ snmpget -v 1 -c public udp6:2001:44b8:62:d0::4 system.sysObjectID.0
SNMPv2-MIB::sysObjectID.0 = OID: SNMPv2-SMI::enterprises.2636.1.1.1.2.41
We have success! For Linux systems that use net-snmp you will need to make the snmp daemon listen on UDPv6 ports as well as adjusting the access control. Access control is a matter of adding com2sec6 lines to /etc/snmp/snmpd.conf They are the same format as com2sec lines and are reasonably straight forward. Next, get the daemon to listen to requests on the IPv6 ports as well. The snmpd(8) man page says:
By default, snmpd listens for incoming SNMP requests on UDP port 161 on all IPv4 interfaces. However, it is possible to modify this behaviour by specifying one or more listening addresses as arguments to snmpd.
It wouldn t of killed them to put comma separated list somewhere, now would it? My /etc/default/snmpd file now looks like:
SNMPDOPTS= -Lf /dev/null -u snmp -g snmp -I -smux -p /var/run/snmpd.pid udp:127.0.0.1,udp6:[::1]:161
The bits in bold are the changes. Then its a quick check:
csmall@elmo$ snmpget -v 1 -c public udp6:[::1] system.sysObjectID.0
SNMPv2-MIB::sysObjectID.0 = OID: NET-SNMP-MIB::netSnmpAgentOIDs.10
And we have working agents! JFFNMS Changes OK, we have a working IPv6 network and snmp works across our network so we can query devices. JFFNMS uses PHP so all should be well, shouldn t it? Database Tables This is an almost classic problem with converting IPv4 to IPv6. You have 15 bytes for a IP address string 111.222.333.444 . Ipv6 can be much larger than that. JFFNMS has things like
  CREATE TABLE  hosts (
    [...]
   ip  char(20) NOT NULL default '',
A simple fix to use a char(39) or varchar(39) will do the trick. Fixing the JFFNMS code One of the most troubling problems with the host entries is that the values entered by a user can be a hostname, IP address or IP address and port separated by a colon. There are lots of bits of code that separate out the port or address by just finding the colon or use non-IPv6 aware functions like gethostbyname() that will need to be fixed. I ve got a function to check for an IPv6 address, using the inet_pton() function.
  function is_ipv6($addr)
 
  $net_addr = @inet_pton($addr);
  if ($net_addr === FALSE   strlen($net_addr) < 16)
    return FALSE;
  return TRUE;
 
PHP functions PHP still has the old IPv4 only functions, but unlike libc does not have the Address Family independent functions. The most significant absence is the replacement functions for gethostbyname(). There is dnsgetrecord() but it is quite low-level (it won t recurse lookups if you get alias or CNAME results). It s a little hit and miss in PHP-land with what functions work with IPv6 addresses and what ones do not. fsockopen() and file() do work with IPv6. PHP and SNMP To query remote devices using SNMP, JFFNMS uses the PHP SNMP functions. Unfortunately, while there is some IPv6 support in PHP, it doesn t extend to the SNMP functions. The library does it, as the net-snmp command lines use the same library, it is just that the small shim between the script and net-snmp that is PHP gets in the way. Looking at the PHP code itself you find things like
    strlcpy(hostname, a1, sizeof(hostname));
        if ((pptr = strchr (hostname, ':')))  
            remote_port = strtol (pptr + 1, NULL, 0);
         
which definitely need to be fixed for IPv6. There is also some simple address lookup going on somewhere as well which will also need to fixed. In short, JFFNMS won t be doing SNMP based queries over IPv6 until php5-snmp can do it. Reachability JFFNMS also has a different sort of query called a reachability type. It s essentially a ping from the server running JFFNMS out to the device being managed. It uses fping to do this work, but there is also a program called fping6. It s a simple matter of checking the address type and then selecting fping or fping6 to do the reachability work. The JFFNMS code soon to be pushed into git has this change in it now. Anything Else? The next stage is to find anything else that will work with IPv6. A likely candidate is the TCP and UDP port types as they use nmap to discover the ports and the fsockopen() function call to poll it. fsockopen() does handle IPv6 addresses if you escape them in square brackets.

3 February 2011

Pietro Abate: Package Managers Comparison

The Mancoosi Team has recently published the details of a study we conducted analyzing different packages managers available in debian. The goal of this study was to compare MPM (the mancoosi package manager) to other legacy solvers and try to get a big picture regarding of the state of the art. A similar study was conduct during EDOS and the results are still available here. As I wrote few months ago, MPM is a proof of concept that we wrote to test the behavior of a number of solvers that have been developed for the MISC competition in a real world scenario. These results do not show anything new w.r.t. the experience of a lot of poeple dueling daily with their machines in order to install a new piece of software. In a nutshell, we have shown that apt-get, aptitude, smart and cupt perform pretty well when used only with one baseline (for example a stable release) : this conforms with the experience of the majority of users of debian based systems. However problems start to arise when a user start mixing more then one baseline putting a lot of stress on the solver in order to find a satisfying solution. This solution of course exists, but it is cleverly hidden in the dependency structure of more then 40K packages... MPM is not as fast as other package solvers (say 10 seconds for mpm, while apt-get is able to find a solution in 3 second), but is remarkably stable. It is always able to find a satisfying solution, even in the harder cases where all the other failed. In these experiments MPM uses the postdam solver aspcud. This solver uses only GPL components and it would be a good candidate for inclusion in debian (there are actually a couple of ITP already filled for clasp and gringo). The results (with a lot of details) are published on the mancoosi website (a more detailed report is in the works). Enjoy ! During fosdem the Mancoosi team that authored this work (Roberto, Ralf, Zack and Me) will be around, so, please stop us for a chat ! And don't miss Ralf's talk !!

Rapha&#235;l Hertzog: People behind Debian: Mike Hommey, Firefox/Iceweasel maintainer

Mike Hommey, our Iceweasel maintainer

For as long as I have known Mike, he s been maintaining one of the largest (and most widely used) package: Iceweasel, Debian s default web browser. It s effectively Mozilla s Firefox although it has been renamed to avoid some rules enforced by Mozilla s trademark policy. But we might see Firefox back in Debian read on to know Mike s plans. His story is also very interesting in that he went from simple user to Iceweasel maintainer, and now he s working for Mozilla Corporation. But you already knew that contributing to free software is a good way to grow skills and gain experience that are valuable on the job market, right? ;-) My questions are in bold, the rest is by Mike. Who are you? I am Mike Hommey. I am French, even if my name doesn t sound so. I have been a GNU/Linux user since 1995, using Debian since 1999, and started packaging in 2003. After a long time in the NM queue, I became a DD in 2005. The main reason I started being actively involved is quite selfish: I wanted some things done, and I just did them. My first packages were extensions for the Mozilla Suite, the old one that directly inherited from Netscape Communicator. Later on, I wanted some of these extensions to work with the new kid on the block: Phoenix; or maybe it was already Firebird. By then, integrating extensions was a hassle, and the mozilla-browser maintainer had come up with a script that would make it easier for extension packages. So I ported that script for use with Phoenix/Firebird and sent it to the BTS. Then Phoenix/Firebird became Firefox, and it grew a real extension manager. Except that is was entirely not suited for extensions installed by the packaging system. So I made it work better. And made a few more changes to make Firefox work better. All through patches or NMUs. Then I became co-maintainer, then Firefox couldn t keep its name and became Iceweasel, and, fast-forward until today, I m now taking care of Iceweasel/Xulrunner and Iceape in stable, unstable, experimental and an external repository. This 8 years-long story (wow, I hadn t even realized it had been that long until I wrote it) tells you something: newcomers shouldn t be afraid to stick their hands anywhere, even in big packages. I was only a Mozilla and Phoenix end-user when I started. Apart from Mozilla-related packaging, I also happen to (co-)maintain some other packages in Debian, most notably webkit, though I haven t been very active in the pkg-webkit team recently. There are several reasons for that, the main one being that there are other team members getting the job done. And I can t say that much about the pkg-mozilla team, since there are effectively no other team members. What s your biggest achievement within Debian? I am quite proud that Squeeze will be the first Debian release where Iceweasel works on all supported architectures. Previously, we only knew the package compiled on all these architectures. Starting with Squeeze, we know it passes several of the test suites coming with the source code. This ensures most of the browser is functional. It was actually far from being the case before, as it appeared when the test suites were first enabled. I think it is very important to have properly working packages on all our supported architectures. The current evolution of hardware shows how important it still is. Who would have thought a few years back that we would see ARM or MIPS based netbooks today ? What are your plans for Debian Wheezy? For Debian Wheezy, I would like to improve Iceweasel/Icedove/Iceape integration with the packaging system. Something that would allow to use the apt database to find extensions or plugins, on top of the current behaviour. I also want to improve desktop integration, most notably for KDE. Another thing I d like to see happen is to package google breakpad independently (its source code is currently embedded in chromium and iceweasel, though not built, as far as I know), and have our Mozilla products builds be able to send their crash reports upstream. It is very important that upstream gets direct visibility over what crashes on GNU/Linux systems. It helps making them aware of issues that would otherwise probably remain unnoticed, as there are much more users using distributions packages than upstream tarballs. And not all Debian users report crashes to the BTS. I know at least Fedora and SuSE are already doing it. More importantly, I want to avoid the sad situation we got with Squeeze, where we re going to release a 18 months old Iceweasel. We got there for a combination of reasons, one being that I was preparing for a freeze in December 2009 and focusing on getting 3.5 in shape instead of the then upcoming 3.6. The result was that we didn t get 3.6 packages before upstream released 3.6.3 in April 2010. By then it was too late, considering the uncertainty of the release schedule, to update reverse dependencies and get 3.6 ready for Squeeze. Things are different for 4.0. I m already getting ahead of things, and while reverse dependencies haven t been taken care of, all beta releases have been made available through http://mozilla.debian.net/, usually within hours after upstream release (sometimes even a few hours before official announcement, but always after availability on ftp.mozilla.org). A great share of our patches were also pushed upstream. Several things made that possible: There will be a couple major upstream releases until Wheezy comes along, and I want to make them available early in unstable, even if that means breaking reverse dependencies until they are fixed. Finally, I would like to get other people involved, starting with the Icedove maintainer. Volunteers are very much welcome, for any kind of involvement: bug triaging, documentation (which we are deeply lacking), testing, code People interested can contact the pkg-mozilla team list. As the Debian packaging gets less time consuming, I should be able to spend more time mentoring, which should help. If you could spend all your time on Debian, what would you work on? I actually have been working almost full time on Debian for a few months during a more-or-less purposeful unemployment period. That s when I attempted to add an alert popup when Iceweasel or extensions are upgraded (which was eventually removed because not working very effectively). That s when I worked on Iceweasel 3.6 and the first 4.0 betas. That s when I pushed more than 70 patches upstream, and got upstream commit access. I know for a fact that keeping up with upstream and answering bug reports is already time consuming, and time comes in finite quantity. Working full time on Debian allows to do more than that, as I ve been able to experience first hand. Unfortunately, that doesn t help pay the bills, so it would be best if time could be shared with other persons. Like, actual members in the pkg-mozilla team. ;) Anyways, if I were to work full time on Debian again, I d like to do more than packaging, and help improve some of our infrastructure, most notably the buildd network, to grow some automated way to get packages built on all or some of our architectures. Call that PPA if you want, though I don t think we need something exactly like PPA. We sure do have porter boxes, but there is a huge overhead to get packages built there: you need to first try, then see you lack build dependencies, then wait for admins to install them, then come back and launch your build, and finally come back again much later to see how it went. And that s when it goes smoothly. Repeat for each architecture. It s already cumbersome when it s a small package, so imagine when it s Iceweasel. I d like to be able to push a (signed) .dsc somewhere, say I want it built on that and that architecture, and get binary packages and logs back. It could then be up to developers to distribute them, or not. Sometimes, you re just interested in test suite results. Another related wish would be a way to access build trees of failed builds, possibly on the buildd where it failed. What s the biggest problem of Debian? Lack of contributors, lack of contributors, and lack of contributors. It can sound weird to say that in a community of 1000+ people. But the sad reality is that we ve seen, repeatedly, calls for help in various places from various understaffed teams (most often of effectively one member) for years. So there is obviously a problem getting people to invest time in these teams either on the long term, or at all. I know this is not a very significant metric, but it already should say a lot: I wouldn t be surprised if more than 2/3 of the number of source lines of code in Debian come from less than 5% of the number of packages in the archive, and see less than 5% of the DD/DM population involved. If someone feels like doing the actual math, I m curious to know how far I m from reality, and in what direction. We also lack manpower to be able to make our release development smoother. In a perfect world where manpower is not a scarce resource, we should work on $release+1 when $release is being frozen. Said otherwise, we should have been able to start working on Wheezy in August 2010, or even earlier. You ve been working for Mozilla Corporation for a few months. How does the work look like and what are you working on? I am actually not working full time for MoCo; I am only contracting for 4 days a week. That was a request I made to keep having enough spare time for Debian or other activities. I am working from home, and am pretty much free about when I work, as long as I work long enough. I take advantage of this freedom and the saved commuting time to live a healthier life by doing some sports in the daytime (which also happens to be cheaper). I mostly work with Taras Glek and other people on various aspects of Firefox startup performance. This involves pretty interesting low-level stuff, in which I actually got involved before being contracted. This work has nothing to do with packaging Iceweasel, though I can have a few opportunities to do things somehow related. You told in your blog that you were discussing with Mozilla how Debian could use the Firefox trademark. Can you tell us a bit about the progress you made? I have got an informal approval of the patches we are applying to the Iceweasel 4.0 beta packages. There are a few concerns about using the system cairo library, and the embedded copy of libpng needs to be used to have support for the animated PNGs that are used in the user interface and that are required in Firefox feature-set. Other than that, there are pretty much no objection on the technical side. We managed to get to some common ground as to how to manage stable updates and more generally, changes to our packages, but I m still waiting for an actual agreement draft. While I, as the maintainer, am fine with the way it should look according to my discussions with the people involved at Mozilla, I ll seek review from my fellow Debian Developers when I ll have something to show. I haven t decided yet how this will reflect on the Debian packages, though. For example, would Iceweasel stay? But I do hope Wheezy will get Firefox back. Is there someone in Debian that you admire for their contributions? Not someone, but all past, present and future release team members. This is a team that receives a lot of critics (and I indirectly gave one a few paragraphs ago), yet manages to keep doing an amazing job and its members are very committed to their task. I admire their dedication.
Thank you to Mike for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

8 comments Liked this article? Click here. My blog is Flattr-enabled.

31 January 2011

David Welton: Parallelized web site creating and selling - underperformingstocks.com

I often have ideas for little web sites, stuff I just get the urge to create for the fun of it. Some of them have done rather well, like langpop.com but even that doesn't make any money, and, sadly, as I age, that is becoming a more important factor in life. So I decided to try a little experiment. I was thinking some about "value" stocks, and fiddling around with Google's stock screener, and thought of the idea of a site that highlights underperforming stocks: http://underperformingstocks.com - so far so good - I went ahead and registered the domain, and put together the beginnings of a Rails application. At the same time though, I decided I have enough things to do in my evenings and weekends, so I decided that it might be interesting to start selling the site at the same time I'm building it. I'm not really sure how it will work out: perhaps people will see that there's not much there and not bid for it. Perhaps someone likes the idea behind it and is willing to buy on that. My idea, in any case, is to keep working on it as the auction progresses. As an additional twist, there are obviously a lot of features that could be added to the site, so I'll prioritize work on features suggested by users who have bid on the site. Here's the auction listing on Flippa.com: www.underperformingstocks.com is For Sale on Flippa! Edit: I'll also be posting updates to the twitter account I created for the project: http://twitter.com/#!/underpstocks

John Goerzen: Unix Password and Authority Management

One of the things that everyone seems to do different is managing passwords. We haven t looked at that in quite some time, despite growth both of the company and the IT department. As I look to us moving some things to the cloud, and shifting offsite backups from carrying tapes to a bank to backups via the Internet, I m aware that the potential for mischief whether intentional or not is magnified. With cloud hosting, a person could, with the press of a button, wipe out the equivalent of racks of machines in a few seconds. With disk-based local and Internet-based offsite backups, the potential for malicious behavior may be magnified; someone could pretty quickly wipe out local and remote backups. Add to that the mysterious fact that many enterprise-targeted services allow only a single username/password for an account, and make no provision for ACLs to delegate permissions to others. Even Rackspace Cloud has this problem, as do their JungleDisk backup product, and many, many other offsite backup products. Amazon AWS seems to be the only real exception to this rule, and their ACL support is more than a little complicated. So one of the questions we will have to address is the balance of who has these passwords. Too many people and the probability of trouble, intentional or not, rises. Too few and productivity is harmed, and potentially also the ability to restore. (If only one person has the password, and that person is unavailable, company data may be as well.) The company does have some storage locations, including locked vaults and safe deposit boxes, that no IT people have access to. I am thinking that putting a record of passwords in those locations may be a good first step, as putting the passwords in the control of those that can t use them seems a reasonable step. But we ve been thinking of this as it pertains to our local systems as well. We have, for a number of years now, assigned a unique root password to every server. These passwords are then stored in a password-management tool, encrypted with a master password, and stored on a shared filesystem. Everyone in the department therefore can access every password. Many places where I worked used this scheme, or some variant of it. The general idea was that if root on one machine was compromised and the attacker got root s password, it would prevent the person from being able to just try that password on the other servers on the network and achieve a greater level of intrusion. However, the drawback is that we now have more servers than anyone can really remember the passwords for. So many people are just leaving the password tool running. Moreover, while the attack described above is still possible, these days I worry more about automated intrusion attempts that most likely won t try that attack vector. A couple of ways we could go may include using a single root password everywhere, or a small set of root passwords. Another option may be to not log in to root accounts at all possibly even disabling their password and requiring the use of user accounts plus sudo. This hasn t been practical to date. We don t want to make a dependency on LDAP from a bunch of machines just to be able to use root, and we haven t been using a tool such as puppet or cfengine to manage this stuff. Using such a tool is on our roadmap and could let us manage that approach more easily. But this approach has risks too. One is that if user accounts can get to root on many machines, then we re not really more secure than a standard root password. Second is that it makes it more difficult to detect and enforce password expiration and systematic password changes. I m curious what approaches other people are taking on this.

30 January 2011

Dominique Dumont: How to configure OpenSsh through a virtual filesystem

Hello My blog How about a configfs with Config::Model ? mentioned the on-going work to provide an access to configuration data through a virtual file system , i.e. through fuse. This new feature is now working since Config::Model 1.231. As an example, let s see how Fuse can be used to your ssh client. I.e. how to use config-edit command to modify your ~/.ssh/config through a virtual file system. Note that this is just an example, Fuse can also be used for other configuration withan existing model. First, you need to install Config::Model and Fuse and the fuse software. If you re on Debian/Sid, installing all this is straightforward:
$ sudo aptitude install libconfig-model-perl libfuse-perl
On other systems, you can use CPAN to install the Perl modules
$ cpanp i Fuse
$ cpanp i Config::Model
Hopefully, fuse library will be available through a package on your favorite repository. Let s start from an empty ssh configuration file. First you need to create a directory that will be used by Fuse as a mount point:
$ mkdir my_fuse_dir
Then run config-edit-ssh (which will be changed later to config-edit -applit ssh ):
$ config-edit-ssh -ui fuse -fuse_dir my_fuse_dir
Mounting config on my_fuse_dir in background.
Use command fusermount -u my_fuse_dir to unmount
Now, all parameters that can be configured in /.ssh/config are shown as files and directories in the fuse directory (the list was reduced to save electrons):
$ tree my_fuse_dir/
my_fuse_dir/
AddressFamily
BatchMode
BindAddress
ForwardX11
ForwardX11Trusted
[snip]
HashKnownHosts
Host
*
ConnectionAttempts
[snip]
User
UserKnownHostsFile
VerifyHostKeyDNS
VisualHostKey
XAuthLocation
HostbasedAuthentication
HostKeyAlgorithms
[snip] 14 directories, 119 files
Let s say that we want to configure specific settings for 2 hosts: First step is to create the repoman entry:
$ mkdir my_fuse_dir/Host/repoman
A ls command in that new directory will show a lot of files:
$ ls my_fuse_dir/Host/repoman
IdentityFile Protocol Tunnel
CheckHostIP ForwardX11 KbdInteractiveAuthentication ProxyCommand TunnelDevice
Lo and behold, one of them is ForwardX11 ! Let s check what s in there:
$ cat my_fuse_dir/Host/repoman/ForwardX11
Since the entry is new, it s still empty. Enabling X11Forward for repoman is quite easy:
$ echo 1 > my_fuse_dir/Host/repoman/ForwardX11
Let s check:
$ cat my_fuse_dir/Host/repoman/ForwardX11
1
Now, unmount the fuse directory:
$ fusermount -u my_fuse_dir
And see the modified .ssh/config file:
$ cat ~/.ssh/config
Host repoman
ForwardX11 yes
You can see that ForwardX11 is indeed set to yes for host repoman. Last but not least, you may consider that handling files and directories through shell commands is not very easy for the end user. You re right. But that s not the point. The idea is to enable programmers to access and edit configuration data with any language (not only Perl). Just like a lot of program can access kernel data through /proc and /sys, configuration data can now be accessed in a unified way through this virtual file system. May be, one day, exploring /config will be as common as exploring /sys All the best

28 January 2011

Amaya Rodrigo: Indignez Vous!

If There is any sort of food for thought worth reading, this is it.
INDIGNEZ-VOUS! GET ANGRY! CRY OUT by St phane Hessel


St phane Hessel, author of Indignez-vous!
After 93 years, it is almost the final act. The end for me is not very far off any more. But it still leaves me a chance to be able to remind others of what acted as the basis of my political engagement. It was the years of resistance to the Nazi occupation -- and the program of social rights worked out 66 years ago by the National Council of the Resistance!

It is to Jean Moulin [murdered founder of the Council] that we owe, as part of this Council, the uniting of all elements of occupied France -- the movements, the parties, the labor unions -- to proclaim their membership in Fighting France, and we owe this to the only leader that it acknowledged, General de Gaulle. From London, where I had joined de Gaulle in March 1941, I learned that this Council had completed a program and adopted it on March 15th, 1944, that offered for liberated France a group of principles and values on which would rest the modern democracy of our country.

These principles and these values, we need today more than ever. It is up to us to see to it, all together, that our society becomes a society of which we are proud, not this society of immigrants without papers -- expulsions, suspicion regarding the immigrants. Not this society where they call into question social security and national retirement and health plans. Not this society where mass media are in the hands of the rich. These are things that we would have refused to give in to if we had been the true heirs of the National Council of the Resistance.









From 1945, after a dreadful drama [WWII], it was an ambitious
resurrection of society to which the remaining contingent of the Council
of the Resistance devoted itself. Let us remember them while creating
national health and pensions plans such as the Resistance wished, as its
program stipulated, "a full plan of French national health and social
security, aimed at assuring all citizens the means of existence whenever
they are unable to obtain them by a job; a retirement allowing the old
workers to finish their days with dignity."

The sources of energy, electricity, and gas, mines, the big banks, were
nationalized. Now this was as the program recommended: "... the return
to the nation of big monopolized means of production, fruits of common
labor, sources of energy, wealth from the mines, from insurance
companies and from big banks; the institution of a true economic and
social democracy involving the ousting of the big economic and financial
fiefdoms from the direction of the economy."

General interest must dominate over special interest. The just man
believes that wealth created in the realm of labor should dominate over
the power of money.

The Resistance proposed, "a rational organization of the economy
assuring the subordination of special interests to general interest, and
the emancipation of 'slaves' of the professional dictatorship that was
instituted just as in the fascist states," which had used the interim
[for two years after the war] government of the Republic as an agent.

A true democracy needs an independent press, and the Resistance
acknowledged it, demanded it, by defending "the freedom of the press,
its honor, and its independence from the State, the power of money and
foreign influence." This is what relieved restrictions on the press from
1944 on. And press freedom is definitely what is in danger today.

The Resistance called for a "real possibility for all French children to
benefit from the most advanced education," without discrimination.
Reforms offered in 2008 go contrary to this plan. Young teachers, whose
actions I support, went so far as refusing to apply them, and they saw
their salaries cut by way of punishment. They were indignant,
"disobeyed," judging these reforms too far from the ideal of the
democratic school, too much in the service of a society of commerce and
not developing the inventive and critical mind enough. 2

All the foundations of the social conquests of the Resistance are
threatened today.

The motive of the Resistance: indignation (Indignez-vous!)

Some dare to say to us that the State cannot afford the expenses of
these measures for citizens any more. But how can there be today a lack
of money to support and extend these conquests while the production of
wealth has been considerably augmented since the Liberation period when
Europe was in ruins? On the contrary, the problem is the power of money,
so much opposed by the Resistance, and of the big, boldfaced, selfish
man, with his own servants in the highest spheres of the State.

Banks, since privatized again, have proved to be concerned foremost for
their dividends and for the very high salaries of their leaders, not the
general interest. The disparity between the poorest and the richest has
never been so great, and amassing money, competition, so encouraged.

The basic motive of the Resistance was indignation!

We, the veterans of the resistance movements and combat forces of Free
France, we call on the young generation to live by, to transmit, the
legacy of the Resistance and its ideals. We say to them: Take our place,
"Indignez-vous!" [Get angry! or Cry out!].

The political, economic, intellectual leaders, and the whole society do
not have to give in, nor allow oppression by an actual international
dictatorship of the financial markets, which threatens peace and
democracy.

I wish for you all, each of you, to have your own motive for
indignation. It is precious. When something outrages you as I was
outraged by Nazism, then people become militant, strong, and engaged.
They join this current of history, and the great current of history must
continue thanks to each individual. And this current goes towards more
justice, more freedom, but not this unbridled freedom of the fox in the
henhouse. The rights contained in the UN Universal Declaration of Human
Rights of 1948 are just that, universal.

If you meet somebody who does not benefit from it, feel sorry for them
but help them to win their rights.

Two visions of history

When I try to understand what caused fascism, what made it so we were
overcome by Hitler and the Vichy [French government that collaborated
with Hitler], I tell myself that the propertied, with their selfishness,
were terrifically afraid of Bolshevik revolution. They were allowed to
lead with their fear.

But if, today as then, an active minority stands up, it will be enough;
we shall be the leavening that makes the bread rise. Certainly, the
experience of a very old person like me, born in 1917, is different from
the experience of the today's young persons. I often ask professors for
the opportunity to interact with their students, and I say to them: You
don't have the same obvious reasons to engage you. For us, to resist was
not to accept German occupation, defeat. It was comparatively simple.
Simple as what followed, decolonization. Then the war in Algeria.

It was necessary that Algeria become independent, it was obvious. As for
Stalin, we all applauded the victory of the Red Army against the Nazis
in 1943. But already we had known about the big Stalinist trials of
1935, and even if it was necessary to keep an ear open towards communism
to compensate against American capitalism, the necessity to oppose this
unbearable form of totalitarianism had established itself as an
obviousness. My long life presented a succession of reasons to outrage
me.

These reasons were born less from an emotion than a deliberate
commitment. As a young student at normal school [teachers college] I was
very influenced by Sartre, a fellow student. His "Nausea" [a novel],
"The Wall," [play], and "The Being and Nothingness" [essay] were very
important in the training of my thought. Sartre taught us, "You are
responsible as individuals." It was a libertarian message. The
responsibility of a person can not be assigned by a power or an
authority. On the contrary, it is necessary to get involved in the name
of one's responsibility as a human being.

When I entered the French Ecole Normale Superieure, Ulm Street, in Paris
in 1939, I entered it as a fervent adherent of the philosopher Hegel,
and I adhered to the thought of Maurice Merleau-Ponty. His teaching
explored concrete experience, that of the body and of its relations with
the senses, one big singular sense faced with a plurality of senses. But
my natural optimism, which wants all that is desirable to be possible,
carried me rather towards Hegel. Hegelism interprets the long history of
humanity as having a meaning: It is the freedom of man progressing step
by step. History is made of successive shocks, and the taking into
account of challenges. The history of societies thus advances; and in
the end, man having attained his full freedom, we have the democratic
state in its ideal form.

There is certainly another understanding of history. It says progress is
made by "freedom" of competition, striving for "always more"; it can be
as if living in a devastating hurricane. That's what it represented to a
friend of my father, the man who shared with him an effort to translate
into German "The Search for Time Lost" [novel] by Marcel Proust.

That was the German philosopher Walter Benjamin. He had drawn a
pessimistic view from a painting by the Swiss painter Paul Klee,
"Angelus Novus," where the face of the angel opens arms as if to contain
and push back a tempest, which he identifies with progress. For
Benjamin, who would commit suicide in September 1940 to escape Nazism,
the sense of history is the overpowering progression of disaster upon
disaster.

Indifference: the worst of attitudes

It is true the reasons to be indignant can seem today less clearly
related or the world too complex. Who's doing the ordering, who decides?
It is not always easy to differentiate between all the currents that
govern us. We are not any more dealing with a small elite whose joint
activities can be clearly seen. It is a vast world, of which we have a
feeling of interdependence.

We live in an interconnectivity as never before. But in this world there
still are intolerable things. To see them, it is well and necessary to
look, to search. I say to the young people, Search little, and that is
what you are going to find. The worst of attitudes is indifference, to
say "I can do nothing there, I'll just manage to get by." By including
yourself in that, you lose one of the essential elements that makes the
human being: the faculty of indignation and the commitment that is a
consequence of it.

They [young people] can already identify two big new challenges:

1. The huge gap which exists between the very poor and the very rich and
that does not cease increasing. It is an innovation of the 20th and 21st
centuries. The very poor in the today's world earn barely two dollars a
day. The new generation cannot let this gap become even greater. The
official reports alone should provoke a commitment.

2. Human rights and state of the planet: I had the chance after the
Liberation to join in the writing of the Universal Declaration of Human
Rights, adopted by the United Nations organization, on December 10th,
1948, in Paris at the palace of Chaillot. It was as principal private
secretary of Henry Laugier, the adjunct general-secretary of the UN, and
as and secretary of the Commission on Human Rights that I with others
was led to participate in the writing of this statement. I wouldn't know
how to forget the role in its elaboration of Ren Cassin, who was
national commissioner of justice and education in the government of Free
France in London in 1941 and won the Nobel peace prize in 1968, nor that
of Pierre Mend s-France in the Economic and Social Council, to whom the
text drafts we worked out were submitted before being considered by the
Third Committee (Social, Humanitarian and Cultural) of the General
Assembly. It was ratified by the 54 member states in session of the
United Nations, and I certified it as secretary.

It is to Ren Cassin that we owe the term "universal rights" instead of
"international rights" as offered by our American and British friends.
This [universal versus international] was key because, at the end of the
Second World War, what was at stake was to becomeereignty," which a
nation can emphasize while it devotes itself to crimes against humanity
on its own soil. Such was the case of Hitler, who felt himself supreme
and authorized to carry out a genocide. This universal statement owed
much to universal revulsion towards Nazism, fascism, and totalitarianism
-- and owes a lot, in our minds, to the spirit of the Resistance.

I had a feeling that it was necessary to move quickly so as not to be
dupes of the hypocrisy that there was in the UN membership, some whom
claimed these values already won but had no intention at all to promote
them faithfully -- claimed that we were trying to impose values on them.

I can not resist the desire to quote Article 15 of the Universal
Declaration of Human Rights (1948): "Everyone has the right to a
nationality." Article 22 says, "Everyone, as a member of society, has
the right to social security and is entitled to realization, through
national effort and international cooperation and in accordance with the
organization and resources of each State, of the economic, social and
cultural rights indispensable for his dignity and the free development
of his personality." And if this statement has a declarative scope, and
not statutory, the Declaration nevertheless has played a powerful role
since 1948. It saw colonized people take it up in their fight for
independence; it sowed minds in a battle for freedom.

I note with pleasure that in the course of last decades there has been
an increase in nongovernmental organizations (NGOs) and social movements
such as ATTAC (Association for the Taxation of Financial Transactions);

also FIDH (International Federation for Human Rights) and Amnesty
International, which are active and competitive. It is obvious that to
be effective today it is necessary to act in a network, to use all
modern means of communication.

To the young people, I say: Look around you, you will find topics that
justify your indignation facts about treatment of immigrants, of
"illegal" immigrants, of the Roma [aka Gypsies]. You will find concrete
situations that lead you to strong citizen action. Search and you shall
find!

My indignation regarding Palestine outrages by Israel [Indignez-vous!]

Today, my main indignation concerns Palestine, the Gaza Strip, and the
West Bank of Jordan. This conflict is outrageous. It is absolutely
essential to read the report by Richard Goldstone, of September 2009, on
Gaza, in which this South African, Jewish judge, who claims even to be a
Zionist, accuses the Israeli army of having committed "acts comparable
to war crimes and perhaps, in certain circumstances, crimes against
humanity" during its "Operation Cast Lead," which lasted three weeks.

I went back to Gaza in 2009 myself, when I was able to enter with my
wife thanks to our diplomatic passports, to study first-hand what this
report said. People who accompanied us were not authorized to enter the
Gaza Strip. There and in the West Bank of Jordan. We also visited the
Palestinian refugee camps set up from 1948 by the United Nations agency
UNRWA, where more than three million Palestinians expelled off their
lands by Israel wait even yet for a more and more problematical return.

As for Gaza, it is a roofless prison for one and a half million
Palestinians. A prison where people get organized just to survive.
Despite material destruction such as that of the Red Crescent hospital
by Operation Cast Lead, it is the behavior of the Gazans, their
patriotism, their love of the sea and beaches, their constant
preoccupation for the welfare of their children, who are innumerable and
cheerful, that haunt our memory. We were impressed by how ingeniously
they face up to all the scarcities that are imposed on them. We saw them
making bricks, for lack of cement, to rebuild the thousands of houses
destroyed by tanks. They confirmed to us that there had been 1400 deaths
including women, children, and oldsters in the Palestinian camp
during this Operation Cast Lead led by the Israeli army, compared to
only 50 injured men on the Israeli side. I share conclusions of the
South African judge. That Jews can, themselves, perpetrate war crimes is
unbearable. Alas, history does not give enough examples of people who
draw lessons from their own history. [The author, St phane Hessel, had
a Jewish father.]

Terrorism, or exasperation?

I know that Hamas [party of Palestine freedom fighters], which had won
the last legislative elections, could not help it that rockets were
launched on Israeli cities in response to the situation of isolation and
blockade in which Gazans exist. I think, naturally, that terrorism is
unacceptable; but it is necessary to acknowledge (from experience in
France) that when people are occupied by forces immensely superior to
their own, popular reaction cannot be altogether bloodless.

Does it serve Hamas to send rockets onto the town of Sd rot [Israeli
town across the border from Gaza]?

The answer is no. This does not serve their purpose, but they can
explain this gesture by the exasperation of Gazans. In the notion of
exasperation, it is necessary to understand violence as the regrettable
conclusion of situations not acceptable to those who are subjected them.

Thus, they can tell themselves, terrorism is a form of exasperation. And
that this "terrorism" is a misnomer. One should not have to resort to
this exasperation, but it is necessary to have hope. Exasperation is a
denial of hope. It is comprehensible, I would say almost natural, but it
still is not acceptable. Because it does not allow one to acquire
results that hope can possibly, eventually produce.

Nonviolence: the way we must learn to follow

I am persuaded that the future belongs to nonviolence, to reconciliation
of different cultures. It is by this way that humanity will have to
enter its next stage. But on this I agree with Sartre: We cannot excuse
the terrorists who throw bombs, but we can understand them. Sartre wrote
in 1947: "I recognize that violence in whatever form it may manifest
itself is a setback. But it is an inevitable setback because we are in a
world of violence. And if it is true that recourse to violence risks
perpetuating it, it is also true it is the sure means to make it stop."

To that I would add that nonviolence is a surer means of making violence
stop. One can not condone the terrorism, using Sartre or in the name of
this principle, during the war of Algeria, nor during the Munich Games
of 1972 the murder attempt made against Israeli athletes. Terrorism is
not productive, and Sartre himself would end up wondering at the end of
his life about the sense of violence and doubt its reason for being.

However, to proclaim "violence is not effective" is more important than
to know whether one must condemn or not those who devote themselves to
it. Terrorism is not effective. In the notion of effectiveness, a
bloodless hope is needed. If there is a violent hope, it is in the poem
of William Apollinaire "that hope is violent," and not in policy.

Sartre, in March 1980, within three weeks of his death, declared: "It is
necessary to try to explain why the world of today, which is horrible,
is only an instant in a long historical development, that hope always
has been one of the dominant forces in revolutions and insurrections,
and how I still feel hope as my conception of the future." [Note 5]

It is necessary to understand that violence turns its back on hope. It
is necessary to prefer to it hope, hope over violence. Nonviolence is
the way that we must learn to follow. So must the oppressors.
10

It is necessary to arrive at negotiations to remove oppression; it is
what will allow you to have no more terrorist violence. That's why you
should not let too much hate pile up.

The message of Mandela and Martin Luther King finds all its pertinence
in the world that overcame the confrontation of ideologies [e.g.,
Nazism] and conquered totalitarianism [e.g.,Hitler]. It is also a
message of hope in the capacity of modern societies to overcome
conflicts by a mutual understanding and a vigilant patience. To reach
that point is necessarily based on rights, against es, such as the
military intervention in Iraq.

We had this economic crisis, but we still did not initiate a new policy
of development. Also, the summit of Copenhagen against climatic warming
did not bring about a true policy for the preservation of the planet.

We are on a threshold between the terror of the first decade and the
possibilities of following decades. But it is necessary to hope, it is
always necessary to hope. The previous decade, that of 1990s, had been a
time of great progress. The United Nations had enough wisdom to call
conferences such as those of Rio on environment, in 1992, and that of
Beijing on women, in 1995. In September 2000, on the initiative of the
general secretary of United Nations, Kofi Annan, the 191 member
countries adopted a statement on the "eight objectives of the millennium
for development," by which they notably promised to reduce poverty in
the world by half before 2015.

My big regret is that neither Obama nor the European Union has yet
committed themselves to what should be the provision for a useful forum
bearing on the fundamental values.

Conclusion

How to conclude this call to be indignant? By saying still what, on the
occasion of the sixtieth anniversary of the program of the National
Council of the Resistance, we said on March 8th, 2004 -- we veterans of
the resistance movements and combat forces of Free France (1940-1945) --
that certainly "Nazism was conquered, thanks to the sacrifice of our
brothers and sisters of the Resistance and United Nations against
fascist barbarism. But this threat did not completely disappear, and our
anger against injustice is ever intact." [Note 6] Also, let us always be
called in "a truly peaceful insurrection against means of mass
communication that offer as a vista for our youth only the consumption
of mass trivia, contempt of the weakest and the culture, a generalized
amnesia, and the hard competition of all against all."

To those who will make the 21st century, we say with our affection:

TO CREATE IS TO RESIST; TO RESIST IS TO CREATE.

Erich Schubert: Google Research Awards

Any Googlers reading this maybe? Anyone from MapReduce or Image Labeler would be best.
I'm looking for someone to sponsor a "research award" application for me. They're written and ready to hand in; but I've read that applications have much better chances when they have a "sponsor" at Google, so someone that is familiar with the topic that considers them to be interesting.
One is about the behaviour of large high-dimensional sets, and we just don't have access to appropriate data. A data excerpt from Google Image Labeler would help us a lot. (No budget, just some data.)
The second is about adapting some outlier detection methods to scale in linear time using a MapReduce cluster that would usually run in quadratic time and thus not be usable for many situations. The first results in this (I already have some students work on this for me) look very promising; but I'd like to have a PhD student like me to continue this work, where some aspects go beyond what is doable in the regular 6-month theses.
If you are interested, please contact me at erich AT debian org with your google address to discuss the ideas. From earlier experience I have the impression that without an interested Google contact my chances are rather low in getting accepted.

Rapha&#235;l Hertzog: 3 ways to not clutter your Debian source package with autogenerated files

It s quite common that the upstream build system generates/updates some files but does not clean them up properly when you call make clean. In that case, when you rebuild the package a second time in the same tree, the generated Debian source package will contain those changes. You usually don t want those changes. They make your package harder to review because they contain unneeded modifications (either directly in the .diff.gz with the old source format, or in a new patch in debian/patches/debian-changes-<ver> with the 3.0 (quilt) source format). I ll show you 3 ways to avoid this problem. They are all workarounds, the proper fix would be to improve the upstream build system to really clean up the generated files. This is usually possible for files that are created , but it s much more cumbersome for files that are updated (you would have to keep a backup of the original file so that you can restore it). The traditional fix Instead of relying on the upstream build system to do the work, we modify the clean target in debian/rules to remove the files that are left-over. Since debian/rules clean is always called before a source package is built, those generated files are not included as changes compared to what upstream provided. A common work-around: always build from a clean state As you have noted, the problem only happens when you build (source and binaries) twice in a row in the same tree. Some VCS-helper tools always build the Debian package in a temporary tree which is exported from the VCS. This is the case of svn-buildpackage by default and of git-buildpackage if you use its --git-export-dir option. I don t like this solution because it solves the problem only for the maintainer. Anyone else who is working on top of the package without using the same VCS-helper tool would be affected by the problem. A new way to avoid the problem Since it s now possible to store dpkg-source options in the source package itself, we can conveniently have everybody use the --extend-diff-ignore option. It tells dpkg-source to ignore some files when checking whether we have made changes to upstream files. For example if you want to ignore changes made on the files config.sub , config.guess and Makefile you could put this in debian/source/options:
# Don't store changes on autogenerated files
extend-diff-ignore = "(^ /)(config\.sub config\.guess Makefile)$"
You need to know a bit about Perl regular expressions since that s what is used by dpkg-source to match the filenames to exclude. Note that this approach always works, even when you can t remove the file. So it saves you having to make a backup of the unmodified file just to be able to restore it before the next build.
Found it useful? Be sure to not miss other packaging tips (or lessons), click here to subscribe to my free newsletter and get new articles by email.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Axel Beckert: Cool new feature in OpenSSH 5.7: scp between two remote hosts

Just a few days after OpenSSH 5.7 was released upstream, our (Debian s as well as Ubuntu s) tireless OpenSSH and GRUB maintainer Colin Watson uploaded a first package of OpenSSH 5.7 to Ubuntu Natty and to Debian Experimental. Besides the obvious new thing, the implementation of Elliptic Curve Cryptography which promises better speed and shorter keys while staying at the same level of security, one other item of his changelog entry stuck out and caught my attention: That s something I always wondered why it didn t just work . While it still doesn t seem to detect such a situation by default, it s now at least possible to copy stuff from on remote box to another without ugly port forwarding and tunneling hacks. Further cool stuff in the changelog: Colin++

23 January 2011

Russell Coker: Autism, Food, etc

James Purser wrote Stop Using Autism to Push Your Own Brand of Lunacy about the organisations that use Autism for their own political agenda [1]. He references the ongoing vaccine issue (which didn t even stop when Andrew Wakefield was proven to have engaged in fraudulent research) and a campaign of PETA.PETA s ClaimsPETA claims that many autistic kids improve dramatically when put on a diet free of dairy foods [2]. They also speculate about the possibility of Autism being caused by dairy.One thing that confuses the issue is the impact of stress on people with Autism Spectrum Disorders (ASDs). Whatever you are trying to do you will probably do it less effectively if under an excessive amount of stress, and the background stress levels of people on the Autism Spectrum tend to be quite high at the best of times. So add in some extra stress from a food allergy and the ability to emulate an NT can decrease a lot.There are people who make credible claims about significant behavior improvements in Autistic children after changing their diet. Eliminate foods that trigger a minor allergic response and the behavior of NT kids will improve too.I m sure that there are some Autistic kids who have a minor intolerance to lactose or casein who s behavior would improve if they had no dairy products. There are probably just as many Autistic kids who have a minor allergy to peanuts who would get the same benefit from avoiding peanut butter. It s not inconceivable that some Autistic people may be allergic to soy or wheat and find that they can cope better if their consume more milk, cheese, and steak and less vegetables!Also some people on the Autism Spectrum have a better sense of smell than NTs, and it s fairly common to be more fussy about the appearance of food. Force children to eat food that smells bad and their behavior might deteriorate.The issues of diet and Autism are more difficult than those of vaccines because while people just imagine the vaccines to have a bad affect, there are people who are proven to get a bad affect from certain foods.Some Ideas for ResearchI think that the biggest problem with PETA s research in this regard is citing a study of only 20 individuals and only testing the removal of milk. A proper study would involve more subjects (maybe 100) and separate tests of removing milk, eggs, wheat, rice, seeds, nuts, soy, fish, spices, and food colorings and additives.It would also test various methods of reducing stress, having a more quiet environment etc.PETA do deserve some credit for leaving this URL in the comments [3], it s a study that disproves the link between milk and Autism. But it s still a small sample size of 14.Why They Target AutismIn spite of psychologists moving to define Asperger Syndrome and Autism as the same thing, the people who make such claims aren t talking about people like me. They want to make claims about people who can t refute them, such as people who fit the Rain Man stereotype. The Low Functioning Autistic (LFA) people have less ability to communicate and are easier to ignore, so if you want to make some stuff up about a group of people then they are an easy target.Parents of LFA kids are often desperate and will latch on to any idea that seems to help. They can imagine that things have improved in spite of a lack of evidence and also fail to realise that children can improve rapidly just due to their natural development, 1 month is not much time for an adult but for a 4yo it s 2% of their life!Also any supposed research that includes testimony of parents should be regarded as rubbish. If you talk to any child psychologist they will tell you about how difficult it is to convince parents that their child is on the Autism Spectrum, a Google search for autism + denial also brings up plenty of hits. Parental denial doesn t stop at refusing to accept a diagnosis, it includes wanting to reject it at any later time based on any shred of evidence that may be available. So when there is a diet change or some other supposed cause followed by some apparently better behavior they want to claim that Autism is cured.I ve written a document titled Some Suggestions for Parents of Aspies [4] which is largely based on the childhood memories and adult experiences of various Aspies. My document covers many issues and few kids on the Spectrum will benefit from all of them, but I think that there is something there that can benefit every child on the Spectrum and probably a lot of geeky NT kids too. It seems to me that when dealing with Autistic kids it s a lot better to seek advice from adults on the Autism Spectrum than from random Neuro-Typical people (NTs), particularly if those NTs happen to have an agenda to push. The sensible parents are the ones who seek advice from psychologists and people like me rather than from random unqualified NTs.

22 January 2011

Lars Wirzenius: Obnam and btree journal dump

For some years now I have kept a private journal, sort of a private blog. It is mainly related to my work and hobby development activities, though some private life gets poured into it as well. It's a private journal. It is not meant for others to read. It has a dual purpose in life: I find this journalling activity to be beneficial to software development. Since I mainly work alone, at home, it is a bit like having someone to talk to across the desk. However, since it is recorded, it is more practical to grep things for useful stuff later on. Re-reading entries also gives me a wonderful sense of how fast or slow things happen. For example, on January 23 last year, exactly one year ago tomorrow, I wrote:
That concludes my list of next actions that should be done before obnam is theoretically able to do normal backups.
I then spent most of last year actually making Obnam be capable of doing backups for real. Recently, I thought that it might be interesting for some to read another programmer's thought processes when designing, writing, and debugging code. For this reason, I've collected some of my journal entries from the past year or so, those related to Obnam and my Python B-tree library (which Obnam uses). I have lightly sanitized them, to protect other people's privacy. Other than that, it's me as raw as you could wish. The dump is on my website. Please be gentle and laugh at me behind my back.

Enrico Zini: A prototype webby markety appy thing

A prototype webby markety appy thing What better way to introduce my work at an Application Installer meeting than to come with a prototype package browser modeled after shopping sites developed in just a few hours? It's a little Flask webapp that just works on any Debian system, using the local apt-xapian-index as a backend. It has fast keyword search, faceted navigation and screenshots, and it runs on your system showing the packages that you have available. Screenshot of packageshelf To try it:
git clone git://git.debian.org/users/enrico/pkgshelf.git
cd pkgshelf
./web-server.py
Then visit http://localhost:5000
It didn't have much interface polishing, as it's just a quick technology demo. However you can see that: Things that need doing: It's essentially JavaScript and CSS work. Anyone wants to play?

21 January 2011

Rapha&#235;l Hertzog: People behind Debian: Michael Vogt, synaptic and APT developer

Michael and his daughter Marie

Michael has been around for more than 10 years and has always contributed to the APT software family. He s the author of the first real graphical interface to APT synaptic. Since then he created software-center as part of his work for Ubuntu. Being the most experienced APT developer, he s naturally the coordinator of the APT team. Check out what he has to say about APT s possible evolutions. My questions are in bold, the rest is by Michael. Who are you? My name is Michael Vogt, I m married and have two little daughters. We live in Germany (near to Trier) and I work for Canonical as a software developer. I joined Debian as a developer in early 2000 and started to contribute to Ubuntu in 2004. What s your biggest achievement within Debian or Ubuntu? I can not decide on a single one so I will just be a bit verbose. From the very beginning I was interested in improving the package manager experience and the UI on top for our users. I m proud of the work I did with synaptic. It was one of the earliest UIs on top of apt. Because of my work on synaptic I got into apt development as well and fixed bugs there and added new features. I still do most of the uploads here, but nowadays David Kalnischkies is the most active developer. I also wrote a bunch of tools like gdebi, update-notifier, update-manager, unattended-upgrade and software-properties to make the update/install situation for the user easier to deal with. Most of the tools are written in python so I added a lot of improvements to python-apt along the way, including the initial high level apt interface and a bunch of missing low-level apt_pkg features. Julian Andres Klode made a big push in this area recently and thanks to his effort the bindings are fully complete now and have good documentation. My most recent project is software-center. Its aim is to provide a UI strongly targeted for end-users. The goal of this project is to make finding and installing software easy and beautiful. We have a fantastic collection of software to offer and software-center tries to present it well (including screenshots, instant search results and soon ratings&reviews). This builds on great foundations like aptdaemon by Sebastian Heinlein, screenshots.debian.net by Christoph Haas, ddtp.debian.org by Michael Bramer, apt-xapian-index by Enrico Zini and many others (this is what I love about free software, it usually adds , rarely takes away ). What are your plans for Debian Wheezy? For apt I would love to see a more plugable architecture for the acquire system. It would be nice to be able to make apt-get update (and the frontends that use this from libapt) be able to download additional data (like debtags or additional index file that contains more end-user targeted information). I also want to add some scripts so that apt (optionally) creates btrfs snapshots on upgrade and provide some easy way to rollback in case of problems. There is also some interesting work going on around making the apt problem resolver a more plugable part. This way we should be able to do much faster development. software-center will get ratings&reviews in the upstream branch, I really hope we can get that into Wheezy. If you could spend all your time on Debian, what would you work on? In that case I would start with a refactor of apt to make it more robust about ABI breaks. It would be possible to move much faster once this problem is solved (its not even hard, it just need to be done). Then I would add a more complete testsuite. Another important problem to tackle is to make maintainer scripts more declarative. I triaged a lot of upgrade bug reports (mostly in ubuntu though) and a lot of them are caused by maintainer script failures. Worse is that depending on the error its really hard for the user to solve the problem. There is also a lot of code duplication. Having a central place that contains well tested code to do these jobs would be more robust. Triggers help us a lot here already, but I think there is still more room for improvement. What s the biggest problem of Debian? That s a hard question :) I mostly like Debian the way it is. What frustrated me in the past were flamewars that could have been avoided. To me being respectful to each other is important, I don t like flames and insults because I like solving problems and fighting like this rarely helps that. The other attitude I don t like is to blame people and complain instead of trying to help and be positive (the difference between it sucks because it does not support $foo instead of it would be so helpful if we had $foo because it enables me to let me do $bar ). For a long time, I had the feeling you were mostly alone working on APT and were just ensuring that it keeps working. Did you also had this feeling and are things better nowadays ? I felt a bit alone sometimes :) That being said, there were great people like Eugene V. Lyubimkin and Otavio Salvador during my time who did do a lot of good work (especially at release crunch times) and helped me with the maintenance (but got interested in other area than apt later). And now we have the unstoppable David Kalnischkies and Julian Andres Klode. Apt is too big for a single person, so I m very happy that especially David is doing superb work on the day-to-day tasks and fixes (plus big project like multiarch and the important but not very thankful testsuite work). We talk about apt stuff almost daily, doing code reviews and discuss bugs. This makes the development process much more fun and healthy. Julian Andres Klode is doing interesting work around making the resolver more plugable and Christian Perrier is as tireless as always when it comes to the translations merging. I did a quick grep over the bzr log output (including all branch merges) and count around ~4300 total commits (including all revisions of branches merged). Of that there ~950 commits from me plus an additional ~500 merges. It was more than just ensuring that it keeps working but I can see where this feeling comes from as I was never very verbose. Apt also was never my only project, I am involved in other upstream work like synaptic or update-manager or python-apt etc). This naturally reduced the time available to hack on apt and spend time doing the important day-to-day bug triage, response to mailing list messages etc. One the python-apt side Julian Andres Klode did great work to improve the code and the documentation. It s a really nice interface and if you need to do anything related to packages and love python I encourage you to try it. Its as simple as:
import apt
cache = apt.Cache()
cache["update-manager"].mark_install()
cache.commit()
Of course you can do much more with it (update-manager, software-center and lots of more tools use it). With pydoc apt you can get a good overview. The apt team always welcomes contributors. We have a mailing list and a irc channel and it s a great opportunity to solve real world problems. It does not matter if you want to help triage bugs or write documentation or write code, we welcome all contributors. You re also an Ubuntu developer employed by Canonical. Are you satisfied with the level of cooperation between both projects? What can we do to get Ubuntu to package new applications developed by Canonical directly in Debian? Again a tricky question :) When it comes to cooperation there is always room for improvement. I think (with my Canonical hat on) we do a lot better than we did in the past. And it s great to see the current DPL coming to Ubuntu events and talking about ways to improve the collaboration. One area that I feel that Debian would benefit is to be more positive about NMUs and shared source repositories (collab-maint and LowThresholdNmu are good steps here). The lower the cost is to push a patch/fix (e.g. via direct commit or upload) the more there will be. When it comes to getting packages into Debian I think the best solution is to have a person in Debian as a point of contact to help with that. Usually the amount of work is pretty small as the software will have a debian/* dir already with useful stuff in it. But it helps me a lot to have someone doing the Debian uploads, responding to the bugmail etc (even if the bugmail is just forwarded as upstream bugreports :) IMO it is a great opportunity especially for new packagers as they will not have to do a lot of packaging work to get those apps into Debian. This model works very well for me for e.g. gdebi (where Luca Falavigna is really helpful on the Debian side). Is there someone in Debian that you admire for his contributions? There are many people I admire. Probably too many to mention them all. I always find it hard to single out individual people because the project as a whole can be so proud of their achievements. The first name that comes to my mind is Jason Gunthorpe (the original apt author) who I ve never met. The next is Daniel Burrows who I met and was inspired by. David Kalnischkies is doing great work on apt. From contributing his first (small) patch to being able to virtually fix any problem and adding big features like multiarch support in about a year. Sebastian Heinlein for aptdaemon. Christian Perrier has always be one of my heroes because he cares so much about i18n. Christoph Haas for screenshots.debian.net, Michael Bramer for his work on debian translated package descriptions.
Thank you to Michael for the time spent answering my questions. I hope you enjoyed reading his answers as I did. Subscribe to my newsletter to get my monthly summary of the Debian/Ubuntu news and to not miss further interviews. You can also follow along on Identi.ca, Twitter and Facebook.

4 comments Liked this article? Click here. My blog is Flattr-enabled.

20 January 2011

John Goerzen: Research on deduplicating disk-based and cloud backups

Yesterday, I wrote about backing up to the cloud. I specifically was looking at cloud backup services. I ve been looking into various options there, but also various options for disk-based backups. I d like to have both onsite and offsite backups, so both types of backup are needed. Also, it is useful to think about how the two types of backups can be combined with minimal overhead. For the onsite backups, I d want to see:
  1. Preservation of ownership, permissions, etc.
  2. Preservation of symlinks and hardlinks
  3. Space-efficient representation of changes ideally binary deltas or block-level deduplication
  4. Ease of restoring
  5. Support for backing up Linux and Windows machines
Deduplicating Filesystems for Local Storage Although I initially thought of block-level deduplicating file systems as something to use for offsite backups, they could also make an excellent choice for onsite disk-based backups. rsync-based dedup backups One way to use them would be to simply rsync data to them each night. Since copies are essentially free, we could do (or use some optimized version of) cp -r current snapshot/2011-01-20 or some such to save off historic backups. Moreover, we d get dedup both across and within machines. And, many of these can use filesystem-level compression. The real upshot of this is that the entire history of the backups can be browsed as a mounted filesystem. It would be fast and easy to find files, especially when users call about that file that they deleted at some point in the past but they don t remember when, exactly what it was called, or exactly where it was stored. We can do a lot more with find and grep to locate these things than we could do with tools in Bacula (or any other backup program) restore console. Since it is a real mounted filesystem, we could also do fun things like make tarballs of it at will, zip parts up, scp them back to the file server, whatever. We could potentially even give users direct access to their files to restore things they need for themselves. The downside of this approach is that rsync can t store all the permissions unless it s running as root on the system. Wrappers such as rdup around rsync could help with that. Another downside is that there isn t a central scheduling/statistics service. We wouldn t want the backup system to be hammered by 20 servers trying to send it data at once. So there d be an element of rolling our own scripts, though not too bad. I d have preferred not to authorize a backup server with root-level access to dozens of machines, but may be inescapable in this instance. Bacula and dedup The other alternative I thought of system such as Bacula with disk-based volumes . A Bacula volume is normally a tape, but Bacula can just write them to disk files. This lets us use the powerful Bacula scheduling engine, logging service, pre-backup and post-backup jobs, etc. Normally this would be an egregious waste of disk space. Bacula, like most tape-heritage programs, will write out an entire new copy of a file if even one byte changes. I had thought that I could let block-level dedupe reduce the storage size of Bacula volumes, but after looking at the Bacula block format spec, this won t be possible as each block will have timestamps and such in it. The good things about this setup revolve around using the central Bacula director. We need only install bacula-fd on each server to be backed up, and it has a fairly limited set of things it can do. Bacula already has built-in support for defining simple or complicated retention policies. Its director will email us if there is a problem with anything. And its logs and catalog are already extensive and enable us to easily find out things such as how long backups take, how much space they consume, etc. And it backs up Windows machines intelligently and comprehensively in addition to POSIX ones. The downsides are, of course, that we don t have all the features we d get from having the entire history on the filesystem all at once, and far less efficient use of space. Not only that, but recovering from a disaster would require a more extensive bootstrapping process. A hybrid option may be possible: automatically unpacking bacula backups after they ve run onto the local filesystem. Dedupe should ensure this doesn t take additional space if the Bacula blocksize aligns with the filesystem blocksize. This is certainly not a given however. It may also make sense to use Bacula for Windows and rsync/rdup for Linux systems. This seems, however, rather wasteful and useless. Evaluation of deduplicating filesystems I set up and tested three deduplicating filesystems available for Linux: S3QL, SDFS, and zfs-fuse. I did not examine lessfs. I ran a similar set of tests for each:
  1. Copy /usr/bin into the fs with tar -cpf - /usr/bin tar -xvpf - -C /mnt/testfs
  2. Run commands to sync/flush the disk cache. Evaluate time and disk used at this point.
  3. Rerun the tar command, putting the contents into a slightly different path in the test filesystem. This should consume very little additional space since the files will have already been there. This will validate that dedupe works as expected, and provide a hint about its efficiency.
  4. Make a tarball of both directories from the dedup filesystem, writing it to /dev/zero (to test read performance)
I did not attempt to flush read caches during this, but I did flush write caches. The test system has 8GB RAM, 5GB of which was free or in use by a cache. The CPU is a Core2 6420 at 2.13GHz. The filesystems which created files atop an existing filesystem had ext4 mounted noatime beneath them. ZFS was mounted on an LVM LV. I also benchmarked native performance on ext4 as a baseline. The data set consists of 3232 files and 516MB. It contains hardlinks and symlinks. Here are my results. Please note the comments below as SDFS could not accurately complete the test.
Test ext4 S3QL SDFS zfs-fuse
First copy 1.59s 6m20s 2m2s 0m25s
Sync/Flush 8.0s 1m1s 0s 0s
Second copy+sync N/A 0m48s 1m48s 0m24s
Disk usage after 1st copy 516MB 156MB 791MB 201MB
Disk usage after 2nd copy N/A 157MB 823MB 208MB
Make tarball 0.2s 1m1s 2m22s 0m54s
Max RAM usage N/A 150MB 350MB 153MB
Compression none lzma none gzip-2
It should be mentioned that these tests pretty much ruled out SDFS. SDFS doesn t appear to support local compression, and it severely bloated the data store, which was much larger than the original data. Moreover, it permitted any user to create and modify files, even if the permissions bits said that the user couldn t. tar gave many errors unpacking symlinks onto the SDFS filesystem, and du -s on the result threw up errors as well. Besides that, I noted that find found 10 fewer files than in my source data. Between the huge memory consumption, the data integrity concerns, and inefficient disk storage, SDFS is out of the running for this project. S3QL is optimized for storage to S3, though it can also store its files locally or on an sftp server a nice touch. I suspect part of its performance problem stems from being designed for network backends, and using slow compression algorithms. S3QL worked fine, however, and produced no problems. Creating a checkpoint using s3qlcp (faster than cp since it doesn t have to read the data from the store) took 16s. zfs-fuse appears to be the most-used ZFS implementation on Linux at the moment. I set up a 2GB ZFS pool for this test, and set dedupe=on and compress=gzip-2. When I evaluated compression in the past, I hadn t looked at lzjb. I found a blog post comparing lzjb to the gzip options supported by zfs and wound up using gzip-2 for this test. ZFS really shone here. Compared to S3QL, it took 25s instead of over 6 minutes to copy the data over and took only 28% more space. I suspect that if I selected gzip -9 compression it would have been closer both in time and space to S3QL. But creating a ZFS snapshot was nearly instantaneous. Although ZFS-fuse probably doesn t have as many users as ZFS on Solaris, still it is available in Debian, and has a good backing behind it. I feel safer using it than I do using S3QL. So I think ZFS wins this comparison. I spent quite some time testing ZFS snapshots, which are instantaneous. (Incidentally, ZFS-fuse can t mount them directly as documented, so you create a clone of the snapshot and mount that.) They worked out as well as could be hoped. Due to dedupe, even deleting and recreating the entire content of the original filesystem resulted in less than 1MB additional storage used. I also tested creating multiple filesystems in the zpool, and confirmed that dedupe even works between filesystems. Incidentally wow, ZFS has a ton of awesome features. I see why you OpenSolaris people kept looking at us Linux folks with a sneer now. Only our project hasn t been killed by a new corporate overlord, so guess that maybe didn t work out so well for you <grin>. The Cloud Tie-In This discussion leaves another discussion: what to do about offsite backups? Assuming for the moment that I want to back them up over the Internet to some sort of cloud storage facility, there are about 3 options:
  1. Get an Amazon EC2 instance with EBS storage and rsync files to it. Perhaps run ZFS on that thing.
  2. Use a filesystem that can efficiently store data in S3 or Cloud Files (S3QL is the only contender here)
  3. Use a third-party backup product (JungleDisk appears to be the leading option)
There is something to be said for using a different tool for offsite backups if there is some tool-level issue, that could be helpful. One of the nice things about JungleDisk is that bandwidth is free, and disk is the same $0.15/GB-mo that RackSpace normally charges. JungleDisk also does block-level dedup, and has a central management interface. This all spells nice for us. The only remaining question would be whether to just use JungleDisk to back up the backup server, or to put it on each individual machine as well. If it just backs up the backup server, then administrative burdens are lower; we can back everything there up by default and just not worry about it. On the other hand, if there is a problem with our main backups, we could be really stuck. So I d say I m leaning towards ZFS plus some sort of rsync solution and JungleDisk for offsite. I had two people suggest CrashPlan Pro on my blog. It looks interesting, but is a very closed product which makes me nervous. I like using standard tools and formats gives me more peace of mind, control, and recovery options. CrashPlan Pro supports multiple destinations and says that they do cloud hosting, but don t list pricing anywhere. So I ll probably not mess with it. I m still very interested in what comments people may have on all this. Let me know!

19 January 2011

Robert Millan: Finally, amd32 is taking shape

When AMD launched its now widely used 64-bit architecture in year 2000, it started marketing it as a significant step forward because of it having longer word and pointer size. One just has to check the advertising material and notice those big 64 being touted as the main improvement. But it is commonly accepted that most applications don t need a 64-bit address space for anything. Building them in LP64 model is just a waste of memory due to increase of pointer size. Even though this made most applications lag behind, the new architecture still was an improvement in terms of speed because of the AMD revised ISA, featuring changes like: To summarize, most of the merit from AMD64 architecture was in fixing some of the insanity of Intel 386 ISA (instruction set architecture), rather than the increased pointer size which was a source of inefficiency most of the time. Perhaps AMD didn t evaluate this correctly, or perhaps its marketing side won over technical merit. However, although the new hardware is biased towards LP64 data model, it s not actually enforced. It was a matter of time until an independent project took over and attempted to fix this, combining the AMD64 ISA with ILP32 data model. I ve been reading with much interest in the binutils and gcc mailing lists that such project is beginning to take shape. A port of binutils, GCC, GDB and Linux is already available. Future plans include porting Glibc which will make it possible to build a standard GNU derivative out of this. Sadly, I don t have the time to devote to this project myself, but I ll continue following its progress. I m looking forward to bringing this speed boost to my machines.

18 January 2011

Russell Coker: Conferences and People on the Autism Spectrum

There have been some recent discussions about issues related to the treatment of women at Free Software conferences, I ve written posts about Aspie Social Skills and Free Software [1] and Empathy, Autism, and Geeks [2] about this. But of course discussion continued on how Aspies supposedly cause problems that no-one seems to be noticing.Lisa of Chaotic Idealism wrote an interesting post about the NT social bias [3]. In summary NTs seem to think that Autism Spectrum Disorders are only about socialisation, not realising that for many people on the Spectrum it s sensory issues that are their main problem. Social problems are exacerbated by sensory issues and other causes of stress, so I think it s worth considering ways in which conferences can be planned to be less stressful for people on the Autism Spectrum, people who have SPD [4] that isn t associated with an ASD, and NTs who just get annoyed by loud noises etc.What Autism Conferences DoAutscape is one of only two conferences for people on the Autism Spectrum and the only one which clearly documents how they plan their conference [5]. The first noteworthy thing that they do is have badge colors to indicate what level of social interaction is desired by each delegate, I don t think that this is relevant to Free Software conferences as people who don t have sufficient social skills to suit at least a green badge probably won t be attending. But I think that when attending a conference about Free Software or any other equally geeky topic (if there is an equally geeky topic) it s worth keeping in mind the fact that there are probably a lot of people who would like to talk to you but lack the social skills to start a conversation.
No initiationRedPlease do not initiate any interaction with me.
Prior PermissionYellowPlease do not initiate unless I have already given you permission to approach me on a yellow badge.
Please initiateGreenI would like to socialise, but I have difficulty initiating. Please initiate with me.
NeutralWhite (or no badge)I am able to regulate my own interaction.
The next thing that they document is a black circle badge which indicates that the wearer shouldn t be photographed. Prior to reading that web site I wasn t aware of this being an Autistic issue, I was only aware of it being an issue for women who don t want zoomed-in pictures of themselves appearing on guys web sites (even pictures that aren t up-skirt or down-blouse can be unwelcome). A conference policy that prohibited photographs that zoom in on one person without that person s consent (or parental consent in the case of minors) would probably be a good idea.Another thing about badges is that it s a really good idea to have the delegate s name on both sides of the badge if the badge is attached in a way that permits it to turn. People on the Autism Spectrum tend to have some difficulty in recognising people and in remembering names. I find it inconvenient when someone expects me to recognise them but has their badge turned around so I can t see their name. Some people get really unhappy if they think that someone doesn t recognise them.Autscape has long breaks between activities and a leisure session each afternoon to allow delegates to recover from the stress of dealing with people. It seems to me that computer conferences in some cases could do with longer breaks between sessions. I find that a lot of the benefit of a conference is in what happens outside sessions and the standard practice of publishing videos of presentations makes personal meetings a more important part of the conference.Autscape has designated quiet spaces. I think that for computer conferences which have hack-labs (which seems to be most Free Software conferences nowadays even if they aren t labelled as such) it would be good to have some lab areas designated as quiet zones. I think that it is a really good thing to meet people you ve only known by email and then play some LAN based games against them and that this should be encouraged as part of a conference, but having that sort of thing separated from people who want to do some quiet coding is a really good idea. Whether people want to do quiet coding, read email, or just escape from the stress of a conference a quiet hack-lab would be a good place for it.An issue that s related to quiet spaces is the ability to escape from social situations. One of my pet hates is corporate meetings on boats, when the boat is in motion there is no escape. Corporate meetings that are only accessible by coach are also bad. Fortunately most conferences aren t like that.The Autscape web site states that they prohibit people from wearing perfume or aftershave to help people who are sensitive to smell. My observation of Free Software conferences is that encouraging everyone to have a shower every morning would be a good idea as there are stronger smells from unwashed people than from perfume.One thing that s interesting about the Autscape web site is that they have different color schemes available and have tested it in multiple web browsers including Safari and Konqueror (which apparently don t work so well). It s interesting to note that they test with such a variety of browsers including free software ones most corporations don t do that.Sensory Processing IssuesThe main SPD issues related to conferences seem to be noise and light related.For social events one difficulty that most people on the Spectrum seem to face is in listening to one person in a noisy crowded room. As well as that sudden noises and loud background noises can be very stressful. While it s sometimes impossible to avoid crowds (which can be another problem) it is often possible to select venues that have less noise problems. A venue with carpet on the floor and soft walls (anything other than brick or concrete) will be a lot quieter than one with hard surfaces that reflect noise. It seems that a quiet venue will benefit NTs as well, there seems to be a strong correlation between the price of food at restaurants and the amount of sound absorbing material on the floor and walls people who can afford a good dining experience seem to want it quiet.In terms of visual issues the relevant problems seem to be related to sudden transitions and lecture halls that are extremely dark. The only thing that can be done by conference organisers is to seek to have the lights in the lecture halls as bright as possible without preventing the viewing of the projector screen, this is usually done anyway.For social events the whole binge drinking at crowded bars thing doesn t work too well due to noise, poor lighting, crowds, and the smell of vomit. But there s no shortage of reasons to discourage binge drinking at conferences.FoodPsychology Today has a good article about preferences for food and picky eaters [6]. Lots of people on the Autism Spectrum have similar issues. The thing to do when arranging a conference meal is to have things neat and without needless combinations. Think about making food look more like what you would expect to see in a Japanese restaurant and less like Paella. Also having some very plain food on offer is a good thing, I think that bread-rolls for dinner and ice-cream for desert makes a viable meal. But any dish with a word like hash or mixed in it s name isn t a good option. Finally some combinations are really bad, I always find rare steak and potatoes on the same plate to be rather disgusting potato that s blood-stained doesn t appeal. The majority of conference meals satisfy these criteria.Autscape provides options of eating outside the main dining hall for people who can t tolerate the noise. For a Free Software conference it might be a better idea to provide seating outside the main area for people who are going to make noise. I don t mind people who want to get really drunk at the conference dinner, but I would prefer them to be in a separate room. Also when assisting a drunk friend to leave the venue before they cause more problems it would be convenient if there was a good place to take them to. The one occasion when I had to strongly encourage a friend to leave a conference dinner to reduce his embarrassment the next day (and reduce the annoyance for everyone else) there was no good place to take him so he just came back!ConclusionI don t expect that anyone will make any significant changes to conference plans after reading this post. I have tried to focus on things that would benefit NTs as well, but there are practical issues that get in the way such as the cost and availability of venues that are large enough. But there are some smaller ideas that can be implemented with relative ease.I hope that the people who are making claims about Autism and conference behavior will refer to this in future. If you think that there is a problem with the way people on the Spectrum act at conferences then the solution will more likely involve the suggestions I make here than anything else.

17 January 2011

Joey Hess: the conversion

It's summer in Spain, and outside there's a dry heat, but in the hack lab, it's frigid. I'm sitting across the table from Otavio, and each of our laptops is showing the same identical screen of text, editing the same file. Slowly. The file is located in Colorado, and we're fighting serious conference lag, plus transatlantic lag. Type a few lines, wait a minute, hope it shows up, and restart the converter to see if it worked. We're struggling with figuring out how to make it work, and pair programming across the table like this helps -- although we also struggle with each others accents sometimes, so we're eventually mostly communicating via text on the computers. Eventually Otavio rolls off to bed. Later, everyone else in the conference goes to bed or whatever -- I don't really notice. At some point a cleaning woman comes in, and seems suprised to see me. I muster up enough Spanish to ask if this muy fio air conditioning can be turned off. A bit later, I make this commit and then go out, look at storks in early morning light, and go to bed.
r59731   joeyh   2009-07-24 12:30:52 -0400 (Fri, 24 Jul 2009)   1 line
up to r15000; also skip the packages/po accient revs
That would have been 6 am local time. The log shows us back at it by 11 am local. There's so much detail in these logs, and it's hard to tell how any individual bit will turn out useful, but the value in aggregate is undeniable and that's why we struggle to retain it as technology marches on.
A few days later and the conversion is caught up to the commits documenting it. Everyone goes home. Later still, and work is done to verify the conversion. Problems are found, fixed. The worst of them involves the Danish language, and absolute evil. Tempers flare, and I have the last argument I'll ever have the pleasure of having with Frans. At some point in there, I do rather more coding in C++ than I ever have before, or ever wanted to. Frans's objections are addressed by that, more or less. We're using a weird tool that was developed as a kind of one-off for another project, but happens to meet our needs pretty well -- but we're stuck dealing with its bugs. And its need for copious handholding and historical research. The file we started out editing grows to two thousand lines, and now covers ten years of odd bits of history.
Suddenly it's a year and a half later. I'm in a cabin in winter, Otavio is down in summer, and I'm once again dealing with lag (dialup lag this time). We run the converter again -- one last time. And a time more.. and four or five more "last times". Finally, after a longish day, it's done.
This was not a major project, just some little bits of time here and there, maybe a week's worth in total, spread out over a few years and four or five people. It hardly seems worth writing about, just another codebase converted from Subversion to Git, a bit behind most of the other ones. Just thought I'd give you a glimpse behind the curtain.

16 January 2011

Russell Coker: Links January 2011

Halla Tomasdottir gave an interesting TED talk about her financial services company which uses feminine values and thus survived the Icelandic financial crash [1]. She also has a lot of other interesting ideas about how to run a corporation or a country.Hillary Clinton gave an interesting TED talk about national security and women s rights [2]. The US State Department is now making women s rights a priority due to the fact that places where women are mistreated tend to pose a security threat to the US. Unfortunately this talk can t be viewed without flash.Richard Dawkins reads some of his hate-mail while sitting by the fire [3], it s interesting to know that a loving God wants his minions to send hate-mail to scientists. There are a few cheap laughs here.Emily Pilloton gave an interesting TED talk about fixing some of the problems with the education system in rural USA by incorporating design [4]. Instead of having useless projects in shop class the students will complete real projects for their community which have to be designed, planned, and made. The students who end up going to university will learn some generally useful skills through this and the ones who don t go to university will probably have an immediate opportunity to apply the skills that they learn.Elizabeth Pisani gave an interesting TED talk about rational behavior and HIV [5]. She explains why certain behavior that carries great risk can be rational for the people who do it. One of her interesting points is the fact that the poor health system in the US makes it economically viable for the government to refrain from providing free needles to junkies as they don t pay to treat them when they get sick. She makes a good case for political change.Tony Porter gave an inspiring TED talk titled A Call to Men about the way the macho culture socialises men towards violence against women [6].Fraud Magazine has an interesting Interview with Dr. Robert D. Hare and Dr. Paul Babiak about psychopathic fraudsters [7]. Most of the interview is generally about psychopaths and the cases of psychopaths in business, there isn t much that is specific to financial fraud. It has a good summary of what some psychopaths are like.Barbara Fister wrote a Liberation Bibliography a call for knowledge to be free [8]. Currently universities pay researchers who then pay for corporations to accept copyright for their work and then resell it back to the universities. This is sucking money from the university system and denying access to knowledge to the people who can t pay.Charles Limb gave an interesting TED talk about using an MRI to study the brains of Jazz musicians doing Improv and Rappers doing Freestyle [9].Deborah Rhodes gave an interesting TED talk about her new method of screening for breast cancer for women with dense breast tissue [10].Cory Doctorow wrote an interesting article about Net Neutrality and it s importance in helping the less famous writers (the majority of writers) earn money [11]. He also makes some interesting points about Google and about the corporate welfare given to telephone companies.Beyond Zero Emissions has an interesting summary of the current situation regarding Carbon Capture and Sequestration (CCS), among other things the only system that s currently operational is leaking CO2 and killing plants and animals [12].The Brisbane council knew that the flood risks were greater than suggested by published information, they kept this news to themselves to assist property developers in selling low-lying land, and this led to increased problems now that the predicted floods have occurred [13]. Sounds like corruption, we could probably do with a Royal Commission into this.Psychology Today has an interesting article titled Wise Words From Fellow Introverts which is a collection of blog comments from introverts [14].

15 January 2011

Jordi Mallach: De mar a mar, hiking across the Pyrenees

Two weeks ago, Maria and I completed one of our dreams when we arrived in Cap de Creus, where the Pyrenean range sinks in the Mediterranean. To get there, we walked hundreds of kilometres during a month, crossed dozens of steep valleys and enjoyed one of the richest experiences of our lives.

We managed to complete this challenge without facing major problems or pains, and after the first five our six days, our legs seemed to have gotten used to the daily effort and it started to be easier and easier. Our morale kept growing as days passed and we advanced east. When I started walking on a cold and rainy morning in Hondarribia, after barely no rest in the night bus to Ir n, I thought for myself that it was improbable that we'd manage to get anywhere near Catalunya, that one of us would get injuried way before, or we'd just give up and go for the easy beach vacation in the Basque Country.


Biad s refuge, under Posets But we didn't, and after a somewhat painful start, with our boots soaking wet during the stages that crossed the enchanting Selva de Irati which ended up with me getting the biggest blister I've ever seen, we started to walk farther every day, extending the stages when we felt strong after reaching their official end. When we crossed from Nafarroa to Aragonese territory, the mental wall that I had built over the toughest stages in our quest started to fall apart. Days later, we found ourselves climbing down to Pineta, leaving Ordesa behind and enjoying a feeling that our adventure could not go better. The weather had been perfect for over a week and our legs and back were strong to go all over the way to the sea.


Ca n de A isclo, on our way to Pineta When we finally saw that huge blue stain on the horizon, the day before getting to the last mark, we got really excited. We had made it, but as we walked towards the cape, happiness slowly got mixed with melancholy. An unforgettable adventure was about to end, and we didn't want to face our return to the city and our routines. It had been many days surrounded only by awesome landscapes, and living without watches, with only sunlight and weariness marking the time to get in our sleeping bags.


At the very last red and white mark of our journey We've had plenty of time to meet great people. Starting with the Navarrian brothers we met in the early stage, which provided us with a good pace to follow while our leg muscles were still building up; or Kike and Ana, who drove from Pamplona to visit us during the first Aragonese stages (thanks for the supplies!). Tom s and Roger, young hikers from Matar , surprised us with their maturity and experience as mountaineers; we had the pleasure to join forces during four days, walking a really cool variant through the Infiernos and Collado del Letrero which avoided going through the ghost city of Panticosa. Roger and Tom s, we really hope to meet you soon! Andreu, Manel, Ghandi and Ga la visited us at the Vall de N ria and were unlucky to suffer a frightening hail and thunder storm when they left our shelter on their way back to their car. Thanks! All of you have been a very special part of our experience!

This adventure through GR 11 has been incredibly positive for us for several reasons. We've learned a lot about ourselves and strengthened our relation, and now I know how powerful drive can be; to get somewhere, no matter how far, it's really easy if you just believe you can do it and desire to get there. Maria and I are proud of what we've done, and no wonder we're looking for new challenges. An obvious one would be repeating this experience, going through the French side of the Pyrenees, but for now, the Corsican GR 20 seems the most appealing. We'll see, next summer!

Next.

Previous.