The general trend seems to be that cheap hard drives are increasing in capacity faster than much of the data that is commonly stored. Back in 1998 I had a 3G disk in my laptop and about 800M was used for my home directory. Now I have 6.2G used for my home directory (and another 2G in ~/src) out of the 100G capacity in my laptop. So my space usage for my home directory has increased by a factor of about 8 while my space available has increased by a factor of about 30. When I had 800M for my home directory I saved space by cropping pictures for my web site and deleting the originals (thus losing some data I would rather have today) but now I just keep everything and it s still doesn t take up much of my hard drive. Similar trends apply to most systems that I use and that I run for my clients. Due to the availability of storage people are gratuitously using a lot of disk space. A relative recently took 10G of pictures on a holiday, her phone has 12G of internal storage so there was nothing stopping her. She might decide that half the pictures aren t that great if she had to save space, but that space is essentially free (she couldn t buy a cheaper phone with less storage) so there s no reason to delete any pictures. When considering backup methods one important factor is the ability to store all of one type of data on one backup device. Having a single backup span multiple disks, tapes, etc has a dramatic impact on the ease of recovery and the potential for data loss. Currently 3TB SATA disks are really cheap and 4TB disks are available but rather expensive. Currently only one of my clients has more than 4TB of data used for one purpose (IE a single filesystem) so apart from that client a single SATA disk can backup anything that I run. Benefits of Hard Drive Backup When using a hard drive there is an option to make it a bootable disk in the same format as the live disk. I haven t done this, but if you want the option of a quick recovery from a hardware failure then having a bootable disk with all the data on it is a good option. For example a server with software RAID-1 could have a backup disk that is configured as a degraded RAID-1 array. The biggest benefit is the ability to read a disk anywhere. I ve read many reports of tape drives being discovered to be defective at the least convenient time. With a SATA disk you can install it in any PC or put it in a USB bay if you have USB 3.0 or the performance penalty of USB 2.0 is bearable a USB 2.0 bay is great if you want to recover a single file, but if you want terabytes in a hurry then it won t do. A backup on a hard drive will typically use a common filesystem. For backing up Linux servers I generally use Ext3, at some future time I will move to BTRFS as having checksums on all data is a good feature for a backup. Using a regular filesystem means that I can access the data anywhere without needing any special software, I can run programs like diff on the backup, and I can export the backup via NFS or Samba if necessary. You never know how you will need to access your backup so it s best to keep your options open. Hard drive backups are the best solution for files that are accidentally deleted. You can have the first line of backups on a local server (or through a filesystem like BTRFS or ZFS that supports snapshots) and files can be recovered quickly. Even a SATA disk in a USB bay is very fast for recovering a single file. LTO tapes have a maximum capacity of 1.5TB at the moment and tape size has been increasing more slowly than disk size. Also LTO tapes have an expected lifetime of only 200 reads/writes of the entire tape. It seems to me that tapes don t provide a great benefit unless you are backing up enough data to need a tape robot. Problems with a Hard Drive Backup Hard drives tend not to survive being dropped so posting a hard drive for remote storage probably isn t a good option. This can be solved by transferring data over the Internet if the data isn t particularly big or doesn t change too much (I have a 400G data set backed up via rsync to another country because most of the data doesn t change over the course of a year). Also if the data is particularly small then solid state storage (which costs about $1 per GB) is a viable option, I run more than a few servers which could be entirely backed up to a 200G SSD. $200 for a single backup of 200G of data is a bit expensive, but the potential for saving time and money on the restore means that it can be financially viable. Some people claim that tape storage will better survive a Carrington Event than hard drives. I m fairly dubious about the benefits of this, if a hard drive in a Faraday Cage (such as a regular safe that is earthed) is going to be destroyed then you will probably worry about security of the food supply instead of your data. Maybe I should just add a disclaimer this backup system won t survive a zombie apocalypse . ;) It s widely regarded that tape storage lasts longer than hard drives. I doubt that this provides a real benefit as some of my personal servers are running on 20G hard drives from back when 20G was big. The fact that drives tend to last for more than 10 years combined with the fact that newer bigger drives are always being released means that important backups can be moved to bigger drives. As a general rule you should assume that anything which isn t regularly tested doesn t work. So whatever your backup method you should test it regularly and have multiple copies of the data to deal with the case when one copy becomes corrupt. The process of testing a backup can involve moving it to newer media. I ve seen it claimed that a benefit of tape storage is that part of the data can be recovered from a damaged tape. One problem with this is that part of a database often isn t particularly useful. Another issue is that in my experience hard drives usually don t fail entirely unless you drop them, drives usually fail a few sectors at a time. How to Implement Hard Drive Backup The most common need for backups is when someone deletes the wrong file. It s usually a small restore and you want it to be an easy process. The best solution to this is to have a filesystem with snapshots such as BTRFS or ZFS. In theory it shouldn t be too difficult to have a cron job manage snapshots, but as I ve only just started putting BTRFS and ZFS on servers I haven t got around to changing my backups. Snapshots won t cover more serious problems such as hardware, software, or user errors that wipe all the disks in a server. For example the only time I lost a significant amount of data from a hosted server was when the data center staff wiped it, so obviously good off-site backups are needed. The easiest way to deal with problems that wipe a server is to have data copied to another system. For remote backups you can rsync to a local system and then use cp -rl or your favorite snapshot system to make a hard linked copy of the tree. A really neat feature is the ZFS ability to send a filesystem snapshot (or the diff between two snapshots) to a remote system [1]. Once you have regular backups on local storage you can then copy them to removable disks as often as you wish, I think I ll have to install ZFS on some of my servers for the sole purpose of getting the send feature! There are NAS devices that provide similar functionality to the ZFS send/receive (maybe implemented with ZFS), but I m not a fan of cheap NAS devices [2]. It seems that the best way to address the first two needs of backup (fast local restore and resilience in the face of site failure) is to use ZFS snapshots on the server and ZFS send/receive to copy the data to another site. The next issue is that the backup server probably won t be big enough for all the archives and you want to be able to recover from a failure on the backup server. This requires some removable storage. The simplest removable backup is to use a SATA drive bay with eSATA and USB connectors. You use a regular filesystem like Ext3 and just copy the files on. It s easy, cheap, and requires no special skill or software. Requiring no special skill is important, you never know who will be called on to recover from backups. When a server is backing up another server by rsync (whether it s in the same rack or another country) you want the backup server to be reliable. However there is no requirement for a single reliable server and sometimes having multiple backup servers will be cheaper. At current purchase prices you can buy two cheap tower systems with 4*3TB disks for less money than a single server that has redundant PSUs and other high end server features. Having two cheap servers die at once seems quite unlikely so getting two backup servers would be the better choice. For filesystems that are bigger than 4TB a disk based backup would require backup software that handles multi part archives. One would hope that any software that is designed for tape backup would work well for this (consider a hard drive as a tape with a very fast seek), but often things don t work as desired. If anyone knows of a good Linux backup program that supports multiple 4TB SATA disks in eSATA or USB bays then please let me know. Conclusion BTRFS or ZFS snapshots are the best way of recovering from simple mistakes. ZFS send/receive seems to be the best way of synchronising updates to filesystems to other systems or sites. ZFS should be used for all servers. Even if you don t currently need send/receive you never know what the future requirements may be. Apart from needing huge amounts of RAM (one of my servers had OOM failures when it had a mere 4G of RAM) there doesn t seem to be any down-side to ZFS. I m unsure of whether to use BTRFS for removable backup disks. The immediate up-sides are checksums on all data and meta-data and the possibility of using built-in RAID-1 so that a random bad sector is unlikely to lose data. There is also the possibility of using snapshots on a removable backup disk (if the disk contains separate files instead of an archive). The down-sides are lack of support on older systems and the fact that BTRFS is fairly new. Have I missed anything?

• [1] http://cuddletech.com/blog/pivot/entry.php?id=984
• [1] http://etbe.coker.com.au/2012/03/23/cheap-nas-devices-suck/

Related posts:

1. New Storage Developments Eweek has an article on a new 1TB Seagate drive....
2. IDE hard drives I just lent two 80G IDE drives to a friend,...
3. Hot-swap Storage I recently had to decommission an old Linux server and...

TLDR: multiarch, multiarch, multiarch, cross buildability, staged build dependencies, wiki page, corrections/hints/requests to debian-bootstrap at lists.mister-muffin.de This summer (and this year's GSoC) is nearing its end and to make it easier for people to make use of the information my tools produced so far, I created a page in the Debian wiki. It lists not only the open issues I see but also statistics that I gathered using the output of my GSoC project. I want to use this blog post to make people aware of that page as well as to get some feedback on it and anything related to it. The biggest blocker my tools face, is that many packages are still missing multiarch information. As long as at least the basic packages do not have their cross build dependencies satisfied via multiarch for an existing foreign architecture, automated tools can not properly analyze the dependency situation in the bootstrapping case, when many packages of the new foreign architecture do not even exist yet. If Debian is supposed to be bootstrappable, then the first stage is to make a set of basic packages cross compile for an existing foreign architecture. Once this is possible, a tool of mine can analyze the cyclic build dependency situation that might occur when cross compiling for an architecture that does not exist yet. Then, staged cross builds can be used to cross compile a minimal foreign system. Due to missing multiarch classification, it is not known yet how big the cyclic build dependency situation is for the base packages. It is not only the conversion of packages to multiarch that is needed but also the adding of the :any (and rare cases :native) qualifier to build dependencies on M-A: allowed packages. Prominent build dependencies that should (but are not yet) be M-A: allowed are python and gettext. Both are needed as a build dependency by many packages of the base system. Unfortunately wanna-build does not understand qualifiers like :any and :native yet. Until it does, no package can be marked :any or :native and cross compilation of many base packages can not succeed. Once the point is reached, where a base system can be cross compiled from nothing, native compilation can start. Since native compilation doesnt depend on multiarch, the dependency situation when trying to natively compiling all of Debian from nothing is understood much better. Unfortunately, the cyclic build dependency situation is also much worse in the native case and there exists a big 1000 node strongly connected component of binary and source packages that all interdepend on each other. This dependency mess can be solved using three approaches:

• using Build-Depends-Indep
• stage building
• cross compilation

The wiki page gives many hints on how to find packages that each method can be applied to. Stage building is a tool that might be useful for cross building (we dont know for sure yet) but is definitely needed for native compilation. It is needed for native compilation because after all possible dependencies are moved to Build-Depends-Indep, the only other alternative to stage building for breaking dependency cycles is to cross build source packages. Since building a package without one of its build dependencies "staged" is often much easier than making the package in question cross compile, it is a preferred alternative. Once more packages have been made multiarch, it might be possible to prove that there is no alternative to introducing a notion of staged builds. Some people (wookey, Patrick McDermott, Guillem Jover, myself) decided that the following format to mark staged build dependencies would be preferred over others:

Build-Depends: huge (>= 1.0) [i386 arm] <!embedded !bootstrap>, tiny

The <> format was proposed by Guillem Jover in bug#661538. Patches for dpkg and dose3 are done. More people need to discuss about this format for a final decision on how to indicate staged build dependencies. For more information on the topic, have a look at the corresponding wiki page. Feel free to direct any comments/critique/hints to debian-bootstrap at lists.mister-muffin.de or directly to me.

While enabling logging for release of rewritten version of SILPA described in my previous post I faced a weird problem. I enabled Logging as described in the Flask documentation but SILPA won't log anything to the log even though log file is created in whatever location I gave while configuring the logging. Flask documentation suggests following way to use logging After banging my head for some time I again wanted to look at Python's logging tutorials see if I'm doing something wrong and I found this example So there it is I need to set the logging level even on logger to same level as what I set on my handler. Now I didn't take time to read the documentation because I wanted to see if my findings really does the trick so I modified function configuring logging to look something like below (I'm taking the example given in Flask documentation itself real code can be seen in SILPA code on Github) This actually did the trick and now Flask is happily logging whatever I give to app.logger.levelname So I don't know this was my stupidity or fault in Flask's documentation by forgetting to mention this crucial point. But in the end all is well :-) Do comment your thoughts.

The recent update by S ren H jsgaard's to his gRbase package for graphical models made it the 75th package to depend on our Rcpp package for R and C++ integration. So in a lighthearted weekend moment, I tweeted about gRbase being number 75 for Rcpp to which Hadley replied, asking if Rcpp was in fact number one among R package Depends. Far from it, and I immediately replied listing lattice and Matrix as packages with way more other packages depending upon them. But as the question seemed deserving of a bit more analysis, I spent a few minutes on this and prepared three charts listing package in order of reverse Depends, reverse Imports and reverse LinkingTo. First off, the reverse Depends:. This is the standard means of declaring a dependence of one package upon another. Unsurprisingly, the MASS package from the classic Venables and Ripley book comes first, with Deepayan Sarkar's powerful lattice package (also covered in a book) coming second. These are both recommended packages which are commonly distributed with R itself. Next are mvtnorm and survival. Our Rcpp is up there in the top-ten, but not a frontrunner. With the advent of namespaces a few R releases ago, it became possible to import functions from other packages. So the Imports: statement now provides an alternative to the (older) Depends:. The next chart displays the same relationship for Imports:: Now lattice still leads, but Hadleys's plyr package grabbed the second spot just before MASS and Matrix. It is interesting to see that the sheer number of Imports: is still not where the Depends: are. On the other hand, we see a number of more recent packages popping up in the second chart. This may reflect more recent coding practices. It will be interesting to see how this stacks up over time when we revisit this chart. Lastly, we can also look at LinkingTo:, a declaration used to provide a C/C++-level dependency at the source code level. We use this in the Rcpp family to provide automatic resolution of the header files needed to compile against our packages. And unsurprisingly, because packages using Rcpp actually use its API (rather than R functions), the package is a little ahead of others. In the package we find three more packages of the Rcpp family, but only a limited number of other packages as C/C++-level dependencies are still somewhat rare in the R universe. There are also fewer packages overall making use of this mechanism. One could of course take this one level further and sum up dependencies in a recursive manner, or visualize the relationship differently. But these dotchart graphs provide a first visual description of the magnitude of Depends, Imports and LinkingTo among CRAN packages for R.

Monthly DPL bits, fresh from the oven.

---

Tip to feel good about the release #476: before reading this, grab and fix one of the RC bugs affecting Wheezy. Done? Now you're ready for a slightly less exciting report of DPL activities. Highlights

• I've wrapped up a couple of years of discussions with FSF representatives, proposing a joint working group on FSF' assessment of Debian Free-ness. Some news coverage ensued. The discussion list that has been set up welcomes participants and, in particular, could use some more DDs, fond of Debian principles but also interested in external review of our free/non-free separation.
• DebConf12 has happened!, and many of us attended it. Due to family reasons I've been able to attend only DebCamp and DebConf day-0, just in time to deliver the traditional "bits from the DPL" speech. Among other topics, in the speech I've discussed governance scaling issues in our governance, and in particular about the role of the DPL --- issue which I'm partly trying to address with the "DPL helpers" initiatives. Slides and videos of the talk are available. As many others DebConf people, I took part in Nicaraguan TV shows about DebConf, showing a good deal of interest by the local media in our presence. I don't have links to the recordings yet, but I suspect DebConf orga people will provide them as soon as they're available.
• Many others DebConf12 events are relevant for future Project directions. I went through many of the events I've missed thanks to the work of the video team and of the gobby note takers. I recommend doing the same. Some events are particularly relevant to recent project-wide discussions, such as EFI, hijacking, and the release process and I'm sure you'll find more If you're a DebConf12 speaker and produced slides, please upload them to penta, so that they could be used as future reference.

Assets

• On the hardware front, we bought another server according to DSA's "5-year plan". It will be used as a ganeti node at the man-da hosting location. We bought it thanks to donated money, for a total of ~6'600 EUR.
• (After ~8 months of pestering,) we got from SPI a data dump of Debian financial transactions there. Thanks for this result goes to the SPI Board and in particular to Robert Brockway and Joerg Jaspert. This was the last blocker to produce detailed financial reports for the past couple of years of Debian activities. Unfortunately, it still is a blocker: Debian auditors found out that the data dump is not complete. SPI will send us remaining data soon , but I don't have an ETA, unfortunately.
• An ftpmasters sprint is being organized and I've been happy to approve its budget. Help is still needed on the -sprints list to keep up with the organization part of sprints, including activities like documenting sprints to show donors what we do with their help.

Logo & trademark

• to stop the absurdity of not using the official logo ("with Debian") as part of our official theme, I advanced a bit on the topic of relicensing the logo under a DFSG-free license. I sought a second legal advice to SFLC (as requested by the SPI board). They confirm we can safely do the relicensing (without undermining our rights on contained marks) under a license like dual LGPLv3+ / CC-BY-SA, provided that a suitable trademark policy is in place
• which is why I've posted to -project a trademark policy draft, based on legal advice from SFLC. I've gather useful feedback already, but more is welcome. Note that, as Russ very eloquently put it, we do not particularly enjoy thinking about which restrictions we put on the usage of our marks. But given we do have (and use) those marks, we should better empower the legal owners (SPI in this case) to properly defend them. Our specification for the policy to SFLC has been "as free as possible", and that is what we're trying to achieve.

Note that the actual logo relicensing should be done by SPI, via their board, upon request of mine (as Debian Project liaison). We won't make it for the next SPI board meeting, as it is on 4 days away. But we can aim for the subsequent meeting, on September 13th. If all goes well (big "if"), we will enjoy a DFSG-free logo after that date. Internal organization Recent flurry of re-organization in the tech-ctte --- which I somewhat triggered pushing for periodic meetings --- seems to be proceeding well. I'm very happy about it, as we all need to trust that tech-ctte decisions will be not only sound, but also prompt. If you're interested into this topic, some recent evidence of the ongoing reorganization and its result can be found in their DebConf12 BoF, minutes of the last meeting, a set of forthcoming GRs, and the recent great decision of posting decisions results to d-d-a (as it happened for node/nodejs). Kudos to tech-ctte members for the recent activism! I haven't worked on it myself directly, but I highlight Enrico's work on server side archival of NM conversations. It has the potential of enabling automatic detection of stuck NM processes. I'm a bit rusty as AM now, but I think missing that ability is one of the main remaining causes of frustration when joining Debian. So, if you are an AM, please opt-in and use this feature with your appicants. If you are not an AM why not? Miscellanea Some misc legal stuff:

• I got the legal advice we asked about the possibility of turning mentors.debian.net into an official service. The worry there was about the potential copyright infringements by mentees. In the end, we don't need to worry about that, provided we've clear contact points (and "reasonable" timing ) for removing infringing material once it's been reported, and that we declare our public policy about that. I've a draft policy document to start with. I'll soon contact mentors.d.n to adapt it where needed and have it vetted by SPI before going live.
• I've worked with GNOME maintainers (thanks in particular to Jordi Mallach) to rework the mixed Debian/GNOME logo on their alioth project page, to stay on the safe side and avoid any ambiguity wrt GNOME trademark policy.

and some misc "political" stuff:

• we've been happy to accept an invitation to attend the GNOME Advisory Board meeting at GUADEC: Jord (thanks, again!) kindly accepted to represent Debian there. An early report has been posted on his blog, a more detailed one is forthcoming to -project.
• OSI has discussed with their affiliates the possibility of signing the Declaration of Internet Freedom. On behalf of Debian, I've vouched for it. By the way, I'm serving pro tempore as Debian liaison at OSI, but if there are people interested in doing that, I'll be happy to consider handing over the responsibility to other (it's a quite lightweight one, at present).

Happy RC bug squashing,
Cheers.

---

PS the boring day-to-day activity log for July 2012 is available at master:/srv/leader/news/bits-from-the-DPL.txt.201207

As a introduction on what is SILPA you can read my previous post This application was written by Santhosh about 6-7 years ago in May 2010 2009 ( Correct me if I'm wrong Santhosh ;-) Thanks to Santhosh for correcting me ) in Python without using any web framework. He crafted whole system single handed and with very nice ideas. He wrote all the modules available in it for all possible Indian languages even though he is native Malayalam speaker. I joined him 3 years back and I was providing some or other improvements and he ultimately made me as the Co-Developer of SILPA. Recently I came across the Flask microframework and thought of rewriting SILPA to use Flask. You might ask me why Flask? So here are my reasonings

 1. Why reinvent Python webframework when we can use existing? Besides maintaining a not so well known framework code which is not written by you will be difficult    and time consuming.
 2. All Python modules are useful even outside SILPA project why make them dependent on SILPA code?
 3. Templating system used is very old and can't be ported to new Python version lets use something which is new and portable
 4. If new contributor wanted to add a module he need to understand SILPA framework to write and integrate it. This might be discouraging new
    contributions so lets use pure python module and make integration with SILPA straight forward.
 5. I was overwhelmed by Flask :P

I had previously tried bottle microframework but for some reason I couldn't crack it so choosing of Flask is not because it was better among other available microframework or something like that. I choose it because I could easily understand it and it suited my needs better. So I proposed about this to Santhosh and he asked me for a POC code. I started working on April 8 2012 and today by August 5 I can say I've first stable version of SILPA rewritten using Flask deployable over WSGI on Apache2. It can also be deployed on other servers supporting WSGI interface. It took me quite a bit of time but to be frank in the middle I completely had stopped coding because of some other works. Here is the commit graph and code frequency graphs from Github.

Changes in New Version Here is some major changes in current version.

1. Newly developed framework provides plug and play features for new modules.
2. You need to install the modules which you want to run on it separately using PIP and only minimal code change is required. Only configuration and webbridge.py 
   will be changed. Ofcourse you need to add html file to *templates* folder.
3. This version uses decent routing logic from Flask for page serving.
4. It also provides JSONRPC interface which is compatible with previous version but only with method name changed.
5. This version separates out run time module loading and routing the request to modules.
6. Dynamic module loading now happens only at the beginning of application, so if I want to implement caching for particular services it will be straight forward
   now.
7. Routing requests to proper module is adapted from old version with few modifications
8. All previous pages are now converted to use *Jinja2* template system.

Things Remaining TODO Below is some stuffs still pending before this version can completely replace http://silpa.org.in

1. Separate all modules from original SILPA and get it uploaded to pypi
2. Provide RESTful interface
3. Provide proper LICENSE, AUTHORS file for all modules 
4. Prepare a Nice UI
5. Find a better name than *SILPA*
...

Update

Forgot to provide the github link and demo link for the new version of SILPA. Thanks to Aravinda for pointing
it out. Here it goes
Github: http://github.com/copyninja/Silpa-Flask
Demo: http://flasksilpa-indic.rhcloud.com

First things first: The issue with the feedback form is fixed. Sorry for the inconvenience. I'm not sure if you followed the schedule of this year's debconf closely, but there was an International Poetry Night happening. Given that Mermaids still is quite important for me, but also, that I like to write poetry every now and then, I had the idea spinning around in my head to present Mermaids there. The simple thought of that made me extremely nervous. Things happened as they liked to happen: the video streaming setup for debconf still was giving me a bit of headaches, and suddenly it was too late to print out the poem because I didn't even had CUPS installed on my new laptop. Also, having reread it recently I noticed that it wasn't flowing too well, so I was uncertain whether it would be possible to properly perform it. I attended nevertheless. It was rather cosy in El Panal, but still a fair amount of people there. And to my surprise, quite a lot of Debian people were presenting poetry. Some in Spanish, some in English, but also one in Japanese, one in German and even one Esperanto. I had the feeling that I really should present something too. I still was extremely nervous, but during the day I had opened some of my poems in the browser, so I started to write down The Girl and the Boy, and when Fito called me to the microphone, I think most people were able to tell how nervous I was about it. But it went well. It even went so well that I felt the need to perform another poem. I chose that's what friends are for, quickly scribbling it down in bad light. The choice was easy, given that it is one that means a lot to me and that an haiku already got presented. It was really a nice experience to not only write these poems and publishing them on deviantART and in my blog, but also to present them to a live audience. Then there was a break. Actually I thought it was over already, but they started a second round. And I somehow liked the way it worked out, so I started to dig for something else to present. I settled for Strange and Jonathan insisted on translating it into Spanish and present that version, too. His version wasn't in Haiku style, but I think it still was a very nice idea. It seem to have been received well, but given that I don't speak Spanish I can only hope he was able to catch the feelings that I did put into this short piece. I am confident he did well. :) As final piece I settled for another short piece, this time it was in German, Wahre Liebe. I think I was able to transport the feelings of it with adding pauses in certain spots, although I fear most people didn't get it because it was ... well, in German. Again, a fair amount of the people who presented poetry were from Debian, actually more than half of it I think. And I am wondering: When will the CfP for next year's debconf open so we can try to establish this event as regular debconf event? And who knows, maybe I'll find the courage then to perform Mermaids.

/personal permanent link Comments: 0 Flattr this

After seeing the Retina display I ve been thinking about the computer products that I ve immediately desired. Here is the list of the ones I can still remember:

1. My first computer which was the TEC-1 [1], in 1982 or 1983.
2. A computer with a full keyboard and a monitor (Microbee), in about 1984. A hex-only keypad is very limiting.
3. Unix, initially SunOS 4.0 in 1991. Primarily the benefits of this were TCP/IP networking, fast email (no multi-day delay for Fidonet mail), IRC, and file transfer from anywhere in the world. Not inherent benefits to Unix, but at the time only Unix systems did TCP/IP at all well.
4. OS/2 2.0 in 1992. At the time OS/2 had the best GUI of any system available (IMHO) and clearly the best multitasking of DOS and Windows programs.
5. Linux in 1992. I started with the TAMU and MCC Interim distributions and then moved to SLS when it was released. The first kernel I compiled was about 0.52. At the time the main use of Linux for almost everyone was to learn about Unix and compile kernels. In 1993 I started running a public access Linux server.
6. Trinitron monitors in 1996. I first saw an IBM Trinitron monitor when working on an IBM project and had to buy one for home use, at the time a 17 Trinitron monitor beat the hell out of any other display device that one could reasonably afford. A bigger screen allowed me to display more code at once which allowed easier debugging.
7. Thinkpad laptops from 1998 until now. They just keep working well and seem to be better than other products every time I compare them. I also like the TrackPoint. 1998 was when a Thinkpad dropped to a mere $3,800 for a system that could run with 96M of RAM, enough compute power for the biggest compiles and it cost less than most cars!
8. The KDE desktop environment in 1998. In 1998 I switched my primary workstation from a PC running OS/2 to a Thinkpad running Linux because of KDE. Prior to KDE nothing on Linux was user-friendly enough.
9. The iPaQ hand-held PC. I got one in 2002 and ran the Familiar distribution of Linux on it. I had it running SE Linux and used it for writing an article for Linux Journal. Being able to get a computer out on public transport to do some work really saved some time. In some ways the iPaQ hardware and the Familiar OS beat modern Android systems.
10. The EeePC 701 which I bought in 2008 [2]. In the last 4 years someone has probably released a system that s no larger or heavier and has the same amount of compute power (enough for web browsing, email, and ssh). But most Netbooks that I ve seen don t compete. The EeePC allowed me to take laptops to places where it previously wasn t convenient.
11. Android, before using Android I never had a smart phone that I used for anything other than taking photos. The other smart phone OSs are either locked down or don t have the app support that Android has. I listed lots of problems with my first phone the Sony Ericsson Xperia X10, but I still really enjoyed using it a lot [3]. Since getting an Android phone I ve read a lot of email while on the go, this means I can respond faster when necessary and use time that might otherwise be wasted. The ssh client means that I don t need to carry a laptop with me when there s a risk that emergency sysadmin work may be required.
12. Cheap rented servers, Amazon defined cloud computing with EC2, Linode offers great deals for small virtual servers, and Hetzner offers amazing deals on renting entire servers. Getting your own Internet connection or running your own physical server in someone s data-center is a lot of effort and expense. Being able to just rent servers is so much easier and allows so many new projects. I can t remember when I first started using such services, maybe 5 years ago.
13. The Apple Retina Display [4] a few days ago.

For the period between 1998 and 2008 I can t think of anything that really excited me apart from the iPaQ. Computers became a lot smaller, faster, cheaper, etc. But it was never a big exciting change. The AMD64 architecture wasn t particularly exciting as most systems didn t need more than 4G of RAM and the ones that did could use PAE. What are the most exciting computer products you have seen?

• [1] http://etbe.coker.com.au/2007/08/23/the-start-of-my-computer-career/
• [2] http://etbe.coker.com.au/2008/07/21/review-of-the-eeepc-701/
• [3] http://etbe.coker.com.au/2011/02/18/xperia-x10i-first-experience/
• [4] http://etbe.coker.com.au/2012/07/31/retina-display/

Related posts:

1. Old PDA vs New Mobile Phone for PDA use Since about 2002 I have been using a iPaQ [1]...
2. CyanogenMod and the Galaxy S Thanks to some advice from Philipp Kern I have now...
3. Moving from a Laptop to a Cloud Lifestyle My Laptop History In 1998 I bought my first laptop,...

Why makes it git so hard to use a self signed SSL certificate in conjunction with the https protocol? At work we have a server for shared git repositories. For some reasons we can t use the ssh protocoll to acces the repositories so we looked into the git-http-backend. So far so good but we want it encrypted, of course. So we used SSL with our self signed certificate:
git clone https://git.example.com/public
Cloning into 'public'...
error: server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none while accessing https://git.example.com/public/info/refs
fatal: HTTP request failed
Huh? Looking deeper into the problem it turns out that git uses curl for the http(s) transport and curl refuses to work with SSL certificates it cannot verify. Ok, that s not a bad thing. To circumvent that you can either set an environment variable (GIT_SSL_NO_VERIFY=1) to make curl ignore the verification or install the certificate on your machine. The first option is not very attractive on the long term as you d have to do it on every operation with the remote server, the second one is not very attractive when dealing with multiple developers working on different operating systems. You ll have to explain to them how to install the certificate on their machine, and that has to be done every time a new developer joins the team, yada, yada. There is also an option in git (http.sslverify) you can set where you can tell git to ignore the verification of the SSL certificate for that repository. The thing is you still have to set the environment variable on the first clone and then you have to tell git to permanently ignore this issue for that repository with the configuration option a lot of stuff to remember. Heck, looking on the interwebs I see may of the people with that problem suggesting to shut of SSL cert verification permanently by setting it globally. I really wonder why git cannot simply tell the user that the SSL certificate cannot be verified and if you want to accept it permanently, temporarily or not. Every browser does that. Right now it just quits with an error and leaves the user with a cryptic error. On the other side, when using git with a server providing the repositories via ssh. Git simply asks if you want to accept the key when you access the server for the first time and never bothers you again.

I am pleased to announce that HP accepted my application for participation in this year's Enhanced Early Retirement program. After 26.5 years, my last day at HP will be 31 August 2012. This isn't because I have a problem with HP, quite the contrary! I feel immensely privileged to have worked on so many interesting things with so many smart people for so long. And there are many exciting things happening right now, including high profile projects like Odyssey and Moonshot expanding both ends of the server scaling continuum with Linux, and the immense commitment to and investment in Open Source represented by the use of Open Stack and Ubuntu LTS in building HP Cloud. The temptation to stay and continue to help guide HP's Open Source engagement was strong. I'm certainly under no pressure to leave... in fact, I think I managed to shock my management with my decision to retire! But I'm very excited about "taking a break" from full-time employment for a while to focus more time on things that matter a lot to me. My son and I have a number of projects we'd like to work on together as he enters high school. The side business I started two years ago with Keith Packard to design, build, and sell avionics solutions for high power model rockets is growing and could use more attention. I'm in the midst of designing the processor board for another amateur radio satellite. And of course, I'm looking forward to putting more "quality time" towards Free Software projects like FreedomBox and Debian! Since they do not depend on my employment status, I do not expect any immediate changes in the various roles I play outside of HP. I will continue to serve on the board of the Linux Foundation representing individual affiliates, on the board of the FreedomBox Foundation, as Chairman of the Debian Technical Committee, and as President of Software in the Public Interest. And while I hope to travel less than I have in recent years, I look forward to continuing to speak and otherwise participate in key Free Software conferences around the world from time to time. After I retire, the primary point of contact with HP on Open Source matters will be the Director of HP's Open Source Program Office, Phil Robb. But if there are open issues regarding HP and Open Source that I can help with over the next month, feel free to contact me and I'll see what I can do to help.

Last night I played with an Apple Macbook Pro with the new Retina Display (Wikipedia link). Wikipedia cites some controversy about whether the display actually has higher resolution than the human eye can perceive. When wearing glasses my vision is considerably better than average (I have average vision without glasses) and while kneeling in front of the Macbook I couldn t easily distinguish pixels. So Apple s marketing claims seem technically correct to me. When I tested the Macbook Pro I found that the quality of the text display was very high, even now the 1680*1050 display on my Thinkpad T61 looks completely crap when compared to the 2880*1800 display on the Macbook. The Macbook was really great for text and for a JPEG that was installed on the system for testing. But unfortunately pictures on web sites didn t look particularly good. Pictures on my blog looked quite poor and pictures returned by a Google search for art didn t look that great either. I wonder if Safari (the Apple web browser) isn t properly optimised for the display or if there is something that we should do when preparing pictures for web sites to make them look better on Safari. The retina display has a 71% greater DPI which means 2.93* the total number of pixels of my Thinkpad. The overall quality of the experience for me (apart from web pictures) seems more like a factor of 2.93 when compared with my Thinkpad than a factor of 1.71. This has to be one of the most desirable products I ve seen from a company that s opposed to freedom for it s users. I m not about to buy one though, $2,300 is a lot of money for a system that can t be upgraded, repaired, or recycled, and doesn t even have an Ethernet port. I m sure that if I bought one I would discover that it some of the hardware features don t work properly with Linux. The new Apple design trend of making it impossible to repair anything works reasonably well for phones and tablets which are cheap enough that they are hardly worth repairing when they have been used for a while. Lots of people can afford to spend about $600 on something that may be discarded after a year or two, but very few people can afford to spend more than $2,000 on such a disposable product. Why is Apple the only company producing systems with such displays? If someone produced regular PCs that have the expected features (including an Ethernet port) with such a display at a lower price then I m sure that there would be a great demand. Related posts:

1. Modern Laptops Suck One of the reasons why I m moving from a laptop...
2. A First Digital Camera I ve just been asked for advice on buying a digital...
3. RPC and SE Linux One ongoing problem with TCP networking is the combination of...

A copy of this post is sent to soc-coordination@lists.alioth.debian.org as well as to debian-bootstrap@lists.mister-muffin.de.

Diary

July 2

• playing around with syntactic dependency graphs and how to use them to flatten dependencies

July 4

• make work with dose 3.0.2
• add linux-amd64 to source architectures
• remove printing in build_compile_rounds
• catch Not_found exception and print warning
• use the whole installation set in crosseverything.ml instead of flattened dependencies
• detect infinite loop and quit in crosseverything.ml
• use globbing in _tags file
• use wildcards and patsubst in makefile

July 5

• throw a warning if there exist binary packages without source packages
• add string_of_list and string_of_pkglist and adapt print_pkg_list and print_pkg_list_full to use them
• fix and extend flatten_deps - now also tested with Debian Sid

July 6

• do not exclude the crosscompiled packages from being compiled in crosseverything.ml
• clean up basebuildsystem.ml, remove old code, use BootstrapCommon code
• clean up basenocycles.ml, remove unused code and commented out code
• add option to print statistics about the generated dependency graph
• implement most_needed_fast_wrong as well as most_needed_slow_correct and make both available through the menu

July 7

• allow to investigate all scc, not only the full graph and the scc containing the investigated package
• handle Not_found in src_list_from_bin_list with warning message
• handle the event of the whole archive actually being buildable
• replace raise Failure with failwith
• handle incorrectly typed package names
• add first version of reduced_dist.ml to create a self-contained mini distribution out of a big one

July 8

• add script to quickly check for binary packages without source package
• make Debian Sid default in makefile
• add *.d.byte files to .gitignore
• README is helpful now
• more pattern matching and recursiveness everywhere

July 9

• fix termination condition of reduced_dist.ml
• have precise as default ubuntu distribution
• do not allow to investigate an already installable package

July 10

• milestone: show all cycles in a graph
• add copyright info (LGPL3+)

July 11

• advice to use dose tools in README

July 16

• write apt_pkg based python filter script replacing grep-dctrl

July 17

• use Depsolver.listcheck more often
• add dist_graph.ml
• refactor dependency graph code into its own module

July 18

• improve package selection for reduced_dist.ml
• improve performance of cycle enumeration code

July 20

• implement buildprofile support into dose3

July 22

• let dist_graph.ml use commandline arguments

July 23

• allow dose3 to generate source package lists without Build- Depends Conflicts -Indep

July 29

• implement crosscompile support into dose3

Results

Readme There is not yet a writeup on how everything works and how all the pieces of the code work together but the current README file provides a short introduction on how to use the tools.

• build and runtime dependencies
• compile instructions
• execution examples for each program
• step by step guide how to analyze the dependency situation
• explanation of general commandline options

A detailed writeup about the inner workings of everything will be part of a final documentation stage.

License All my code is now released under the terms of the LGPL either version 3, or (at your option) any later version. A special linking exception is made to the license which can be read at the top of the provided COPYING file. The exception is necessary because Ocaml links statically, which means that without that exception, the conditions of distribution would basically equal GPL3+.

reduced_dist.ml Especially the Debian archive is huge and one might want to work on a reduced selection of packages first. Having a smaller selection of the archive would be significantly faster and would also not add thousands of packages that are not important for an extended base system. I call a reduced distribution a set of source packages A and a set of binary packages B which fulfill the following three properties:

• all source packages A must be buildable with only binary packages B being available
• all binary packages B except for architecture:all packages must be buildable from source packages A

The set of binary packages B and source packages A can be retrieved using the reduced_dist program. It allows to either build the most minimal reduced distribution or one that includes a certain package selection. To filter out the package control stanzas for a reduced distribution from a full distribution, I originally used a call to grep-dctrl but later replaced that by a custom python script called filter-packages.py. This script uses python-apt to filter Packages and Sources files for a certain package selection.

dist_graph.ml It soon became obvious that there were not many independent dependency cycle situation but just one big scc that would contain 96% of the packages that are involved in build dependency cycles. Therefor it made sense to write a program that does not iteratively build the dependency graph starting from a single package, but which builds a dependency graph for a whole archive.

Cycles I can now enumerate all cycles in the dependency graph. I covered the theoretical part in another blog post and wrote an email about the achievement to the list. Both resources contain more links to the respective sourcecode. The dependency graph generated for Debian Sid has 39486 vertices. It has only one central scc with 1027 vertices and only eight other scc with 2 to 7 vertices. All the other source and binary packages in the dependency graph for the archive are degenerate components of length one. Obtaining the attached result took 4 hours on my machine (Core i5 @ 2.53GHz). 1.5 h of that were needed to build the dependency graph, the other 2.5 hours were needed to run johnson's algorithm on the result. Memory consumption of the program was at about 700 MB. It is to my joy that apparently the runtime of the cycle finding algorithm for a whole Debian Sid repository as well as the memory requirements are within orders of magnitude that are justifiable when being run on off-the-shelf hardware. It must also be noted that nothing is optimized for performance yet. A list of all cycles in Debian Sid up to length 4 can be retrieved from this email. This cycle analysis assumes that only essential packages, build-essential and dependencies and debhelper are available. Debhelper is not an essential or build-essential package but 79% of the archive build-depends on it. The most interesting cycles are probably those of length 2 that need packages that they build themselves. Noticeable examples for these situations are vala, python, mlton, fpc, sbcl and ghc. Languages seem love to need themselves to be built.

Buildprofiles There is a long discussion of how to encode staged build dependency information in source packages. While the initial idea was to use Build-Depends-StageN fields, this solution would duplicate large parts of the Build-Depends field, which leads to bitrot as well as it is inflexible to possible other build "profiles". To remedy the situation it was proposed to use field names like Build-Depends[stage1 embedded] but this would also duplicate information and would break with the rfc822 format of package description files. A document maintained by Guillem Jover gives even more ideas and details. Internally, Patrick and me decided for another idea of Guillem Jover to annotate staged build dependencies. The format reads like:

Build-Depends: huge (>= 1.0) [i386 arm] <!embedded !bootstrap>, tiny

So each build profile would follow a dependency in <> "brackets" an have a similar format as architecture options. Patrick has a patch for dpkg that implements this functionality while I patched dose3.

Dropping Build-Depends-Indep and Build-Conflicts-Indep When representing the dependencies of a source package, dose3 concatenates its Build-Depends and Build-Depends-Indep dependency information. So up to now, a source package could only be compiled, if it manages to compile all of its binary packages including architecture:all packages. But when bootstrapping a new architecture, it should be sufficient to only build the architecture dependent packages and therefor to only build the build-arch target in debian/rules and not the build-indep target. Only considering the Build-Depends field and dismissing the Build-Depends-Indep field, reduced the main scc from 1027 vertices to 979 vertices. The amount of cycles up to length four reduced from 276 to 206. Especially the cycles containing gtk-doc-tools, doxygen, debiandoc-sgml and texlive-latex-base got much less. Patrick managed to add a Build-Depends-Indep field to four packages so far which reduced the scc further by 14 vertices down to 965 vertices. So besides staged build dependencies and cross building there is now a third method that can be applied to break dependency cycles: add Build-Depends-Indep information to them or update existing information. I submitted a list of packages that have a binary-indep and/or a build-indep target in their debian/rules to the list. I also submitted a patch for dose3 to be able to specify to ignore Build-Depends-Indep and Build-Conflicts-Indep information.

Dose3 crossbuilding So far I only looked at dependency situations in the native case. While the native case contains a huge scc of about 1000 packages, the dependency situation will be much nicer when cross building. But dose3 was so far not able to simulate cross building of source packages. I wrote a patch that implements this functionality and will allow me to write programs that help analyze the cross-situation as well.

Debconf Presentation Wookey was giving a talk at debconf12 for which I was supplying him with slides. The slides in their final version can be downloaded here

Future Patrick maintains a list of "weak" build dependencies. Those are dependencies that are very likely to be droppable in either a staged build or using Build-Depends-Indep. I must make use of this list to make it easier to find packages that can easily be removed of their dependencies. I will have to implement support for resolving the main scc using staged build dependencies. Since it is unlikely that Patrick will be fast enough in supplying me with modified packages, I will need to create myself a database of dummy packages. Another open task is to allow to analyze the crossbuilding dependency situation. What I'm currently more or less waiting on is the inclusion of my patches into dose3 as well as a decision on the buildprofile format. More people need to discuss about it until it can be included into tools as well as policy. Every maintainer of a package can help making bootstrapping easier by making sure that as many dependencies as possible are part of the Build-Depends-Indep field.

I have happily used FVWM as my window manager for more than 10 years. However, I recently got tired of manually arranging windows and using the mouse so much. A window manager is one of the handful pieces of software getting in your way at every moment which explains why there are so many of them and why we might put so much time in it. I decided to try a tiling window manager. While i3 seemed pretty hot and powerful (watch the screencast!), I really wanted something configurable and extensible with some language. So far, the common choices are:

• awesome, written in C, configurable and extensible in Lua,
• StumpWM, written, configurable and extensible in Common Lisp,
• xmonad, written, configurable and extensible in Haskell.

I chose awesome, despite the fact that StumpWM vote for Lisp seemed a better fit (but it is more minimalist). I hope there is some parallel universe where I enjoy StumpWM. Visually, here is what I got so far:

Awesome configuration Without a configuration file, awesome does nothing. It does not come with any hard-coded behavior: everything needs to be configured through its Lua configuration file. Of course, a default one is provided but you can also start from scratch. If you like to control your window manager, this is somewhat wonderful. awesome is well documented. The wiki provides a FAQ, a good introduction and the API reference is concise enough to be read from the top to the bottom. Knowing Lua is not mandatory since it is quite easy to dive into such a language. I have posted my configuration on GitHub. It should not be used as is but some snippets may be worth to be stolen and adapted into your own configuration. The following sections put light on some notable points.

Keybindings Ten years ago was the epoch of scavanger hunts to recover IBM Model M keyboards from waste containers. They were great to type on and they did not feature the infamous Windows keys. Nowadays, this is harder to get such a keyboard. All my keyboards now have Windows keys. This is a major change with respect to configure a window manager: the left Windows key is mapped to Mod4 and is usually unused by most applications and can therefore be dedicated to the window manager. The main problem with the ability to define many keybindings is to remember the less frequently used one. I have monkey-patched awful.key module to be able to attach a documentation string to a keybinding. I have documented the whole process on the awesome wiki.

Quake console A Quake console is a drop-down terminal which can be toggled with some key. I was heavily relying on it in FVWM. I think this is still a useful addition to any awesome configuration. There are several possible solutions documented in the awesome wiki. I have added my own¹ which works great for me.

XRandR XRandR is an extension which allows to dynamically reconfigure outputs: you plug an external screen to your laptop and you issue some command to enable it:

$ xrandr --output VGA-1 --auto --left-of LVDS-1

awesome detects the change and will restart automatically. Laptops usually come with a special key to enable/disable an external screen. Nowadays, this key does nothing unless configured appropriately. Out of the box, it is mapped to XF86Display symbol. I have associated this key to a function that will cycle through possible configurations depending on the plugged screens. For example, if I plug an external screen to my laptop, I can cycle through the following configurations:

• only the internal screen,
• only the external screen,
• internal screen on the left, external screen on the right,
• external screen on the left, internal screen on the right,
• no change.

The proposed configuration is displayed using naughty, the notification system integrated in awesome.

Widgets I was previously using Conky to display various system-related information, like free space, CPU usage and network usage. awesome comes with widgets that can fit the same use. I am relying on vicious, a contributed widget manager, to manage most of them. It allows one to attach a function whose task is to fetch values to be displayed. This is quite powerful. Here is an example with a volume widget:

local volwidget = widget(  type = "textbox"  )
vicious.register(volwidget, vicious.widgets.volume,
         '<span font="Terminus 8">$2 $1%</span>',
        2, "Master")
volwidget:buttons(awful.util.table.join(
             awful.button(   , 1, volume.mixer),
             awful.button(   , 3, volume.toggle),
             awful.button(   , 4, volume.increase),
             awful.button(   , 5, volume.decrease)))

You can also use a function to format the text as you wish. For example, you can display a value in red if it is too low. Have a look at my battery widget for an example.

Miscellaneous While I was working on my awesome configuration, I also changed some other desktop-related bits.

Keyboard configuration I happen to setup all my keyboards to use the QWERTY layout. I use a compose key to input special characters like . I have also recently use Caps Lock as a Control key. All this is perfectly supported since ages by X11 I am also mapping the Pause key to XF86ScreenSaver key symbol which will in turn be bound to a function that will trigger xautolock to lock the screen. Thanks to a great article about extending the X keyboard map with xkb, I discovered that X was able to switch from one layout to another using groups². I finally opted for this simple configuration:

$ setxkbmap us,fr '' compose:rwin ctrl:nocaps grp:rctrl_rshift_toggle
$ xmodmap -e 'keysym Pause = XF86ScreenSaver'

I switch from us to fr by pressing both left Control and left Shift keys.

Getting rid of most GNOME stuff Less than one year ago, to take a step forward to the future, I started to heavily rely on some GNOME components like GNOME Display Manager, GNOME Power Manager, the screen saver, gnome-session, gnome-settings-daemon and others. I had numerous problems when I tried to setup everything without pulling the whole GNOME stack. At each GNOME update, something was broken: the screensaver didn t start automatically anymore until a full session restart or some keybindings were randomly hijacked by gnome-settings-daemon. Therefore, I have decided to get rid of most of those components. I have replaced GNOME Power Manager with system-level tools like sleepd and the PM utilities. I replaced the GNOME screensaver with i3lock and xautolock. GDM has been replaced by SLiM which now features ConsoleKit support³. I use ~/.gtkrc-2.0 and ~/.config/gtk-3.0/settings.ini to configure GTK+. The future will wait.

Terminal color scheme I am using rxvt-unicode as my terminal with a black background (and some light transparency). The default color scheme is suboptimal on the readability front. Sharing terminal color schemes seems a popular activity. I finally opted for the derp color scheme which brings a major improvement over the default configuration. I have also switched to Xft for font rendering using DejaVu Sans Mono as my default font (instead of fixed) with the following configuration in ~/.Xresources:

Xft.antialias: true
Xft.hinting: true
Xft.hintstyle: hintlight
Xft.rgba: rgb
URxvt.font: xft:DejaVu Sans Mono-8
URxvt.letterSpace: -1

The result is less crisp but seems a bit more readable. I may switch back in the future.

Next steps My reliance to the mouse has been greatly reduced. However, I still need it for casual browsing. I am looking at luakit a WebKit-based browser extensible with Lua for this purpose.

---

1. The console gets its own unique name. This allows awesome to reliably detect when it is spawned, even on restart. It is how the Quake console works in the mod of FVWM I was using.
2. However, the layout is global, not per-window. If you are interested by a per-window layout, take a look at kbdd.
3. Nowadays, you cannot really survive without ConsoleKit. Many PolicyKit policies do not rely on groups any more to grant access to your devices.

As I recently reported, I have been trying to configure my Linksys WRT54GL running the OpenWRT firmware (version 0.9 a.k.a. White Russian) to support SLAAC and DHCPv6 out of the box.Most of the feedback I got involved upgrading the OpenWRT firmware to a more recent release that might not fit the available storage capacity of the device, which felt like too risky of an operation to attempt.The solution I found instead involved adding ipkg sources to OpenWRT's backports repository, which offers a minimalistic DHCPv6 client. Although I haven't been able to find any proper documentation for this client, the content of its configuration file is mostly intelligible. What remains now is to actually configure it to cooperate with the kernel's native SLAAC implementation and, whenever an IPv6 network is found on the ISP side, to propagate the address space, DNS and routing information to my LAN using radvd.Would anyone located within the Helsinki metropolitan area happen to have an IPv6 testing environment that would at least allow me to configure and test my router?

I've been thinking a lot recently about information security and passwords. It's widely agreed that two-factor authentication, which combines something-you-know (a password) with something you have (eg. a smart card or some one-time-password) is superior to using a password by itself. Most solutions I've seen, short of client-side certificates, both require a trusted third party and provide little auditability.

I'm working on a project which attempts to address those problems. Basically, the idea is that you attempt to perform an action on a website, like logging in, moving some money between accounts, or restarting a cluster. Then, your mobile device ((iPhone used in the graphic to the right because a SVG was handy. Image adapted from work by Virgile Pypaert [CC-BY-SA-3.0], via Wikimedia Commons)) (Android, iPhone, or some custom hardware) starts buzzing, displaying the text of the action. When you approve or deny the action, your device uses a locally stored private key to sign the text of the approval and send it back to the server. The server checks that you approved the same transaction number and text, then allows the action. The server can store the approval notice to allow auditors later to determine that somebody in possession of the private key signed the action request.

So far, I've written a small Django server application, and a Python CLI device and client. They're not yet ready for a public release; I need to write up installation instructions and test it a bit more, but I'll publish them soon on Launchpad. On the usability side of things, I plan to conduct some trials in January comparing it to things like Google Authenticator.

This should be less susceptible to MITM attacks than one-time-passwords, since you can authenticate specific transactions, rather than entire sessions.

On the other hand, I'm no security researcher. Dear lazyweb, are there any problems with the above approach? Also, if I'm going to release this (as I plan to once I get something working), I need a name. Ideas?

On a lark, I signed up for the pilot program of the Google Cr-48 about a week ago. To my surprise, I found the distinctive box on my doorstep a week later. After initially mistaking it for an overpacked, late-model OLPC XO-1 I would have to repair, I sort of did a double-take as I realized what I had received.
Using Chrome OS


Wow, they weren't kidding about the boot time. Chrome OS responds (to suspend / resume and boot / power-off) very quickly. Responsiveness elsewhere in the OS of course varies on system load and other factors. On the "terms" page, there is a "system security setting" which tells me I have a Trusted Platform Module, and that my TPM has a randomly generated password. The dialog could use some rewording; I'm pretty familiar with the idea of a TPM, but I'm still not sure what I'm to do with that information.

Cloud print is really a problem for me, right now. I use school printers for the majority of my work, which I was previously able to use by adding them by IP address. Unless there's some magic I'm missing here, I can't do that with Chrome OS, and such a feature will not be supported. (except by "connectors" on Windows and Mac computers)
General thoughts
I can't imagine using a system running Chrome OS as a primary computer. The biggest missing feature is PGP (or any sort of encryption) support for email. This is probably not terribly difficult to implement as an extension, but the idea of my cryptographic software being automatically updated is rather unsettling. I think I'll have to agree with Paul Buchheit that ChromeOS will have to merge with Android; there is a utility for local apps, even if they're becoming less and less critical.

For the last months, I've been working on a metering project for OpenStack, so it's time to talk a bit about it. OpenStack is a growing cloud platform providing IaaS. A problem easily identified by everyone building a public cloud platform is that nothing is provided to retrieve the platform usage data. Some data are available in some places, but not everything is, and you have to do a lot of processing from the various components to get something useful in the end. But in order to bill customers that are using your public cloud platform, you need to dot his. In this regard, a lot of companies running public OpenStack based infrastructure wrote their own solution to cover this functional areas, and to become able to bill theirs customers. To avoid everybody doing and maintaining such a stack in their corners, the Ceilometer has been created. The project aims to cover the metering aspect of the OpenStack components, pulling usage data from every components and storing them into a single place. It then offer a retrieving point for this data via a REST API. The initial specifications have been written in April this year, and actual implementation started in May. The project is currently worked on by me, Dreamhost and Canonical. We already have designed an architecture that we are implementing, and we hope to release a first usable version with Folsom. I did a presentation of this project yesterday at XLCloud, which has been very well received. <iframe allowfullscreen="true" frameborder="0" height="389" mozallowfullscreen="true" src="https://docs.google.com/presentation/embed?id=11ALGC4xuWcRvXKTSnnsteJUkArsTfQW-7IAfWRRI5kQ&amp;start=false&amp;loop=false&amp;delayms=3000" webkitallowfullscreen="true" width="480"></iframe> If you are interested in helping us and contributing, feel free to join us during one of our weekly IRC meeting or fix some bugs. :-)

The brief outage was due to a scheduled move of the servers to a separate rack and subnet dedicated to our work with the Center for Information Assurance & Cybersecurity (ciac) at the University of Washington Bothell (uwb), and a11y.com I am currently exercising the new (to us) equipment and hope to winnow the less than awesome equipment over the next quarter. I spent the last six months finding the best in breed of the surplussed DL385 and DL380 chassis we (work) were going to have recycled. The team and I were able to find enough equipment to bring up one of each with eight and six gigs of memory, respectively. These will make excellent hypervisors for provisioning embedded instances of Slackware, Fedora, RHEL, CentOS, Debian, FreeBSD, OpenSolaris, OpenIndiana, FreeDOS, etc. When I initially configured this xen paravirt environment, I failed to plan for integration with libvirt, so I am now re-jiggering the software bridges so that they are managed using virt-manager. My experience doing this in the Eastsound lab has proved to be quite rewarding. Once network, storage and memory allocation are handled by libvirt, it will be trivial to automate the process of deploying, reclaiming, and managing these Telecommunications Resources efficiently and effectively using Perl. The new equipment we racked over the last weekend all has hardware assisted virtualization and is capable of hosting operating systems which have not been patched to communicate with the host hypervisor. This provides a higher level of assurance that guests will receive the full amount of provisioned resources and reduces the likelihood of resource conflicts, should the cluster operations team mistakenly over-commit. It also makes it possible for these nodes to host Windows-based operating systems. Oh, and they will also host these not insignificant things called BIG-IP Local Traffic Manager Virtual Edition (VE) and BIG-IP GTM Virtual Edition.

The brief outage was due to a scheduled move of the servers to a separate rack and subnet dedicated to our work with the Center for Information Assurance & Cybersecurity (ciac) at the University of Washington Bothell (uwb), and a11y.com I am currently exercising the new (to us) equipment and hope to winnow the less than awesome equipment over the next quarter. I spent the last six months finding the best in breed of the surplussed DL385 and DL380 chassis we (work) were going to have recycled. The team and I were able to find enough equipment to bring up one of each with eight and six gigs of memory, respectively. These will make excellent hypervisors for provisioning embedded instances of Slackware, Fedora, RHEL, CentOS, Debian, FreeBSD, OpenSolaris, OpenIndiana, FreeDOS, etc. When I initially configured this xen paravirt environment, I failed to plan for integration with libvirt, so I am now re-jiggering the software bridges so that they are managed using virt-manager. My experience doing this in the Eastsound lab has proved to be quite rewarding. Once network, storage and memory allocation are handled by libvirt, it will be trivial to automate the process of deploying, reclaiming, and managing these Telecommunications Resources efficiently and effectively using Perl. The new equipment we racked over the last weekend all has hardware assisted virtualization and is capable of hosting operating systems which have not been patched to communicate with the host hypervisor. This provides a higher level of assurance that guests will receive the full amount of provisioned resources and reduces the likelihood of resource conflicts, should the cluster operations team mistakenly over-commit. It also makes it possible for these nodes to host Windows-based operating systems. Oh, and they will also host these not insignificant things called BIG-IP Local Traffic Manager Virtual Edition (VE) and BIG-IP GTM Virtual Edition.

Playing around with LLC controls on Sandybridge today. What I wanted to know was how caching backbuffers affects application performance. The reason for this question is that when the driver uses page flipping to display a new frame, we force the new scanout buffer to be uncached because the display hardware only scans out from main memory. However, when we go back to using this buffer as the next back-buffer, we don t turn the caching bits back on as that requires writing GTT entries to change caching modes. This means that swapping via page-flipping and swapping via copying has a large change in main memory access patterns during rendering page flipping applications use an uncached back buffer while copying applications use a cached back buffer. What effect does the back-buffer caching-mode have on application performance? The changes In the kernel, I have three modes 1) Existing code. Set buffer objects to cache mode none when preparing them as scanout buffers. The first time the application flips to a back buffer, caching is disabled on that buffer forever 2) Flip back and forth. Set buffers to cache mode none when used as scanout. When that stops, flip them back to cache mode llc. This involves a whole lot of clflushing and GTT rewriting, obviously. 3) Flush to memory, but don t disable caching. In this mode, the buffer gets flushed (from both CPU and GPU) before being used as a scanout, but the cache mode isn t changed. Benchmarking with glxgears The first test was pretty simple I hacked up glxgears (yes, glxgears is not a benchmark) to redraw the same frame multiple times (including the clear) per swap to provide a scalable amount of work per swap. No pixel operations, so this is just writing to the back buffer. Thes numbers are the median of 5 runs, except for those obviously limited by the refresh rate.

Reps	Existing	Flip	Flush
10	60	60	60
20	60	60	60
40	42.070	34.113	37.130
80	21.259	17.957	18.658

Note the linear scaling over reps (once you get below refresh). I think this means that the swap overhead is in the noise, including flushing and rewriting GTT entries. Given that my screen is (yuck) 1366x768 pixels, or just over 4MB, and my total LLC is 4MB, it s seems like the performance changes here are likely caused by the rendering ejecting every other object from the cache. What s wrong with benchmarking with glxgears? Of course, glxgears is not a benchmark, and I realized while collecting this data that, in particular, it never reads from the backbuffer, and most pixels are only written once by the clear operation at the start of the rendering code. So, it forces everything else out of the cache at the start of each drawing cycle, but doesn t stick anything useful into the cache. Benchmarking nexuiz, a (more) real application For the next test, I ran nexuiz demo1.dem in all three modes. Numbers out of that weren t nearly as stable as glxgears, and I m afraid I didn t write all of them down, but I got numbers between 45 and 50 frames per second in all three modes. None significantly faster, none significantly slower. So, for these two tests, caching has no positive effect on overall rendering performance. Obviously, I need to collect data from more applications to see if the effect is general. I sure hope so, because the alternative will be to find some heuristic to direct when to enable caching. The patch This is on top of kernel version 3.5.

diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index b0b676a..f69fba8 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1335,6 +1335,10 @@ int __must_check
 i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
                     u32 alignment,
                     struct intel_ring_buffer *pipelined);
+
+void
+i915_gem_object_unpin_from_display_plane(struct drm_i915_gem_object *obj);
+
 int i915_gem_attach_phys_object(struct drm_device *dev,
                struct drm_i915_gem_object *obj,
                int id,
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 288d7b8..b606bd2 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2851,6 +2851,9 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
    return 0;
  
+#define JUST_FLUSH 0
+#define FLIP_CACHING   0
+
 /*
  * Prepare buffer for display plane (scanout, cursors, etc).
  * Can be called from an uninterruptible phase (modesetting) and allows
@@ -2883,7 +2886,11 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
     * of uncaching, which would allow us to flush all the LLC-cached data
     * with that bit in the PTE to main memory with just one PIPE_CONTROL.
     */
+#if JUST_FLUSH
+   ret = i915_gem_object_finish_gpu(obj);
+#else
    ret = i915_gem_object_set_cache_level(obj, I915_CACHE_NONE);
+#endif
    if (ret)
        return ret;
@@ -2913,6 +2920,16 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
    return 0;
  
+void
+i915_gem_object_unpin_from_display_plane(struct drm_i915_gem_object *obj)
+ 
+   i915_gem_object_unpin(obj);
+#if FLIP_CACHING
+   if (HAS_LLC(obj->base.dev))
+       i915_gem_object_set_cache_level(obj, I915_CACHE_LLC);
+#endif
+ 
+
 int
 i915_gem_object_finish_gpu(struct drm_i915_gem_object *obj)
  
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index a8538ac..76a2012 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -1821,7 +1821,7 @@ err_interruptible:
 void intel_unpin_fb_obj(struct drm_i915_gem_object *obj)
  
    i915_gem_object_unpin_fence(obj);
-   i915_gem_object_unpin(obj);
+   i915_gem_object_unpin_from_display_plane(obj);
  
 static int i9xx_update_plane(struct drm_crtc *crtc, struct drm_framebuffer *fb,