The elections for the new OpenStack board are coming closer
and this time the Open Source community has a great
opportunity of representation: Giuseppe Patern is standing as a candidate for the board.
Although Giuseppe is considered by HP and Forrester Research
among the top talented consultants in the world,
Gippa (as he s largely known in the industry) is still one of us ,
a nerd that grew up with a keyboard on his hands.
As he s one of the candidates of the OpenStack board,
Fabio Marzocca wishing to know more has interviewed him.

[FM] The hard question. You re a techie. Why the hell are you running for the board?

[GP] This is indeed a good question It all started as a challenge from some clients and friends that are working in the OpenStack project. The truth is that the board and most of the management of the foundation are from vendors. I m not questioning here if they do a good job or not, it is very likely that they tend to protect their own interests. In my opinion it lacks some community spirit that have fostered Linux development such as Debian and Ubuntu. That s why I m running for it, to bring the community where it should be.

[FM] Back to Debian and Ubuntu, could you tell us your story with Linux?

[GP] I discovered Linux in 1994, but only in 1996 things were serious. By the time I just finished high school and I applied for a job in a local Internet Service Provider. At 15 years I was well known in the local community as I was installing and maintaining several BBSes, so it wasn t hard to get the job. I can say it was love at first sight. I started with Slackware (was the first distro), but I moved into redhat first and then debian. When I was working for the IBM Linux Technology Center, I was in charge of helping porting Linux to PowerPC and backporting LVM to make it similar to AIX. Sun was also a good playground as they acquired Cobalt, a hardware appliance based on debian. Then I shifted more towards Enterprise Linux adoption with 6 years in RedHat and then I went to Canonical. I was happy to go back to Debian and Ubuntu community, because I still believe that Ubuntu Developer Summits (UDS) were the real spirit of a Linux community.

[FM] Another hard question. We know you re somehow involved in the rebellion of Devuan.org. What is your opinion about systemd?

[GP] Let me tell you that it s not totally black/white and let s see the two sides here. Something like systemd was indeed needed. Each distro has its own way of init ing the system and for a package maintainer or commercial software maker, maintaining different init behaviour is insane. And as an init replacement it totally makes sense. However, IMHO systemd went too far away, incorporating into its code something that should not happen. A DHCP client into an init system, seriously? I doubt it was in the spirit of the Unix and Linux system
However, in the real world of pets vs cattle , where application matters more than systems, having a systemd as it is, doesn t change that much.

[FM] OpenStack was incubated in Ubuntu and the roots are quite clear. Is there something else that you would like to see from Debian and Ubuntu in OpenStack?

[GP] Stability, if I can name just one. Currently OpenStack is released every 6 months, which was probably the best choice to speed up the development. However, this is now becoming a weakness, as enterprise customers can t upgrade their critical infrastructures every 6 months. Traditionally Debian is maniacally focused on given a bullet-proof distribution, this is something that in my opinion is missing from OpenStack.

[FM] Gippa, tell us just 2 or 3 topics you will bring to action in case of election

[GP] I d like to introduce an OpenStack LTS process, following the Ubuntu approach: while releasing every 6 months is fair enough for development and testing environments, having a stable release every 2-3 years can give enterprise customers the peace of mind they need while running production environments.
I d also love to see a consolidation of the core (Nova/Neutron/Cinder/Swift): vendors and developers are introducing new features and projects while I d love to see for example- a more stable and scalable Neutron and a more stable connection to Oslo (in particular rabbitmq).
In general, I would encourage more attention to who is actually deploying, integrating and using OpenStack every day. I would also try to foster the ecosystem of ISVs in order to release and certify their software for OpenStack. And last but not least- to see interoperation between regional datacenters: I dream of a world where companies in a given territory can work together , and this is only possible through standards. I hope that OpenStack can represent this standard.

[FM] When are the elections and how can we vote?

[GP] Individual Member Director elections for the 2016 Board will be held online from Monday January 11, 2016 to Friday January 15, 2016. More informations on the website.

Making it easier to use C++ in R has been a really nice and rewarding side effect of the work on Rcpp. One really good idea came from something Jay Emerson, Michael Kane and had I kicked about for way too long: shipping Boost headers for easier use by R and CRAN packages (such as their family of packages around bigmemory). The key idea here is headers: Experienced C++ authors such as library writers can organise C++ code in such a way that one can (almost always) get by without any linking. Which makes deployment so much easier in most use cases, and surely also with R which knows how to set an include path. So after years of "yes we really should", Jay deserves the credit for actually sitting down almost three years ago and creating the first version. So at the end of January 2013, we released BH version 1.51.0-0. By now the package is used fifty-four other CRAN packages --- clearly a much stronger uptake then we ever expected. I took over maintenance at some point, and we are currently in-line with the most recent Boost release 1.60.0. But some people were always lusting after (some) parts of Boost which also require linking. For some libraries (such as Boost.Date_Time in my RcppBDT package) you can set a #define to not require linking (and forego some string formatting or parsing we get from R anyway). Some others are trickier; Boost Regex is one example. I do not think you can use it without linking (but then C++11 helps and finally brings regular expression in the default library). Another library which the systems / network programmers at work would frequently rely upon is Boost.Asio, a cross-platform C++ library for network and low-level I/O programming. It was already used on CRAN by the iptools package by Bob Rudis and Oliver Keyes which shied away from building on Windoze (while 'doze and Boost do get along, but the CRAN system makes it a little more involved). A couple of days ago, I somehow learned that the Asio library actually comes in two flavours: the well-known Boost.Asio (requiring linking), and also as a header-only standalone library! Well, well, well -- so I ran that by Bob and Oliver asking if that would be useful. And the response was a resounding and pretty much immediate Hell, Yes!. So I wrapped it in a package, told Bob about it, who tested it and gave the thumbs up. And after a slightly longer-than-usual on-boarding, the package is now on CRAN. As network (and systems) programming is not entirely uncommon even at CRAN, I hope that this package may find a few more use cases. A new version of iptools should be forthcoming "shortly", including for the first time a Windows build thanks to this package. The indefatigable Martin Morgan already spotted our GitHub repo and scored the first fork. Comments and suggestions about AsioHeaders are welcome via the issue tracker at the GitHub GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

With the world passing 10% IPv6 penetration over the weekend, we see the same old debates coming up again; people claiming IPv6 will never happen (despite several years now of exponential growth!), and that if they had only designed it differently, it would have been all over by now. In particular, people like to point to a 2002 3 article by D. J. Bernstein, complete with rants about how Google would never set up useless IPv6 addresses (and then they did that in 2007 I was involved). It's difficult to understand exactly what the article proposes since it's heavy on calling people idiots and light on actual implementation details (as opposed to when DJB's gotten involved in other fields; e.g. thanks to him we now have elliptical curve crypto that doesn't suck, even if the reference implementation was sort of a pain to build), but I will try to go through it nevertheless and show how I cannot find any way it would work well in practice. One thing first, though: Sorry, guys, the ship has sailed. Whatever genius solution DJB may have thought up that I'm missing, and whatever IPv6's shortcomings (they're certainly there), IPv6 is what we have. By now, you can not expect anything else to arise and take over the momentum; we will either live with IPv6 or die with IPv4. So, let's see what DJB says. As far as I can see, his primary call is for a version of IPv6 where the address space is an extension of the IPv4 space. For sake of discussion, let's call that IPv4+ , although it would share a number of properties with IPv6. In particular, his proposal requires changing the OS and other software on every single end host out there, just as IPv6; he readily admits that and outlines how it's done in rough terms (change all structs, change all configuration files, change all databases, change all OS APIs, etc.). From what I can see, he also readily admits that IPv4 and IPv4+ hosts cannot talk to each other, or more clearly, we cannot start using the extended address space before almost everybody has IPv4+ capable software. (E.g., quote: Once these software upgrades have been done on practically every Internet computer, we'll have reached the magic moment: people can start relying on public IPv6 addresses as replacements for public IPv4 addresses. ) So, exactly how does the IPv4 address space fit into the IPv4+ address space? The article doesn't really say anything about this, but I can imagine only two strategies: Build the IPv4+ space around the IPv4 space (ie., the IPv4 space occupies a little corner of the IPv4+ space, similar to how v4-mapped addresses are used within software but not on the wire today, to let applications do unified treatment of IPv4 addresses as a sort of special IPv6 address), or build it as a hierarchical extension. Let's look at the former first; one IPv4 address gives you one IPv4+ addresses. Somehow this seems to give you all the disadvantages of IPv4 and all the disadvantages of IPv6. The ISP is not supposed to give you any more IPv4+ addresses (or at least DJB doesn't want to contact his ISP about more also saying that the fact that automatic address distribution does not change his argument), so if you have one, you're stuck with one. So you still need NAT. (DJB talks about proxies , but I guess that the way things evolved, this either actually means NAT, or it talks about the practice of application-level proxies such as Squid or SOCKS proxies to reach the Internet, which really isn't commonplace anymore, so I'll assume for the sake of discussion it means NAT.) However, we already do NAT. The IPv4 crunch happened despite ubiquitous NAT everywhere; we're actually pretty empty. So we will need to hand out IPv4+ addresses at the very least to new deployments, and also probably reconfigure every site that wants to expand and is out of IPv4 addresses. ( Site here could mean any organizational unit, such as if your neighborhood gets too many new subscribers for your ISP's local addressing scheme to have enough addresses for you.) A much more difficult problem is that we now need to route these addresses on the wire. Ironically, the least clear part of DJB's plan is step 1, saying we will extend the format of IP packets to allow 16-byte addresses ; how exactly will this happen? For this scheme, I can only assume some sort of IPv4 option that says the stuff in the dstaddr field is just the start and doesn't make sense as an IPv4 address on its own; here are the remaining 12 bytes to complete the IPv4+ address . But now your routers need to understand that format, so you cannot do with only upgrading the end hosts; you also need to upgrade every single router out there, not just the end hosts. (Note that many of these do routing in hardware, so you can't just upgrade the software and call it a day.) And until that's done, you're exactly in the same situation as with IPv4/IPv6 today; it's incompatible. I do believe this option is what DJB talks about. However, I fail to see exactly how it is much better than the IPv6 we got ourselves into; you still need to upgrade all software on the planet and all routers on the planet. The benefit is supposedly that a company or user that doesn't care can just keep doing nothing, but they do need to care, since they need to upgrade 100% of their stuff to understand IPv4+ before we can start even deploying it alongside IPv4 (in contrast with IPv6, where we now have lots of experience in running production networks). The single benefit is that they won't have to renumber until they need to grow, at which point they need to anyway. However, let me also discuss the other possible interpretation, namely that of the IPv4+ address space being an extension of IPv4, ie. if you have 1.2.3.4 in IPv4, you have 1.2.3.4.x.x.x.x or similar in IPv4+. (DJB's article mentions 128-bit addresses and not 64-bit, though; we'll get to that in a moment.) People keep bringing this up, too; it's occasionally been called BangIP (probably jokingly, as in this April Fool's joke) due to the similarity with how explicit mail routing would work before SMTP became commonplace. I'll use that name, even though others have been proposed. The main advantage of BangIP is that you can keep your Internet core routing infrastructure. One way or the other, they will keep seeing IPv4 addresses and IPv4 packets; you need no new peering arrangements etc.. The exact details are unclear, though; I've seen people suggest GRE tunneling, ignoring problems they have through NAT, and I've seen suggestions of IPv4 options for source/destination addresses, also ignoring that someting as innocious as setting the ECN bits has been known to break middleboxes left and right. But let's assume you can pull that off, because your middlebox will almost certainly need to be the point that decapsulates BangIP anyway and converts it to IPv4 on the inside, presumably with a 10.0.0.0/8 address space so that your internal routing can keep using IPv4 without an IPv4+ forklift upgrade. (Note that you now lose the supposed security benefit of NAT, by the way, although you could probably encrypt the address.) Of course, your hosts will need to support IPv4+ still, and you will need some way of communicating that you are on the inside of the BangIP boundary. And you will need to know what the inside is, so that when you communicate on this side, you'll send IPv4 and not IPv4+. (For a home network with no routing, you could probably even just do IPv4+ on the inside, although I can imagine complications.) But like I wrote above, experience has shown us that 32 extra bits isn't enough. One layer of NAT isn't doing it, we need two. You could imagine the inter-block routability of BangIP helping a fair bit here (e.g., a company with too many machines for 10.0.0.0/8 could probably easily get more addresses for more external IPv4 addresses, yielding 10.0.0.0/8 blocks), but ultimately, it is a problem that you chop the Internet off in two distinct halves that work very differently. My ISP will probably want to use BangIP for itself, meaning I'm on the outside of the core; how many of those extra bits will they allocate for me? Any at all? Having multiple levels of bang sounds like pain; effectively we're creating a variable-length address. Does anyone ever want that? From experience, when we're creating protocols with variable-length addresses, people just tend to use the maximum level anyway, so why not design it with 128-bit to begin with? (The original IP protocol proposals actually had variable-length addresses, by the way.) So we can create our 32/96 BangIP , where the first 32 bits are for the existing public Internet, and then every IPv4 address gives you a 2^96 addresses to play with. (In a sense, it reminds me of 6to4, which never worked very well and is now thankfully dead.) However, this makes the inside/outside-core problem even worse. I now need two very different wire protocols coexisting on the Internet; IPv4+ (which looks like regular IPv4 to the core) for the core, and a sort of IPv4+-for-the-outside (similar to IPv6) outside it. If I build a company network, I need to make sure all of my routers are IPv4+-for-the-outside and talk that, while if I build the Internet core, I need to make sure all of my connections are IPv4 since I have no guarantee that I will be routable on the Internet otherwise. Furthermore, I have a fixed prefix that I cannot really get out of, defined by my IPv4 address(es). This is called hierarchical routing , and the IPv6 world gave it up relatively early despite it sounding like a great idea at first, because it makes multihoming a complete pain: If I have an address 1.2.3.4 from ISP A and 5.6.7.8 from ISP B, which one do I use as the first 32 bits of my IPv4+ network if I want it routable on the public Internet? You could argue that the solution for me is to get an IPv4 PI netblock (supposedly a /24, since we're not changing the Internet core), but we're already out of those, which is why we started this thing to begin with. Furthermore, if the IPv4/IPv4+ boundary is above my immediate connection to the Internet (say, ISP A doesn't have an IPv4 address, just IPv4+), I'm pretty hosed; I cannot announce an IPv4 netblock in BGP. The fact that the Internet runs on largely the same protocol everywhere is a very nice thing; in contrast, what is described here really would be a mess! So, well. I honestly don't think it's as easy to just do extension instead of alternative when it comes to the address spaces. We'll just need to deal with the pain and realize that upgrading the equipment and software is the larger part of the job anyway, and we'll need to do that no matter what solution we go with. Congrats on reaching 10%! Now get to work with the remaining 90%.

Today I committed a set of changes to the TeX Live subversion repository that should pave the way for better security handling in the future. Work is underway to use strong cryptographic signatures to verify that packages downloaded and installed into a TeX Live installation have not been tinkered with. While there is still a long way to go and to figure out, the current changes already improve the situation considerably. Status up to now Although we did ship size and checksum information within the TeX Live database, these information were only considered by the installer when re-starting an installation to make sure that the downloaded packages are the ones we should use. Neither the installer nor tlmgr did use the checksum to verify that the downloaded packages is correct, relying mostly on the fact that the packages are xz-compressed and would create rubbish when there is a transfer error. Although none of us believes that there is a serious interest in tinkering with the TeX Live distribution maybe to steal just another boring scientific paper? the door was still open. Changes implemented The changes committed to the repository today, which will be in a testing phase to get rid of bugs, consists of the following:

• unification of installation routines: it didn t make sense to have duplication of the actual download and unpack code in the installer and in tlmgr, so the code base was simplified and unified, and both the installer and tlmgr now use the same code paths to obtain, unpack, and install a package.
• verification of size and checksum data: hand-in-hand with the above change, verification of downloaded packages based on both the size as well as the checksum is now carried out.

Together these two changes will allow install-tl/tlmgr to verify that a package (.tar.xz) is according to the information in the accompanying texlive.tlpdb. This still leaves the option of tampering with a package and updating the texlive.tlpdb for the fixes checksums/sizes. Upcoming changes For the future we do plan mostly two things:

• switch to stronger hashing algorithm: till now we use md5, but we will switch to sha512 instead.
• GnuPG signing of the checksum file of the texlive.tlpdb, that is detached signature of texlive.tlpdb.checksum

The last step above will give a very high level of security, as it will be not practically possible to alter the information in the texlive.tlpdb, thus no tampering with the checksum information of the containers, and in turn no tampering with the actual containers will be possible. Restrictions Due to the wide range of supported architectures and operating systems, we will not make verification obligatory, but if a gpg binary is found, it will be used to verify the downloaded texlive.tlpdb.checksum. Details have to be hammered out and the actual programming has to be done, but we are on the way. Enjoy.

I m organizing an event at the University of Washington in Seattle that involves a reading, the screening of a documentary film, and a Q&A about Aaron Swartz. The event coincides with the third anniversary of Aaron s death and the release of a new book of Swartz s writing that I contributed to. The event is free and open the public and details are below:

WHEN: Wednesday, January 13 at 6:30-9:30 p.m.

WHERE: Communications Building (CMU) 120, University of Washington

We invite you to celebrate the life and activism efforts of Aaron Swartz, hosted by UW Communication professor Benjamin Mako Hill. The event is next week and will consist of a short book reading, a screening of a documentary about Aaron s life, and a Q&A with Mako who knew Aaron well details are below. No RSVP required; we hope you can join us.

Aaron Swartz was a programming prodigy, entrepreneur, and information activist who contributed to the core Internet protocol RSS and co-founded Reddit, among other groundbreaking work. However, it was his efforts in social justice and political organizing combined with his aggressive approach to promoting increased access to information that entangled him in a two-year legal nightmare that ended with the taking of his own life at the age of 26.

January 11, 2016 marks the third anniversary of his death. Join us two days later for a reading from a new posthumous collection of Swartz s writing published by New Press, a showing of The Internet s Own Boy (a documentary about his life), and a Q&A with UW Communication professor Benjamin Mako Hill a former roommate and friend of Swartz and a contributor to and co-editor of the first section of the new book. If you re not in Seattle, there are events with similar programs being organized in Atlanta, Chicago, Dallas, New York, and San Francisco. All of these other events will be on Monday January 11 and registration is required for all of them. I will be speaking at the event in San Francisco.

The Munin project is moving slowly closer to a Munin 3 release. In parallel, the Debian packaging is changing, too. The new web interface is looking much better than the traditional web-1.0 interface normally associated with munin.

New package layout

perl libraries All the Munin perl libraries are placed in libmunin-*-perl , and split into separate packages, where the split is decided mostly on dependencies. If you don t want to monitor samba, or SNMP, or MySQL, there should be no need to have those libraries installed. That does mean more binary packages, on the other hand.

Munin master Munin now runs as a standalone HTTPD, it no longer graphs from cron, nor does it run as CGI or FastCGI scripts. The user munin grants read-write access, while the group munin grants read only access. The new web interface runs as the munin-httpd user, which is member of the munin group. There is a munin service. For now, it runs rrdcached for the munin user and RRD directory.

munin node The perl munin-node and the compiled munin-node-c should be interchangeable, and be able to run the same plugins. Munin node, and Munin async node, should be wholly separate from the munin master. It should be possible to use the perl munin-node package, and the

munin plugins The munin plugins are placed separate packages named munin-plugins-* . The split is based on monitoring subject, or dependencies. They depend on appropriate libmunin-plugin-*-perl packages The munin-plugins-c package, which is is from the munin-node-c source, contains a number of compiled plugins which should use less resources than their shell, perl or python equivalents. Plugins from other sources than munin must work similar to the ones from munin . More work on this is needed.

Testing Late December 2015, I set up Jenkins, with jenkins-debian-glue to build packages, test with autopkgtest and and update my development apt repository on each commit. That helped developing and testing the new Munin packages. The packages are not quite ready to upload to experimental, but they are continuously deployed to weed out bugs. They can be found in my packaging apt repo. (The usual non-guarantees apply, handle with care, keep away from small children, etc )

Comments Munin developers, packagers and users hang out on #munin on the OFTC network. Please drop by if you have questions or comments.

What happened in the reproducible builds effort between December 27th and January 2nd: Infrastructure dak now silently accepts and discards .buildinfo files (commit 1, 2), thanks to Niels Thykier and Ansgar Burchardt. This was later confirmed as working by Mattia Rizzolo. Packages fixed The following packages have become reproducible due to changes in their build dependencies: banshee-community-extensions, javamail, mono-debugger-libs, python-avro. The following packages became reproducible after getting fixed:

• avrdude/6.2-5 by Milan Kupcevic.
• blosxom/2.1.2-2 uploaded by Rhonda D'Vine, original patches (#777292, #793001) by Chris Lamb and akira.
• buzztrax/0.10.2-2 uploaded by Sebastian Dr ge, original patch by Chris Lamb, fixed upstream.
• dx/1:4.4.4-8 by Graham Inggs.
• gap-guava/3.12+ds1-3 by Jerome Benoit.
• goffice/0.10.26-1 uploaded by Dmitry Smirnov, fixed upstream.
• gunroar/0.15.dfsg1-8 uploaded by Markus Koschany, original patch by Reiner Herrmann.
• iceweasel/43.0.2-1 by Mike Hommey.
• ii-esu/1.0a.dfsg1-7 uploaded by Markus Koschany, original patch by Reiner Herrmann.
• jing-trang/20131210+dfsg+1-4 by Samuel Thibault.
• mstflint/4.1.0+1.46.gb1cdaf7-1 by Mehdi Dogguy.
• mu-cade/0.11.dfsg1-9 uploaded by Markus Koschany, original patch by Reiner Herrmann.
• mumble/1.2.12-1 by Christopher Knadle.
• netris/0.52-10 by Rhonda D'Vine, original patches (#778201, #793707) by Chris Lamb and akira.
• onboard/1.1.2-2 uploaded by Mike Gabriel, original patch by Reiner Herrmann.
• parsec47/0.2.dfsg1-7 uploaded by Markus Koschany, original patch by Reiner Herrmann.
• pathological/1.1.3-14 by Markus Koschany.
• projectl/1.001.dfsg1-8 uploaded by Markus Koschany, original patch by Reiner Herrmann.
• re2c/0.15.3-1 by JCF Ploemen.
• s3d/0.2.2-14 by Sven Eckelmann.
• tulip/4.8.0dfsg-2 by Yann Dirson.
• val-and-rick/0.1a.dfsg1-5 uploaded by Markus Koschany, original patch by Reiner Herrmann.
• xterm/321-1 by Sven Joachim.

Some uploads fixed some reproducibility issues, but not all of them:

• debian-installer/20160101 uploaded by Cyril Brulebois with several fixes from Steven Chamberlain.
• drbd-utils/8.9.5-1 by Apollon Oikonomopoulos.
• hhvm/3.11.0+dfsg-1 by Faidon Liambotis.
• rkward/0.6.4-1 uploaded by Thomas Friedrichsmeier, original patch by Philip Rinn.
• rsbackup/3.0-2 uploaded by Matthew Vernon, original patches (#777394, #793716) by Chris Lamb and akira.
• tin/1:2.3.2-1 by Marco d'Itri.
• transdecoder/2.0.1+dfsg-2 uploaded by Andreas Tille, original patch by Chris Lamb.
• tumiki-fighters/0.2.dfsg1-7 uploaded by Markus Koschany, original patch by Reiner Herrmann.

Untested changes:

• fltk1.1/1.1.10-20 by Aaron M. Ucko, currently FTBFS.
• fltk1.3/1.3.3-5 by Aaron M. Ucko, currently FTBFS.

reproducible.debian.net The testing distribution (the upcoming stretch) is now tested on armhf. (h01ger) Four new armhf build nodes provided by Vagrant Cascandian were integrated in the infrastructer. This allowed for 9 new armhf builder jobs. (h01ger) The RPM-based build system, koji, is now in unstable and testing. (Marek Marczykowski-G recki, Ximin Luo). Package reviews 131 reviews have been removed, 71 added and 53 updated in the previous week. 58 new FTBFS reports were made by Chris Lamb and Chris West. New issues identified this week: nondeterminstic_ordering_in_gsettings_glib_enums_xml, nondeterminstic_output_in_warnings_generated_by_breathe, qt_translate_noop_nondeterminstic_ordering. Misc. Steven Chamberlain explained in length why reproducible cross-building across architectures mattered, and posted results of his tests comparing a stage1 debootstrapped chroot of linux-i386 once done from official Debian packages, the others cross-built from kfreebsd-amd64.

The New Press has published a new collection of Aaron Swartz s writing called The Boy Who Could Change the World: The Writings of Aaron Swartz. I worked with Seth Schoen to introduce and help edit the opening section of book that includes Aaron s writings on free culture, access to information and knowledge, and copyright. Seth and I have put our introduction online under an appropriately free license (CC BY-SA). Over the last week, I ve read the whole book again. I think the book really is a wonderful snapshot of Aaron s thought and personality. It s got bits that make me roll my eyes, bits that make me want to shout in support, and bits that continue to challenge me. It all makes me miss Aaron terribly. I strongly recommend the book. Because the publication is post-humous, it s meant that folks like me are doing media work for the book. In honor of naming the book their progressive pick of the week, Truthout has also published an interview with me about Aaron and the book. Other folks who introduced and/or edited topical sections in the book are David Auerbach (Computers), David Segal (Politics), Cory Doctorow (Media), James Grimmelmann (Books and Culture), and Astra Taylor (Unschool). The book is introduced by Larry Lessig.

Would you like to hike a mountain? That question caught me by surprise. It was early in 2000, and I had flown to Tucson for a job interview. Ian Murdock was starting a new company, Progeny, and I was being interviewed for their first hire. Well, I thought, hiking will be fun. So we rode a bus or something to the top of the mountain and then hiked down. Our hike was full of well, everything. Ian talked about Tucson and the mountains, about his time as the Debian project leader, about his college days. I asked about the plants and such we were walking past. We talked about the plans for Progeny, my background, how I might fit in. It was part interview, part hike, part two geeks chatting. Ian had no HR telling him you can t go hiking down a mountain with a job candidate, as I m sure HR would have. And I am glad of it, because even 16 years later, that is still by far the best time I ever had at a job interview, despite the fact that it ruined the only pair of shoes I had brought along I had foolishly brought dress shoes for a, well, job interview. I guess it worked, too, because I was hired. Ian wanted to start up the company in Indianapolis, so over the next little while there was the busy work of moving myself and setting up an office. I remember those early days Ian and I went computer shopping at a local shop more than once to get the first workstations and servers for the company. Somehow he had found a deal on some office space in a high-rent office building. I still remember the puzzlement on the faces of accountants and lawyers dressed up in suits riding in the elevators with us in our shorts and sandals, or tie-die, next to them. Progeny s story was to be a complicated one. We set out to rock the world. We didn t. We didn t set out to make lasting friendships, but we often did. We set out to accomplish great things, and we did some of that, too. We experienced a full range of emotions there elation when we got hardware auto-detection working well or when our downloads looked very popular, despair when our funding didn t come through as we had hoped, being lost when our strategy had to change multiple times. And, as is the case everywhere, none of us were perfect. I still remember the excitement after we published our first release on the Internet. Our little server that could got pegged at 100Mb of outbound bandwidth (that was something for a small company in those days.) The moment must have meant something, because I still have the mrtg chart from that day on my computer, 15 years later. We made a good Linux distribution, an excellent Debian derivative, but commercial success did not flow from it. In the succeeding months, Ian and the company tried hard to find a strategy that would stick and make our big break. But that never happened. We had several rounds of layoffs when hoped-for funding never materialized. Ian eventually lost control of the company, and despite a few years of Itanium contract work after I left, closed for good. Looking back, Progeny was life compressed. During the good times, we had joy, sense of accomplishment, a sense of purpose at doing something well that was worth doing. I had what was my dream job back then: working on Debian as I loved to do, making the world a better place through Free Software, and getting paid to do it. And during the bad times, different people at Progeny experienced anger, cynicism, apathy, sorrow for the loss of our friends or plans, or simply a feeling to soldier on. All of the emotions, good or bad, were warranted in their own way. Bruce Byfield, one of my co-workers at Progeny, recently wrote a wonderful memoriam of Ian. He wrote, More than anything, he wanted to repeat his accomplishment with Debian, and, naturally he wondered if he could live up to his own expectations of himself. That, I think, was Ian s personal tragedy that he had succeeded early in life, and nothing else he did with his life could quite measure up to his expectations and memories. Ian was not the only one to have some guilt over Progeny. I, for years, wondered if I should have done more for the company, could have saved things by doing something more, or different. But I always came back to the conclusion I had at the time: that there was nothing I could do a terribly sad realization. In the years since, I watched Ubuntu take the mantle of easy-to-install Debian derivative. I saw them reprise some of the ideas we had, and even some of our mistakes. But by that time, Progeny was so thoroughly forgotten that I doubt they even realized they were doing it. I had long looked at our work at Progeny as a failure. Our main goal was never accomplished, our big product never sold many copies, our company eventually shuttered, our rock-the-world plan crumpled and forgotten. And by those traditional measurements, you could say it was a failure. But I have come to learn in the years since that success is a lot more that those things. Success is also about finding meaning and purpose through our work. As a programmer, success is nailing that algorithm that lets the application scale 10x more than before, or solving that difficult problem. As a manager, success is helping team members thrive, watching pieces come together on projects that no one person could ever do themselves. And as a person, success comes from learning from our experiences, and especially our mistakes. As J. Michael Straczynski wrote in a Babylon 5 episode, loosely paraphrased: Maybe this experience will be a good lesson. Too bad it was so painful, but there ain t no other kind. The thing about Progeny is this Ian built a group of people that wanted to change the world for the better. We gave it our all. And there s nothing wrong with that. Progeny did change the world. As us Progeny alumni have scattered around the country, we benefit from the lessons we learned there. And many of us were different , sort of out of place before Progeny, and there we found others that loved C compilers, bootloaders, and GPL licenses just as much as we did. We belonged, not just online but in life, and we went on to pull confidence and skill out of our experience at Progeny and use them in all sorts of ways over the years. And so did Ian. Who could have imagined the founder of Debian and Progeny would one day lead the cause of an old-guard Unix turning Open Source? I run ZFS on my Debian system today, and Ian is partly responsible for that and his time at Progeny is too. So I can remember Ian, and Progeny, as a success. And I leave you with a photo of my best memento from the time there: an original unopened boxed copy of Progeny Linux.

What happened in the reproducible builds effort between December 6th and December 12th: Toolchain fixes

• Steven Chamberlain uploaded makefs/20100306-6 which adds a -T flag which will clamp superblock and file timestamps to a given time in epoch format.
• Emmanuel Bourg uploaded maven-debian-helper/2.0~exp3 which disable the timestamps and set the locale to en_US when generating the javadoc.

Reiner Herrmann rebased our experimental version of doxygen on version 1.8.9.1-6. Chris Lamb submitted a patch to make the manpages generated by ruby-ronn reproducible by using the locale-agnostic %Y-%m-%d for the dates. Daniel Kahn Gillmor took another shot at the issue of source path captured in DWARF symbols. A patch has been sent for review by GCC upstream to add the ability to read an environment variable with -fdebug-prefix-map. Packages fixed The following 24 packages have become reproducible due to changes in their build dependencies: gkeyfile-sharp, gprbuild, graphmonkey, gthumb, haskell-yi-language, ion, jackson-databind, jackson-dataformat-smile, jackson-dataformat-xml, jnr-ffi, libcommons-net-java, libproxy, maven-shared-utils, monodevelop-database, mydumper, ndesk-dbus, nini, notify-sharp, pixz, protozero, python-rtslib-fb, slurm-llnl, taglib-sharp, tomboy-latex. The following packages became reproducible after getting fixed:

• canl-java/2.1.1-4 by Mattias Ellert.
• codonw/1.4.4-2 by Sascha Steinbiss.
• cpl-plugin-amber/3.5.1+dfsg-4 by Ole Streicher.
• cpl-plugin-visir/3.5.1+dfsg-4 by Ole Streicher.
• flightcrew/0.7.2+dfsg-6 by Mattia Rizzolo.
• git-annex/5.20151208-1 uploaded by Richard Hartmann, fix by Joey Hess.
• gnudatalanguage/0.9.5-5 uploaded by Axel Beckert, fix by Ole Streicher.
• gramps/4.2.1~dfsg-2 by Ross Gammon.
• jglobus/2.1.0-3 by Mattias Ellert.
• kbtin/1.0.16-1 by Adam Borowski, also reported by Chris Lamb.
• lava-tool/0.14-1 by Neil Williams.
• logbook/0.12.3-1 by Agustin Henze.
• ltrsift/1.0.2-5 by Sascha Steinbiss.
• maven-war-plugin/2.2-1 by Emmanuel Bourg.
• relion/1.4+dfsg-1 by Roland Fehrenbacher.
• u-boot/2015.10+dfsg1-4 by Vagrant Cascadian.
• voms-api-java/3.0.5-3 by Mattias Ellert.
• voms-clients-java/3.0.6-3 by Mattias Ellert.
• xgalaga/2.1.1.0-5 by Markus Koschany.

Some uploads fixed some reproducibility issues, but not all of them:

• aptitude/0.7.5-1 by Manuel A. Fernandez Montecelo.
• asc/2.6.1.0-1 by Markus Koschany.
• grib-api/1.14.3-2 by Enrico Zini.
• jacal/1b9-6 uploaded by Barak A. Pearlmutter, original patch by Chris Lamb.

These uploads might have fixed reproducibility issues but could not be tested yet:

• ncbi-tools6/6.1.20120620-9 by Aaron M. Ucko; currently FTBFS.
• vowpal-wabbit/8.1.1-1 uploaded by Yaroslav Halchenko, original patch by Chris Lamb; currently FTBFS.

Patches submitted which have not made their way to the archive yet:

• #807159 on monit by Chris Lamb: add support for setting the build date using SOURCE_DATE_EPOCH (already fixed upstream).
• #807161 on suomi-malaga by Chris Lamb: add support for setting the build date using SOURCE_DATE_EPOCH.
• #807475 on glance by Chris Lamb: stop recording the number of CPUs on the build system.
• #807605 on guiqwt by Chris Lamb: add support for setting the copyright year using SOURCE_DATE_EPOCH.

reproducible.debian.net Files created with diffoscope now have diffoscope in their name instead debbindiff. (h01ger) Hostnames of first and second build node are now recorded and shown in the build history. (Mattia Rizzolo) Exchanges have started with F-Droid developers to better understand what would be required to test F-Droid applications. (h01ger) A first small set of Fedora 23 packages is now also being tested while development on a new framework for testing RPMs in general has begun. A new Jenkins job has been added to set up to mock, the build system used by Fedora. Another new job takes care of testing RPMs from Fedora 23 on x86_64. So far only 151 packages from the buildsys-build group are tested (currently all unreproducible), but the plan is to build all 17,000 source packages in Fedora 23 and rawhide. The page presenting the results should also soon be improved. (h01ger, Dhiru Kholia) For Arch Linux, all 2223 packages from the extra repository will also be tested from now on. Packages in extra" are tested every four weeks, while those from core every week. Statistics are now displayed alongside the results. (h01ger) jenkins.debian.net has been updated to jenkins-job-builder version 1.3.0. Many job configurations have been simplified and refactored using features of the new version. This was another milestone for the jenkins.debian.org migration. (Phil Hands, h01ger) diffoscope development Chris Lamb announced try.diffoscope.org: an online service that runs diffoscope on user provided files. Improvements are welcome. The application is licensed under the AGPLv3. On diffoscope itself, most pending patches have now been merged. Expect a release soon! Most of the code implementing parallel processing has been polished. Sadly, unpacking archive is CPU-bound in most cases, so the current thread-only implementation does not offer much gain on big packages. More work is still require to also add concurrent processes. Documentation update Ximin Luo has started to write a specification for buildinfo files that could become a larger platform than the limited set of features that were thought so far for Debian .buildinfo. Package reviews 113 reviews have been removed, 111 added and 56 updated in the previous week. 42 new FTBFS bugs were opened by Chris Lamb and Niko Tyni. New issues identified this week: timestamps_in_documentation_generated_by_docbook_dbtimestamp, timestamps_in_sym_l_files_generated_by_malaga, timestamps_in_edj_files_generated_by_edje_cc. Misc. Chris Lamb presented reproducible builds at skroutz.gr.

I woke up this morning and realized that for the first time since 17 April 2001, I am no longer a member of the Debian Technical Committee. My departure from the committee is a consequence of the Debian General Resolution "limiting the term of the technical committee members" that was passed amending the Debian Constitution nearly a year ago. As the two longest-serving members, both over the term limit, Steve Langasek and I completed our service yesterday. In early March 2015, I stepped down from the role of chairman after serving in that role for the better part of a decade, to help ensure a smooth transition. Don Armstrong is now serving admirably in that role, I have the utmost respect for the remaining members of the TC, and the process of nominating replacements for the two now-vacant seats is already well underway. So, for the Debian project as a whole, today is really a non-event... which is exactly as it should be! Debian has been a part of my life since 1994, and I sincerely hope to be able to remain involved for many years to come!

FTP assistant Due to Christmas, I only marked 254 packages for accept and rejected 17 of them. I had to send 14 emails to maintainers. Squeeze LTS This was my eighteenth month that I did some work for the Squeeze LTS initiative, started by Raphael Hertzog at Freexian. As other members of the LTS team had to give back some hours of their workload, this month my initial workload of 18.25h had been increased to 21.25h. Altogether I uploaded those DLAs:

• [DLA 370-1] bind9 security update
• [DLA 373-1] libxml2 security update
• [DLA 375-1] libpng security update

I also started to work on CVEs for packages t-coffee and pitivi, only to recognize that the versions in Squeeze are not affected. Further I prepared patches for passenger and srtp but I could not test them yet, so an upload will be in January. This month I also experienced something strange. Due to the upload of the new version of mysql, I had to process a package for squeeze-lts in NEW. I seldom see a package, that creates so much red output from lintian. I assume this always happens when an old package will be checked by the latest lintian and is a good sign for all the development within Debian. I also tested Raphaels patch for #796784 and could confirm that everything works as expected. Unfortunately the php5 upload must be delayed until Januar. This month I also had another week of frontdesk duties. Other stuff The Advent season is over and the Debian Med Advent Calendar is full to bursting. The incredible number of 150 bugs have been closed this year! Due to the GSL transition a new upload of all meep packages had to be done and all in all I could close #748822, #807210, #807212, #807213, #807214, #807215 and #807747.

Here is my monthly update covering a large part of what I have been doing in the free software world (previously):

• Released try.diffoscope.org, a hosted version of the diffoscope in-depth and content aware diff utility. Thanks to Bytemark for sponsoring the hardware.
• Announced travis.debian.net, a hosted script to easily test and build Debian packages on Travis CI.
• Wrote an Chrome extension for the FastMail web interface that reminds you if you neglect to attach a file if you mention one in the body. (Repo)
• Added graphical image comparison using to diffoscope using libcaca. (commit)
• Did further work on a free-software healthcare project.
• Added the ability to hide Zwift activities in my Strava Enhancement Suite Chrome extension. (#40)
• Updated the Django web framework documentation to discourage the use of filenames in /tmp with predictable names. (#5864)
• Corrected a Python 3.5-ism and a double-encoding issue in django-slack.
• Increased the size of the search form on codesearch.debian.net. (#59)
• Added support for filenames alongside the existing URL support for book cover images in my media database. (code)
• Corrected a trivial typo in FreeBSD's pkg package management tool. (#1353)
• Attended the 32nd Chaos Communication Congress.

---

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I have been paid to work 21.25 hours on Debian LTS. During this time I worked on the following things:

• Sent a first patch and later an updated patch to modify DAK so that it can send the accept/reject mails to the signer of the upload instead of the maintainer. Details in #796784.
• Uploaded MySQL 5.5 compabitility fixes for phpmyadmin and postfix-policyd so that we could release MySQL 5.5 as an upgrade option MySQL 5.1 (see DLA 359-1).
• Released DLA 361-1 on bouncycastle after having gotten the green light from upstream.
• Released DLA 362-1 on dhcpd fixing three CVE.
• Released DLA 366-1 on arts fixing one CVE.
• Released DLA 367-1 on kdelibs fixing one CVE.
• Handled the LTS frontdesk for a whole week.
• Sponsored the upload of foomatic-filters for DLA 371-1.
• Filed #808256 and #808257 to get libnsbmp/libnsgif removed. Both packages had recent CVE and were sitting unused in Debian since their introduction 6 years ago
• Released DLA 372-1 announcing the end of support of virtualbox-ose.
• Updated git repository of debian-security-support to account for the former change and also took care of a few pending issues.
• Released DLA 376-1 on mono to fix one CVE.
• Added some initial DEP-8 tests to python-django that will help to ensure that a security update doesn t break the package.

Distro Tracker I put a big focus on tracker.debian.org work this month. I completed the switch of the mail interface from packages.qa.debian.org to tracker.debian.org and I announced the change on debian-devel-announce. The changes resulted in a few problems that I quickly fixed (like #807073) and some other failures seen only by me and that were generated by weird spam messages (did you know that a subject can t have a newline character but that it can be encoded and folded over multiple lines?). Related to that I fixed some services so that they send their mails to tracker.debian.org directly instead of relying on the old emails (they get forwarded for now but it would be nice to be able to get rid of that forward). I updated (with the help of Lucas Nussbaum) the service that forwards the Launchpad bugs to the tracker, I sent a patch to update the @packages.debian.org aliases (not yet applied), I updated the configuration of all git commit notice scripts in the Alioth collab-maint and python-modules project (many remain to be done). I asked Ubuntu s Merge-O-Matic to use the new emails as well (see LP 1525497). DAK and the Debian BTS still have to be updated, as of yet nobody reacted to my announce last but not least I updated many wiki pages which duplicated the instructions to setup the commit notice sent to the PTS. While on a good track I opted to tackle the long-standing RC bug that was plaguing tracker.debian.org (#789183), so I updated the codebase to rely on Twitter s bootstrap v4 instead of v2. I had to switch to something else for the icons since glyphicons is no longer provided as part of bootstrap and the actual license for the standalone version was not suitable for use. I opted for Github s Octicons. I made numerous little improvements while doing that (closing some bugs in the process) and I believe that the result is more pleasant to use. I also did a lot of bug triage and fixed a few small issues like the incomplete architecture list (#793547), or fixing a page used only by people with javascript disabled that was not working. Or the invalid links for packages still using CVS (ugh, see #561228). Misc packaging Django. After having added DEP-8 tests (as part of my LTS work, see above), I discovered that the current version in unstable did not pass its test suite so I filed the issue upstream (ticket 26016) and added the corresponding patch. And I encouraged others to update python-bcrypt in Debian to a newer version that would have worked with Django 1.9 (see #803096). I also fixed another small issue in Django (see ticket 26017 with my pull request that got accepted). I asked the release managers to consider accepting the latest 1.7.x version in jessie (see #807654) but I have gotten zero answer so far. And I m not the only one waiting an answer. It s a bit of a sad situation we still have a few weeks until the next point release but for once I do it in advance and I would love to have timely feedback. Last but not least, I started the maintaining the current LTS release (1.8.x) in jessie-backports. Tryton. I upgraded to Tryton 3.8 and discovered an issue that I filed in #806781. I sponsored 5 new tryton modules for Matthias Behrle (who is DM) as well as one security upload (for CVE-2015-0861). Debian Handbook. I uploaded a new version to Debian Unstable and requested (to the release managers) the permission to upload a backport of it to jessie so that jessie has a version of the package that documents jessie and not wheezy contrary to my other Django request, this one should be non-controversial but I also have had zero answer so far, see #807515. Misc. I filed #808583 when sbuild stopped working with Perl 5.22. I handled #807860 on publican, I found the corresponding upstream ticket and discovered a work around with the help of upstream (see here). Kali related work I reported a bug to #debian-apt about apt miscalculating download size (ending up with 18 EB!) which resulted in a fix here in version 1.1.4. Installing a meta-package that needed more than 2GB was no longer possible without this fix and we have a kali-linux-all metapackage in that situation that gets regularly installed in a Jenkins test. I added captcha support to Distro Tracker and enabled this feature on pkg.kali.org. I filed #808863 against uhd-host because it was not possible to install the package in a systemd-nspawn s managed chroot where /proc is read-only. And we started using this to test dist-upgrade from one version of Kali to the next Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

As of a few minutes ago, the old weblog on christoph-egger.org is past. I've added redirects for all the entries to the new one at weblog.siccegge.de.if you find any dead links please contact me so I can fix it up. Note that comments are gone. I'll try to include the already present comments on the new blog some time in the future. Not sure if I will ever add a comment function again (though chronicle seems to have some support for that)

I made a change recently on the gjay program. gjay is a gtk program that basically analyzes your music and makes playlists. There is a gui frontend and a analyzer back-end and they communicate through a pipe. One really useful debugging option gtk has is to make warnings fatal, so when gtk finds one it crashes at that point and you can use gdb to trap it. The flag is g-fatal-warnings. I have been updating gjay and initially it didn t have this option, so I needed to add the gtk options, which is a simple one-liner. But then gjay gave some odd errors about XCB. Often, but not every time gjay started on stderr the following cryptic messages appear:

[xcb] Unknown sequence number while processing queue
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
[xcb] Aborting, sorry about that.
forktest: ../../src/xcb_io.c:274: poll_for_event: Assertion  !xcb_xlib_threads_sequence_lost' failed.

OK, so part of my init sequence was unhappy. The odd thing was it only appeared when I added the gtk options. I narrowed it down with some test code which displayed the error but stripped all the other parts out. What we got going on here is a simple gtk program. Line 11 with the gtk_get_option_group() was the line I added. The problem is that the child process is not quite setup and when the parent goes into gtk_main then you get those xcb errors. The options need to be parsed before the fork, because one of the options is to not fork (it just runs the child code directly and uses stdout with no GUI). Gjay is obviously a lot more complex than this, but follows the same pattern. There is the front-end looping through gtk_main and the back-end looping on g_main_run. The backend needs to use glib not gtk as there is a non-gui option which can be used. The solution is actually (but not obviously to me) in the documentation for gtk_get_option_group. The parameter for that function whether to open the default display when parsing the commandline arguments . Changing the TRUE to FALSE on line 11 stops XCB from complaining! The screen still appears fine but not more strange XCB messages.

Not only do I keep incrementing version numbers faster than ever before, APT also keeps getting faster. But not only that, it also has some bugs fixed and the cache is now checked with a hash when opening. Important fix for 1.1.6 regression Since APT 1.1.6, APT uses the configured xz compression level. Unfortunately, the default was set to 9, which requires 674 MiB of RAM, compared to the 94 MiB required at level 6. This caused the test suite to fail on the Ubuntu autopkgtest servers, but I thought it was just some temporary hickup on their part, and so did not look into it for the 1.1.7, 1.1.8, and 1.1.9 releases. When the Ubuntu servers finally failed with 1.1.9 again (they only started building again on Monday it seems), I noticed something was wrong. Enter git bisect. I created a script that compiles the APT source code and runs a test with ulimit for virtual and resident memory set to 512 (that worked in 1.1.5), and let it ran, and thus found out the reason mentioned above. The solution: APT now defaults to level 6. New Features APT 1.1.8 introduces /usr/lib/apt/apt-helper cat-file which can be used to read files compressed by any compressor understood by APT. It is used in the recent apt-file experimental release, and serves to prepare us for a future in which files on the disk might be compressed with a different compressor (such as LZ4 for Contents files, this will improve rred speed on them by factor 7). David added a feature that enables servers to advertise that they do not want APT to download and use some Architecture: all contents when they include all in their list of architectures. This is to allow archives to drop Architecture: all packages from the architecture-specific content files, to avoid redundant data and (thus) improve the performance of apt-file. Buffered writes APT 1.1.9 introduces buffered writing for rred, reducing the runtime by about 50% on a slowish SSD, and maybe more on HDDs. The 1.1.9 release is a bit buggy and might mess up things when a write syscall is interrupted, this is fixed in 1.1.10. Cache generation improvements APT 1.1.9 and APT 1.1.10 improve the cache generation algorithms in several ways: Switching a lookup table from std::map to std::unordered_map, providing an inline isspace_ascii() function, and inlining the tolower_ascii() function which are tiny functions that are called a lot. APT 1.1.10 also switches the cache s hash function to the DJB hash function and increases the default hash table sizes to the smallest prime larger than 15000, namely 15013. This reduces the average bucket size from 6.5 to 4.5. We might increase this further in the future. Checksum for the cache, but no more syncs Prior to APT 1.1.10 writing the cache was a multi-part process:

1. Write the the cache to a temporary file with the dirty bit set to true
2. Call fsync() to sync the cache
3. Write a new header with the dirty bit set to false
4. Call fsync() to sync the new header
5. (Rename the temporary file to the target name)

The last step was obviously not needed, as we could easily live with an intact cache that has its dirty field set to false, as we can just rebuild it. But what matters more is step 2. Synchronizing the entire 40 or 50 MB takes some time. On my HDD system, it consumed 56% of the entire cache generation time, and on my SSD system, it consumed 25% of the time. APT 1.1.10 does not sync the cache at all. It now embeds a hashsum (adler32 for performance reasons) in the cache. This helps ensure that no matter what parts of the cache are written in case of some failure somewhere, we can still detect a failure with reasonable confidence (and even more errors than before). This means that cache generation is now much faster for a lot of people. On the bad side, commands like apt-cache show that previously took maybe 10 ms to execute can now take about 80 ms. Please report back on your performance experience with 1.1.10 release, I m very interested to see if that works reasonably for other people. And if you have any other idea how to solve the issue, I d be interested to hear them (all data needs to be written before the header with dirty=0 is written, but we don t want to sync the data). Future work We seem to have a lot of temporary (?) std::string objects during the cache generation, accounting for about 10% of the run time. I m thinking of introducing a string_view class similar to the one proposed for C++17 and make use of that. I also thought about calling posix_fadvise() before starting to parse files, but the cache generation process does not seem to spend a lot of its time in system calls (even with all caches dropped before the run), so I don t think this will improve things. If anyone has some other suggestions or patches for performance stuff, let me know.
Filed under: Debian, Ubuntu

24 Dec. 63 Dear Uncle Ron and Aunt Helen, It has been quite awhile since I have written you folks. I am not too much of one for writing, and when I do write, I keep putting it off, and putting it off; and the next thing you know, the weeks turn into months and then the months into years. I haven't sent any Christmas cards, so at this time I'll offer my best wishes for the season to you. It is Christmas Eve and I am working tonight. I will also work New Year's eve and day. It really doesn't seem like Christmas here, and there isn't that much to do in the evening's; So I'm just as well off working. As I had mentioned, I had intended to send you some coins from Korea but I believe that I put them in with a package that I had sent to Mom and told her to give them to you. In any event, here are some in this box. The Korean coin with the 10 on it is 10 Won, and worth about $.0769. The large copper coin is Japanese ( 10 Yen) and about 2.8 Wait! I was wrong on the Korean coins.They are 10 Hwan which is the old money. When they Changed to Won, they made 10 Hwan equal to 1 Won. (Sounds confusing and tongue-tying, doesn't it?) So these 10 copper pieces with the 10 on them altogether are only worth 7 / The Viet Namese coins with the former president Diem picture on them are soon being taken out of circulation. ( Since he was overthrown and killed, they are going to replace the money also.) The small pieces marked 1 dong are 1 piastre; also called one Viet Namese dollar. 1 piastre is about .7 . I also recall sending some coins home in my hold baggage. I'll tell Mom to look for them and give them to you. Now that I look back on it, the year I spent in Korea wasn't bad. It was difficult at first having to work for an arrogant Jew boss, but after a few months I got out of his office and was on my own working directly for the army people ( U.S.) The country of Korea isn't too bad. As you can see on the coins they were in the year 4294 last year. It is surprising for a civilization to be that far along in years and still live under very primitive conditions. Going from Korea to Japan is like going from the dark ages into the future science fiction world. Those people are really progressing in Japan. I spent several days in Japan after I left Korea. Then I went to Honolulu for almost a month. Then I went back to Japan for Leave for 3 weeks and then down here to Viet Nam in October. They were building like crazy in Japan to get ready for the Olympics. Working 4 hours a day, 7 days a week on construction. You could drive into Tokyo early in the evening and return late in the night and find the road you had gone in on was now completely torn away. And downtown Tokyo is vey nice. Lots of modern building and just about any modern gadget you can think of. I really like their cooking also. Their Kobe Beef is excellent. In this beef, the cattle are pampered from the time they are born until they are slaughtered. They massage them, feed them beer, and let them live a lazy life. I would really like to work a year or two in Japan. The hot baths and big tubs are also great. Only it is almost impossible to stand the hot temperatures that they can. Westerners always have to add quite a lot of cold water before they can get in the tub. The Viet Namese people are much dirtier than the Japanese or Koreans. They don't have any hot baths here in Viet Nam. In fact, the majority of hotels ( built by the French) don't even have hot water in them. The Korean houses don't look like much from the outside, but inside they are spotlessly clean. The older Japanese buildings look nic efrom the outside and are also clean inside. I am here in the town of Nha Trang. It is a small town on the coast of the China Sea. I like it here. My hotel is less than a hundred yards from the beach. We are now in the winter season also, but it just got a little cooler from having winds the past few weeks. This afternoon as we came to work there were all sorts of people in swimming. Lobster is also plentiful here and I eat quite a bit of it. Most of the restaurant make a creamed corn soup with lots of chunks of crab meat in it. This is very good. Being out here in Nha Trang, we missed the coup d'etat. Things were quiet here and the only way we knew what was happening was to listen to the radio and talk to our co-workers of our radio system in Saigon. When they lifted martial law and the curfew in Saigon, it took them several days later to follow suit here. There seems to be a noticed improvement since the coup. The people in general seem much happier and there appears to be much more general activity going on. Well, I guess that's about it for now. I think I may have some Korean Won notes back in my room and I'll check and enclose them if I can find them. Say hello to Ira & family when you see them, and also to Aunt Marie if she is still there with you. 'Bye for now Your nephew PS The Viet Namese coins with 50 XU on them are piastres.