Search Results: "cas"

6 January 2016

Ingo Juergensmann: Letsencrypt: challenging challenges

On December 3rd 2015 the Letsencrypt project went to public beta and this is a good thing! Having more and more websites running on good and valid SSL certificates for their HTTPS is a good thing, especially because Letsencrypt takes care of renewing the certs every now and then. But there are still some issues with Letsencrypt. Some people criticize the Python client needing root priviledges and such. Others complain that Letsencrypt currently only supports webservers. Well, I think for a public beta this is what we could have expected from the start: the Letsencrypt project focussed on a reference implementation and there are already other implementations being available. But one thing seems to a problem within the design of how Letsencrypt works as it uses a challenge response method to verify that the requesting user is controlling the domain for which the certificate shall be issued. This might work well in simple deployments, but what about a little more complex setups like multiple virtual machines and different protocols involved. For example: you're using domain A for your communication like user@example.net for your mail, XMPP and SIP. Your mailserver runs on one virtual machine, whereas the webserver is running on a different virtual machine. The same for XMPP and SIP: a seperate VM as well. Usually the Letsencrypt approach would be that you configure your webserver (by configure /.well-known/acme-challenge/* location or use a standalone server on port 443) to handle the challenge response requests. This would give you a SSL cert for your webserver example.net. Of course you could copy this cert to your mail-, XMPP- and SIP-server, but then again you have to do this everytime the SSL cert gets renewed. Another challenge is of course that you are not only have one or two domains, but a bunch of domains. In my case I host approximately >60 domains. Basically the mail for all domains are handled by my mailserver running on its own virtual machine. The webserver is located on a different VM. For some domains I offer XMPP accounts on a third VM. What is the best way to solve this problem? Moving everything to just one virtual machine? Naaah! Writing some scripts to copy the certs as needed? Not very smart as well. Using a network share for the certs between all VMs? Hmmm... would that work? And what about TLSA entries of your DNSSEC setup? When a SSL cert is renewed than the fingerprint might need an update in your DNS zone - for several protocols like mail, XMPP, SIP and HTTPS. At least the Bash implementation of Letsencrypt offers a "hook" which is called after the SSL cert has been issued. What are you ways to deal with this kind of handling the ACME protocol challenges and multi-domain, multi-VM setup?
Kategorie:
Debian
Tags:
Debian
LetsEncrypt
Linux
Software

5 January 2016

Lior Kaplan: PHP 5 Support Timeline

With the new year starting the PHP project is being asked to decide about the PHP 5 support timeline. While Aligning PHP 5.6 support timeline with the release date of PHP 7.0 seems like common sense to keep the support schedule continuous, there s a big question whether to extend it further to an additional one year of security support till the end of 2018. This would make PHP 5.6, the last of the PHP 5 branch, to have 2 years of security support and de facto getting the same life span as PHP 7.0 would (ending support of both in Dec 2018). But beside of support issues, this also affects adoption rate of PHP 7.0, serving as a catalyst due to end of support for the PHP 5 branch. My concerns are that with the additional security support the project would need to deal with the many branches (5.6, 7.0, 7.1, 7.2 and then to be 7.3 branch). I think we should limit what we guarantee (meaning keeping only one year of security support till end of 2017), and encourage project members and the eco-system (e.g. Linux distributions) to maintain further security based on best effort. This is already the case for out of official support releases like the 5.3 and 5.4 branches (examples for backports done by Debian: 5.3 and 5.4). And of course, we also have companies that make their money out of long term support (e.g. RedHat). On the other hand, we should help the eco system in doing such extended support, and hosting backported fixes in the project s git repo instead of having each Linux disto do the same patch work on its own.
Filed under: PHP

Daniel Pocock: Promoting free software and free communications on popular social media networks

(You can Follow or Tweet about this blog on Twitter) Sites like Twitter and Facebook are not fundamentally free platforms, despite the fact they don't ask their users for money. Look at how Facebook's censors confused Denmark's mermaid statue with pornography or how quickly Twitter can make somebody's account disappear, frustrating public scrutiny of their tweets and potentially denying access to vital information in their "direct message" mailbox. Then there is the fact that users don't get access to the source code, users don't have a full copy of their own data and, potentially worst of all, if most people bothered to read the fine print of the privacy policy they would find it is actually a recipe for downright creepiness. Nonetheless, a significant number of people have accounts in these systems and are to some extent contactable there. Many marketing campaigns that have been successful today, whether they are crowdfunding, political activism or just finding a lost cat claim to have had great success because of Twitter or Facebook. Is this true? In reality, many users of those platforms follow hundreds of different friends and if they only check-in once a day, filtering algorithms show them only a small subset of what all their friends posted. Against these odds, just posting your great idea on Facebook doesn't mean that more than five people are actually going to see it. Those campaigns that have been successful have usually had something else going in their favour, perhaps it was a friend working in the media who gave their campaign a plug on his radio show or maybe they were lucky enough to be slashdotted. Maybe it was having the funds for a professional video production with models who pass off as something spontaneous. The use of Facebook or Twitter alone did not make such campaigns successful, it was just part of a bigger strategy where everything fell into place. Should free software projects, especially those revolving around free communications technology, use such platforms to promote themselves? It is not a simple question. In favour, you could argue that everything we promote through public mailing lists and websites is catalogued by Google anyway, so why not make it easier to access for those who are on Facebook or Twitter? On top of that, many developers don't even want to run their own mail server or web server any more, let alone a self-hosted social-media platform like pump.io. Even running a basic SIP proxy server for the large Debian and Fedora communities involved a lot of discussion about the approach to support it. The argument against using Facebook and Twitter is that you are shooting yourself in the foot, when you participate in those networks, you give them even more credibility and power (which you could quantify using Metcalfe's law). The Metcalfe value of their network, being quadratic rather than linear, shoots ahead of the Metcalfe value of your own solution, putting your alternative even further out of reach. On top of that, the operators of the closed platform are able to evaluate who is responding to your message and how they feel about it and use that intelligence to further undermine you. In some cases, there may be passive censorship, such as WhatsApp silently losing messages that link to rival Telegram. How do you feel about this choice? How and when should free software projects and their developers engage with mainstream social media technology? Please come and share your ideas on the Free-RTC mailing list or perhaps share and Tweet them.

4 January 2016

Clint Adams: Real world Cryptol

cryptol is now in testing. If you hurry, you can contribute to upstream implementations of CAST5 and Twofish before Christmas this Thursday.

Bernd Zeimetz: open-vm-tools updated

In January 2014 the open-vm-tools package was orphaned and I took the chance to take over the maintenance. Unfortunately the package is still not 100% in the shape I d like to see it, but I m getting closer. I have to say Thank You for a lot of good bug reports, especially for those use cases which are hard to test/reproduce for me (running Debian in a Windows-based VMware Workstation Player for example .). At conova communications GmbH, the company I work for, we are using the package on all of our Debian VMs, both for customer and internal use. It is essential for us to have properly working open-vm-tools - not only to be able to shutdown the VM from VMware vCenter, but also because tools like vSphere Data Protection and Veeam depend on it. Good thing is that I can work on and test the package at work and breakages are detected early and fast normally. Getting a good contact to the VMware upstream was easy and the developers there are helpful and reply pretty fast to their emails. Also as it seems there are finally real commits showing up in the open-vm-tools github repository again, not only huge single commits with a full release. It is not only nice to see that they are moving into the right direction again, but also this is really helpful in fixing (urgent) bugs before the next release of open-vm-tools - or to backport a fix to the versoin in stable/oldstable. Since a few days we have open-vm-tools 10.0.5-3227872 in If you are using VMware ESX 5.5 or newer, you should upgrade to the backports versions. Same if you use a recent VMware player version. Please note that since 10.0.0 the open-vm-dkms package is only necessary if you need the legacy vmxnet module. This is only the case if you are using very old VM hardware versions. vmxnet3 is shipped in the Debian kernel, so you don t need to compile extra modules to use it. The vmhgfs module was replaced by a fuse-based implementation. If you d like to help maintaining the package, please send bugs/patches via the Debian BTS or even better - send pull requests for pkg-open-vm-tools. The repository is mirrored to git.bzed.at in case you want to avoid github.

Lunar: Reproducible builds: week 36 in Stretch cycle

What happened in the reproducible builds effort between December 27th and January 2nd: Infrastructure dak now silently accepts and discards .buildinfo files (commit 1, 2), thanks to Niels Thykier and Ansgar Burchardt. This was later confirmed as working by Mattia Rizzolo. Packages fixed The following packages have become reproducible due to changes in their build dependencies: banshee-community-extensions, javamail, mono-debugger-libs, python-avro. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Untested changes: reproducible.debian.net The testing distribution (the upcoming stretch) is now tested on armhf. (h01ger) Four new armhf build nodes provided by Vagrant Cascandian were integrated in the infrastructer. This allowed for 9 new armhf builder jobs. (h01ger) The RPM-based build system, koji, is now in unstable and testing. (Marek Marczykowski-G recki, Ximin Luo). Package reviews 131 reviews have been removed, 71 added and 53 updated in the previous week. 58 new FTBFS reports were made by Chris Lamb and Chris West. New issues identified this week: nondeterminstic_ordering_in_gsettings_glib_enums_xml, nondeterminstic_output_in_warnings_generated_by_breathe, qt_translate_noop_nondeterminstic_ordering. Misc. Steven Chamberlain explained in length why reproducible cross-building across architectures mattered, and posted results of his tests comparing a stage1 debootstrapped chroot of linux-i386 once done from official Debian packages, the others cross-built from kfreebsd-amd64.

John Goerzen: Hiking a mountain with Ian Murdock

Would you like to hike a mountain? That question caught me by surprise. It was early in 2000, and I had flown to Tucson for a job interview. Ian Murdock was starting a new company, Progeny, and I was being interviewed for their first hire. Well, I thought, hiking will be fun. So we rode a bus or something to the top of the mountain and then hiked down. Our hike was full of well, everything. Ian talked about Tucson and the mountains, about his time as the Debian project leader, about his college days. I asked about the plants and such we were walking past. We talked about the plans for Progeny, my background, how I might fit in. It was part interview, part hike, part two geeks chatting. Ian had no HR telling him you can t go hiking down a mountain with a job candidate, as I m sure HR would have. And I am glad of it, because even 16 years later, that is still by far the best time I ever had at a job interview, despite the fact that it ruined the only pair of shoes I had brought along I had foolishly brought dress shoes for a, well, job interview. I guess it worked, too, because I was hired. Ian wanted to start up the company in Indianapolis, so over the next little while there was the busy work of moving myself and setting up an office. I remember those early days Ian and I went computer shopping at a local shop more than once to get the first workstations and servers for the company. Somehow he had found a deal on some office space in a high-rent office building. I still remember the puzzlement on the faces of accountants and lawyers dressed up in suits riding in the elevators with us in our shorts and sandals, or tie-die, next to them. Progeny s story was to be a complicated one. We set out to rock the world. We didn t. We didn t set out to make lasting friendships, but we often did. We set out to accomplish great things, and we did some of that, too. We experienced a full range of emotions there elation when we got hardware auto-detection working well or when our downloads looked very popular, despair when our funding didn t come through as we had hoped, being lost when our strategy had to change multiple times. And, as is the case everywhere, none of us were perfect. I still remember the excitement after we published our first release on the Internet. Our little server that could got pegged at 100Mb of outbound bandwidth (that was something for a small company in those days.) The moment must have meant something, because I still have the mrtg chart from that day on my computer, 15 years later. Progeny's Bandwidth Chart We made a good Linux distribution, an excellent Debian derivative, but commercial success did not flow from it. In the succeeding months, Ian and the company tried hard to find a strategy that would stick and make our big break. But that never happened. We had several rounds of layoffs when hoped-for funding never materialized. Ian eventually lost control of the company, and despite a few years of Itanium contract work after I left, closed for good. Looking back, Progeny was life compressed. During the good times, we had joy, sense of accomplishment, a sense of purpose at doing something well that was worth doing. I had what was my dream job back then: working on Debian as I loved to do, making the world a better place through Free Software, and getting paid to do it. And during the bad times, different people at Progeny experienced anger, cynicism, apathy, sorrow for the loss of our friends or plans, or simply a feeling to soldier on. All of the emotions, good or bad, were warranted in their own way. Bruce Byfield, one of my co-workers at Progeny, recently wrote a wonderful memoriam of Ian. He wrote, More than anything, he wanted to repeat his accomplishment with Debian, and, naturally he wondered if he could live up to his own expectations of himself. That, I think, was Ian s personal tragedy that he had succeeded early in life, and nothing else he did with his life could quite measure up to his expectations and memories. Ian was not the only one to have some guilt over Progeny. I, for years, wondered if I should have done more for the company, could have saved things by doing something more, or different. But I always came back to the conclusion I had at the time: that there was nothing I could do a terribly sad realization. In the years since, I watched Ubuntu take the mantle of easy-to-install Debian derivative. I saw them reprise some of the ideas we had, and even some of our mistakes. But by that time, Progeny was so thoroughly forgotten that I doubt they even realized they were doing it. I had long looked at our work at Progeny as a failure. Our main goal was never accomplished, our big product never sold many copies, our company eventually shuttered, our rock-the-world plan crumpled and forgotten. And by those traditional measurements, you could say it was a failure. But I have come to learn in the years since that success is a lot more that those things. Success is also about finding meaning and purpose through our work. As a programmer, success is nailing that algorithm that lets the application scale 10x more than before, or solving that difficult problem. As a manager, success is helping team members thrive, watching pieces come together on projects that no one person could ever do themselves. And as a person, success comes from learning from our experiences, and especially our mistakes. As J. Michael Straczynski wrote in a Babylon 5 episode, loosely paraphrased: Maybe this experience will be a good lesson. Too bad it was so painful, but there ain t no other kind. The thing about Progeny is this Ian built a group of people that wanted to change the world for the better. We gave it our all. And there s nothing wrong with that. Progeny did change the world. As us Progeny alumni have scattered around the country, we benefit from the lessons we learned there. And many of us were different , sort of out of place before Progeny, and there we found others that loved C compilers, bootloaders, and GPL licenses just as much as we did. We belonged, not just online but in life, and we went on to pull confidence and skill out of our experience at Progeny and use them in all sorts of ways over the years. And so did Ian. Who could have imagined the founder of Debian and Progeny would one day lead the cause of an old-guard Unix turning Open Source? I run ZFS on my Debian system today, and Ian is partly responsible for that and his time at Progeny is too. So I can remember Ian, and Progeny, as a success. And I leave you with a photo of my best memento from the time there: an original unopened boxed copy of Progeny Linux. IMG_6197_v1

3 January 2016

Lunar: Reproducible builds: week 35 in Stretch cycle

What happened in the reproducible builds effort between December 20th to December 26th: Toolchain fixes Mattia Rizzolo rebased our experimental versions of debhelper (twice!) and dpkg on top of the latest releases. Reiner Herrmann submited a patch for mozilla-devscripts to sort the file list in generated preferences.js files. To be able to lift the restriction that packages must be built in the same path, translation support for the __FILE__ C pre-processor macro would also be required. Joerg Sonnenberger submitted a patch back in 2010 that would still be useful today. Chris Lamb started work on providing a deterministic mode for debootstrap. Packages fixed The following packages have become reproducible due to changes in their build dependencies: bouncycastle, cairo-dock-plug-ins, darktable, gshare, libgpod, pafy, ruby-redis-namespace, ruby-rouge, sparkleshare. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet: reproducible.debian.net Statistics for package sets are now visible for the armhf architecture. (h01ger) The second build now has a longer timeout (18 hours) than the first build (12 hours). This should prevent wasting resources when a machine is loaded. (h01ger) Builds of Arch Linux packages are now done using a tmpfs. (h01ger) 200 GiB have been added to jenkins.debian.net (thanks to ProfitBricks!) to make room for new jobs. The current count is at 962 and growing! diffoscope development Aside from some minor bugs that have been fixed, a one-line change made huge memory (and time) savings as the output of transformation tool is now streamed line by line instead of loaded entirely in memory at once. disorderfs development Andrew Ayer released disorderfs version 0.4.2-1 on December 22th. It fixes a memory corruption error when processing command line arguments that could cause command line options to be ignored. Documentation update Many small improvements for the documentation on reproducible-builds.org sent by Georg Koppen were merged. Package reviews 666 (!) reviews have been removed, 189 added and 162 updated in the previous week. 151 new fail to build from source reports have been made by Chris West, Chris Lamb, Mattia Rizzolo, and Niko Tyni. New issues identified: unsorted_filelist_in_xul_ext_preferences, nondeterminstic_output_generated_by_moarvm. Misc. Steven Chamberlain drew our attention to one analysis of the Juniper ScreenOS Authentication Backdoor: Whilst this may have been added in source code, it was well-disguised in the disassembly and just 7 instructions long. I thought this was a good example of the current state-of-the-art, and why we'd like our binaries and eventually, installer and VM images reproducible IMHO. Joanna Rutkowska has mentioned possible ways for Qubes to become reproducible on their development mailing-list.

Russ Allbery: podlators 4.04

Now that I've fixed the major test suite problems with podlators, failure reports are now (mostly) useful. There's still the occasional failure to create a directory for temporary files, which I think is just a problem with the testing system, but Windows test failures revealed an actual test suite bug. This release also merges the changes made for merging podlators into blead Perl, which were relatively minor, and changes the package metadata to point to GitHub as the repository. That seems to be the growing convention in the Perl community and makes it easier for people to submit pull requests. I'm still maintaining my own public repository, but Git makes it easy to replicate things in multiple places. (Bug tracking will remain in the CPAN RT.) You can get the latest release from the podlators distribution page.

Iustin Pop: Orcas Island day trip, June 2015

I just finished going through my last set of pending-review pictures from 2015, so I'm starting 2016 with a post about the past. In June 2015 I travelled to Seattle/Kirkland area for work purposes, and took advantage of a weekend to plan some more outdoors stuff. After looking around on maps, I settled on the San Juan islands, so I started looking at hiking possibilities, and in the end Orcas island looked the best choice - all the others had much lower elevations. So, early in the morning, I started driving from Kirkland to Anacortes ferry terminal. The drive itself is quite nice: after getting past the more populated areas, passing Everett, the the view are very nice, especially in the early morning hours and with very few traffic. At Anacortes, there was already a small queue, fortunately I had a pre-ordered ticket, and there was not much to do until the ferry arrived except to look forward at the day, and hope that the weather will stay nice. On the ferry then, crossing the straits and enjoying the very nice views: Perfect blue Catching the morning wind The ferry stops at Orcas (is it a town or just the terminal), and the next stop is Eastsound town. I pre-planned here a stop to get a second mini-breakfast: however, I misjudged what the portion sizes are and got myself a maxi-cinnamon roll at Caffe Olga: Second breakfast :) At least I knew I wasn't going to be hungry for a while :) Driving on, briefly stopping at Cascade Lake (I also stopped on the way back, the view is nice), then reaching the parking at the Twin Lakes trail on the shore of Mountain Lake. Good think I arrived somewhat early the parking was quite full already. I also got a bit confused on which way the hike starts, since it's not well marked, but after that I started the hike. It's also possible to drive up to Mount Constitution, but that's just lame; hiking from the base it's quite easy, if you find how to start the hike. Anyway: Starting to climb Finished the steepest part At one point, one meets this particular sign: Which way now? Beware the Little Summit is not to be missed! After ~40 minutes of hiking, with some parts a tiny bit strenuous, the view is really breathtaking. It's definitely worth stopping by, as the view is (IMHO) nicer than the view from the top of Mt. Constitution: Wow! The reason I say this is better is because you look towards ocean, whereas later the view is back towards the continent. And looking towards the big ocean is just perfect! Plus, the many small island, fully covered with forest are also nice. Onwards then towards the peak of Mount Constitution. You cross the "ridge" of the island, and your view shifts to the other side. Which means you see back to the Mountain Lake where the hike starts: Loocking back towards the start Here the path is more exposed, not through tall forest like at the beginning: Watching the horizon Right before reaching the peak, you pass through an interesting forest: A different kind of forest And then you're finally reaching the peak. Compared to Switzerland, it's very much not impressive (730m), but nevertheless, being so close to the ocean results in some very nice views: Couldn't have asked for better weather You can go into the small tower, and read through the history of the location, including the personal life of Robert Moran (shipbuilder), who retired in 1905 to Orcas island to live what (his doctors said to be) his last months, and who instead ended living until 1943. Not bad! To be filled under "too much stress is bad, nature is good" heading, I think. After eating a small packed lunch, I started back. At the beginning the forest is similar to the one back at the beginning of the hike, but then, as you reach the level of lakes, it is slightly different. More tall (old?) trees, more moss and ferns: Afternoon sun in the forest I passed briefly by the Twin Lakes, which were interesting (lots of submerged trunks), and then finally on the Twin Lakes trail back to the start. The views of Mountain Lake from here are also nice, especially in the less harsh afternoon sun: Reached Mountain Lake How did those trees get there? And then the hike was over. I still had some time to spend before the ferry I had a ticket on was scheduled, so I drove down to Olga town, as I was curious what was at the end of "Olga Road". Not much, but again nice views, and this very picturesque pier: Nice pier in Olga And then it was back to the ferry, waiting in line, getting on the ferry, and crossing back: Goodbye Orcas! Overall, it was a day well spent, part different, part similar to last year's mostly road trip. Definitely recommended if you're in the area, and there are a couple of other hikes on Orcas Island, plus all the other islands which make up the San Juans. However, traffic on the way back was not that awesome :/ Small price though!

2 January 2016

Lunar: Reproducible builds: week 33 in Stretch cycle

What happened in the reproducible builds effort between December 6th and December 12th: Toolchain fixes Reiner Herrmann rebased our experimental version of doxygen on version 1.8.9.1-6. Chris Lamb submitted a patch to make the manpages generated by ruby-ronn reproducible by using the locale-agnostic %Y-%m-%d for the dates. Daniel Kahn Gillmor took another shot at the issue of source path captured in DWARF symbols. A patch has been sent for review by GCC upstream to add the ability to read an environment variable with -fdebug-prefix-map. Packages fixed The following 24 packages have become reproducible due to changes in their build dependencies: gkeyfile-sharp, gprbuild, graphmonkey, gthumb, haskell-yi-language, ion, jackson-databind, jackson-dataformat-smile, jackson-dataformat-xml, jnr-ffi, libcommons-net-java, libproxy, maven-shared-utils, monodevelop-database, mydumper, ndesk-dbus, nini, notify-sharp, pixz, protozero, python-rtslib-fb, slurm-llnl, taglib-sharp, tomboy-latex. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: These uploads might have fixed reproducibility issues but could not be tested yet: Patches submitted which have not made their way to the archive yet: reproducible.debian.net Files created with diffoscope now have diffoscope in their name instead debbindiff. (h01ger) Hostnames of first and second build node are now recorded and shown in the build history. (Mattia Rizzolo) Exchanges have started with F-Droid developers to better understand what would be required to test F-Droid applications. (h01ger) A first small set of Fedora 23 packages is now also being tested while development on a new framework for testing RPMs in general has begun. A new Jenkins job has been added to set up to mock, the build system used by Fedora. Another new job takes care of testing RPMs from Fedora 23 on x86_64. So far only 151 packages from the buildsys-build group are tested (currently all unreproducible), but the plan is to build all 17,000 source packages in Fedora 23 and rawhide. The page presenting the results should also soon be improved. (h01ger, Dhiru Kholia) For Arch Linux, all 2223 packages from the extra repository will also be tested from now on. Packages in extra" are tested every four weeks, while those from core every week. Statistics are now displayed alongside the results. (h01ger) jenkins.debian.net has been updated to jenkins-job-builder version 1.3.0. Many job configurations have been simplified and refactored using features of the new version. This was another milestone for the jenkins.debian.org migration. (Phil Hands, h01ger) diffoscope development Chris Lamb announced try.diffoscope.org: an online service that runs diffoscope on user provided files. Screenshot of try.diffoscope.org Improvements are welcome. The application is licensed under the AGPLv3. On diffoscope itself, most pending patches have now been merged. Expect a release soon! Most of the code implementing parallel processing has been polished. Sadly, unpacking archive is CPU-bound in most cases, so the current thread-only implementation does not offer much gain on big packages. More work is still require to also add concurrent processes. Documentation update Ximin Luo has started to write a specification for buildinfo files that could become a larger platform than the limited set of features that were thought so far for Debian .buildinfo. Package reviews 113 reviews have been removed, 111 added and 56 updated in the previous week. 42 new FTBFS bugs were opened by Chris Lamb and Niko Tyni. New issues identified this week: timestamps_in_documentation_generated_by_docbook_dbtimestamp, timestamps_in_sym_l_files_generated_by_malaga, timestamps_in_edj_files_generated_by_edje_cc. Misc. Chris Lamb presented reproducible builds at skroutz.gr.

Niels Thykier: dput change-all-of-debian.changes

Lucas Nussbaum recently did a blog post called Debian is still changing . I found it a very welcome continuation of his previous blog post on the same topic. I find the graphs very interesting and was very happy to learn that he included relative graphs this time. Now I can with relatively ease say that 69% of all Debian packages are using a dh-style build (source). We have another 15% using classic debhelper, which means that at least 84% of all packages uses debhelper directly. Assuming all CDBS based packages rely on the debhelper class , we are at 99%! The latter is certainly an assumption, although I suspect it is probably pretty accurate[1]. Now, it is very cute to have world dominance , but that is not my primary interest in these numbers. Instead, we can use these numbers to determine that: Such as automatic dbgsym packages, indexable build-id from dbg(sym) packages via Packages files[2], and replacing maintscripts with ldconfig triggers. All of these changes happen to be changes that could be trivially deployed with very little risk and very high efficiency[3]. Notably, none of them required a compat bump (or a new debhelper tool). Of course, I do not intend to say that every change can (or should) be deployed via debhelper and much less into an existing dh_cmd -tool. Notably, dh_strip is reaching its breaking point for content. And if we were to require a compat bump for your change, we can now at least see the adoption rate via lintian. :) Nevertheless, it is nice to know that (politics aside) there is some agility in the Debian build system! :) [1] I would very much love to see numbers to (dis)prove my assumption about CDBS + debhelper. In fact, an absolute number of packages not using debhelper (indirectly) in Debian would be very intriguing. [2] New fields by default end up the Packages file. See e.g. the Packages.xz file on the debug mirror or your apt-cache via: 
apt-cache show mscgen-dbgsym   grep ^Build-Ids
The latter assumes that you have the debug mirror in your sources list. [3] Efficiency here being features people rarely override/disable.
Filed under: Debhelper, Debian

1 January 2016

Russ Allbery: 2015 Book Reading in Review

2015 was another very busy year, but one of stabilization, rebuilding, and recovery. I got through the ramp-up period with my new job, found a better role inside the company for my personal talents and preferences, and ended the year on a professional high note. I also moved, to a place I like much better. It was a year for taking things as they come, focusing on priorities, letting other things slip, and being realistic about how much I can do. All of that, plus quite a lot of company, a business trip, and a few other unexpected distractions, meant less reading than I would have preferred. However, I did catch up completely on review writing, which is another happy sign of stabilization. Reviews came in spotty bursts, but they did come. The only explicit reading goal I'm making for 2016 is to read more than I did in 2015. I'm still working out the best priorities and schedule for me, and finding the best work/life balance points, so a predictable reading schedule will have to wait a while longer. The below statistics are confined to the books I reviewed in 2015, but thanks to significant catch-up work, I've only read one book that I have not yet reviewed (and I finished that one on December 31st). That book will be counted in 2016. Once again, the year saw two 10 out of 10 books, and once again, my favorite book of the year was written by Ann Leckie. The conclusion of the Imperial Radch trilogy, Ancillary Mercy, is as good or better than the start. The second book of the trilogy, Ancillary Sword, was also among my 2015 reviews and got 9 out of 10. I highly recommend the entire trilogy, beginning with Ancillary Justice (my book of the year in 2014), to anyone who hasn't read it. The second 10 out of 10 was non-fiction: Randall Munroe's What If? collection, featuring some material from the web site feature that accompanies xkcd and some original material. These are longer essays exploring interesting bits of science, math, and guesswork in the context of hypothetical questions that usually become surprisingly destructive. As the review says, try a few samples from the web site and see if this is your thing. I loved it. Despite my continuing low reading totals, this was a year full of fiction stand-outs. Becky Chambers's The Long Way to a Small, Angry Planet was the surprise of the year for me: a heart-warming, delightful story of chosen family. Jo Walton's My Real Children was less of a surprise because I already knew she is an excellent writer, but it was probably the best-written book I read all year. In turns sad, thoughtful, and determined, it's slice-of-life fiction so good that it overcame my normal dislike of that subject matter. Other fiction highlights are parts of series: the first two Steerswomen books by Rosemary Kirstein (The Steerswoman and The Outskirter's Secret), which dance between fantasy and scientific discovery, and Seanan McGuire's One Salt Sea, the best of all the October Daye books I've read. In non-fiction, the other book that stands out is Jenny Lawson's Let's Pretend This Never Happened. This combination of memoir and stand-up comedy in book form is one of the funniest things I've read, and it mixes that humor with self-awareness and generous openness. It's a book about being a little crazy and a lot anxious, finding ways to cope by laughing at yourself, and inviting the rest of the world to join in. Finally, Sydney Padua's The Thrilling Adventures of Lovelace and Babbage straddles the line between fiction and non-fiction, but certainly deserves a place in the year-end round-up. Full of great art, humor, steampunk, footnotes galore, and numerous forms of geekery, it's a collection I've been waiting for since Padua's very occasional comic got its moment of Internet fame. The full analysis includes some additional personal reading statistics, probably only of interest to me.

Russ Allbery: Review: One Salt Sea

Review: One Salt Sea, by Seanan McGuire
Series: October Daye #5
Publisher: DAW
Copyright: September 2011
ISBN: 1-101-54760-X
Format: Kindle
Pages: 368
This is the fifth book of the October Daye series. It's the best book of the series to date, at least in my opinion, but this is a series with a substantial cast and political complications. You could probably start here and work out things on the fly, since each story is largely self-contained, but much of the emotional resonance would be lost. As expected for this series, Toby's life doesn't have much time to stabilize after the events of Late Eclipses. This time, though, the initial threat is less personal and more political. The children of the rulers of the Duchy of Saltmist have been kidnapped, and war between Saltmist and the Kingdom of the Mists (Toby's home) is very near. All that stands in the way of horrible casualties, of fae that can scarce afford more death, is Toby's desperate detective work. Toby's confidence and belief in her own abilities has been growing steadily throughout this series. One of my favorite themes in fiction is someone who doesn't really believe in themselves but tries anyway, because they couldn't live with themselves if they didn't, and discovers they're more capable than they thought. In this series, I think that works best when Toby is less scared and more angry, which is very true in One Salt Sea. That's one reason why I liked this book better than either An Artificial Night or Late Eclipses. Another is that I love the politics and the depth of world-building and lore that McGuire brings to this series, and One Salt Sea is a showcase of both. We get the Undersea Duchy of Saltmist, which comes from both different traditions and different fae than the series so far. We get a plot that combines political maneuvering with a more traditional detective story than the last few books. We get more hidden traditions, past alliances and enmity that Toby didn't know about, and further development of Toby's own abilities. She has to tug hard on some threads, move between levels of the fae world, and draw on her network for uniquely fae forms of forensic analysis. I thoroughly enjoyed it, particularly the rocks. (You'll know what I mean when you get to that part.) But the best thing about this book, by far, is the Luidaeg. I've mentioned in reviews of previous books that the Luidaeg is my favorite character of the series. One Salt Sea just cements that. This is the deepest that the Luidaeg has been involved in a story: she gives Toby her mission at the start, substantial help along the way, and is deeply involved in the ending. We even learn some of her own background, and some of her own worries and pain. (And foreshadowing that there's more of this to come, which I'm eagerly looking forward to.) She's moved from a strange ally through a cautious friend to some combination of aunt and mother to Toby, while always staying her irascible and occasionally painful self. The bits with her in this book are the best so far, and there are a lot of them. It's a delightful reading experience. McGuire also uses this story to clear up a plot element that's been lingering since the first book of the series, but that never worked emotionally for me. (I'm talking about this indirectly since it's a significant spoiler.) It's a great resolution, one with heft and emotion and hard choices and pain, and largely makes up for the showing rather than telling that we've gotten in the previous books, while also (at least I suspect) graciously shuffling that plot out of the way in future books. It's a good bit of series work and personal growth for Toby, and I think frees the series to focus on the bits I enjoy the most. The ending of One Salt Sea hurts, but it's a good hurt, full of hard choices made well, bravery, and sacrifice. The best moments of this series have always had that, but I think this book is the most successful delivery of the whole package to date. This has always been a solid urban fantasy series, but I think it's getting better as it goes along. One major reason is that McGuire doesn't seem to be running out of world-building oomph. In a lot of urban fantasy, particularly the werewolf and vampire sort, I feel like the lore gets thinner, more rehashed, and more mundane as the series ages. But McGuire is still introducing new elements, showing existing elements from new directions and in new light, and has been slowly building major puzzles around Toby's life and abilities that intrigue me. And, even more rare, the eventual revelations usually live up to the preceding tension. It's certainly a recipe to keep me reading. Followed by Ashes of Honor. Rating: 9 out of 10

31 December 2015

Raphaël Hertzog: My Free Software Activities in December 2015

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I have been paid to work 21.25 hours on Debian LTS. During this time I worked on the following things: Distro Tracker I put a big focus on tracker.debian.org work this month. I completed the switch of the mail interface from packages.qa.debian.org to tracker.debian.org and I announced the change on debian-devel-announce. The changes resulted in a few problems that I quickly fixed (like #807073) and some other failures seen only by me and that were generated by weird spam messages (did you know that a subject can t have a newline character but that it can be encoded and folded over multiple lines?). Related to that I fixed some services so that they send their mails to tracker.debian.org directly instead of relying on the old emails (they get forwarded for now but it would be nice to be able to get rid of that forward). I updated (with the help of Lucas Nussbaum) the service that forwards the Launchpad bugs to the tracker, I sent a patch to update the @packages.debian.org aliases (not yet applied), I updated the configuration of all git commit notice scripts in the Alioth collab-maint and python-modules project (many remain to be done). I asked Ubuntu s Merge-O-Matic to use the new emails as well (see LP 1525497). DAK and the Debian BTS still have to be updated, as of yet nobody reacted to my announce last but not least I updated many wiki pages which duplicated the instructions to setup the commit notice sent to the PTS. While on a good track I opted to tackle the long-standing RC bug that was plaguing tracker.debian.org (#789183), so I updated the codebase to rely on Twitter s bootstrap v4 instead of v2. I had to switch to something else for the icons since glyphicons is no longer provided as part of bootstrap and the actual license for the standalone version was not suitable for use. I opted for Github s Octicons. I made numerous little improvements while doing that (closing some bugs in the process) and I believe that the result is more pleasant to use. I also did a lot of bug triage and fixed a few small issues like the incomplete architecture list (#793547), or fixing a page used only by people with javascript disabled that was not working. Or the invalid links for packages still using CVS (ugh, see #561228). Misc packaging Django. After having added DEP-8 tests (as part of my LTS work, see above), I discovered that the current version in unstable did not pass its test suite so I filed the issue upstream (ticket 26016) and added the corresponding patch. And I encouraged others to update python-bcrypt in Debian to a newer version that would have worked with Django 1.9 (see #803096). I also fixed another small issue in Django (see ticket 26017 with my pull request that got accepted). I asked the release managers to consider accepting the latest 1.7.x version in jessie (see #807654) but I have gotten zero answer so far. And I m not the only one waiting an answer. It s a bit of a sad situation we still have a few weeks until the next point release but for once I do it in advance and I would love to have timely feedback. Last but not least, I started the maintaining the current LTS release (1.8.x) in jessie-backports. Tryton. I upgraded to Tryton 3.8 and discovered an issue that I filed in #806781. I sponsored 5 new tryton modules for Matthias Behrle (who is DM) as well as one security upload (for CVE-2015-0861). Debian Handbook. I uploaded a new version to Debian Unstable and requested (to the release managers) the permission to upload a backport of it to jessie so that jessie has a version of the package that documents jessie and not wheezy contrary to my other Django request, this one should be non-controversial but I also have had zero answer so far, see #807515. Misc. I filed #808583 when sbuild stopped working with Perl 5.22. I handled #807860 on publican, I found the corresponding upstream ticket and discovered a work around with the help of upstream (see here). Kali related work I reported a bug to #debian-apt about apt miscalculating download size (ending up with 18 EB!) which resulted in a fix here in version 1.1.4. Installing a meta-package that needed more than 2GB was no longer possible without this fix and we have a kali-linux-all metapackage in that situation that gets regularly installed in a Jenkins test. I added captcha support to Distro Tracker and enabled this feature on pkg.kali.org. I filed #808863 against uhd-host because it was not possible to install the package in a systemd-nspawn s managed chroot where /proc is read-only. And we started using this to test dist-upgrade from one version of Kali to the next Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

30 December 2015

Bits from Debian: Debian lamenta el dec s de l'Ian Murdock

Ian Murdock Amb el cor compungit, Debian lamenta la p rdua d'Ian Murdock, ferm partidari del programari lliure i de codi obert, pare, fill i l'"ian" a Debian. L'Ian va comen ar el projecte Debian l'agost de 1993, i va publicar les primeres versions m s tard el mateix any. Debian es convertiria en el sistema operatiu universal pel m n, que funciona en tot tipus de dispositius, des de sistemes encastats fins a l'estaci especial. L'Ian es va centrar en crear una distribuci i una cultura de comunitat que fessin el correcte, ticament o t cnica. Publicar quan estigui a punt i la ferma posici sobre la llibertat de programari s n les regles d'or en el m n del programari lliure i de codi obert. La seva devoci per fer el correcte va guiar la seva feina, tant a Debian com els seg ents anys, sempre treball per assolir el millor futur possible. El somni de l'Ian est viu, la comunitat de Debian continua incre blement activa, amb milers de desenvolupadors que treballen innumerables hores per proporcionar un sistema operatiu segur i fiable. Els pensaments de la comunitat de Debian s n amb la fam lia de l'Ian en aquests moments tant durs. La seva fam lia ens ha demanat privacitat en aquests moments dif cils i volem respectar-ho. Des de Debian i des de la comunitat de Linux en general podem enviar mostres de condol a in-memoriam-ian@debian.org; seran guardades i arxivades.

Steve Kemp: I joined the internet of things.

In my old flat I had a couple of simple radio-controlled switches, which allowed me to toggle power to a pair of standing lamps - one at each side of the bed. This was very lazy, but also really handy and I've always been curious about automation.. When it comes to automation there seems to be three main flavours:
X10
The original standard, with stuff produced by many vendors and good Linux support. X10 supports two ways of sending/receiving commands - over the electrical wiring, and over RF.
Z-Wave
This is the newcomer, which despite that seems to be well-supported and extensible. It allows "measurements" to be sent/received in addition to the broadcast of events like "switch on", and "switch off".
Other systems - often lighting-centric
There are toy-things like the previously noted power-controlling things, there are also stand-alone devices from people like Philips with their philips hue system, but given how Philips recently crippled their devices to disable third-party bulbs I've no desire to use them. One company caught my eye though, Osram make a smart lightbulb and mini-hub to work with it.
So I bought one of the osram lightify systems, consisting of a magic box and a pair of lightbulbs. The box connects to your wifi, and gets an IP address. The IP address is then used by the application on your mobile phone (i.e. the magic box does the magic, not the bulbs). The phone application can be used to trigger "on", "off", "dim", "brighter", and the various colour-changing commands, as you would expect. You absolutely must use the phone-based application to do the setup, but after that the whole point was that I could automate things. I wanted to be able to setup my desktop computer to schedule events, and started hacking. I've written a simple Perl module to let me discover bulbs, and turn them off and on. No doubt it'll be on CPAN in the near future, once I can pick a suitable name for it: 
$ ol --bridge=192.168.10.136 --list
hall       MAC:8418260000d9c70c RGBW:255,255,255,255 STATE:On
kitchen    MAC:8418260000cb433b RGBW:255,255,255,255 STATE:On
$ ol --bridge=192.168.10.136 --off=kitchen
$ ol --bridge=192.168.10.136 --list
hall       MAC:8418260000d9c70c RGBW:255,255,255,255 STATE:On
kitchen    MAC:8418260000cb433b RGBW:255,255,255,255 STATE:Off
The only niggle was the fiddly pairing, and the lack of any decent documentation. The code I wrote was loosely based on the python project python-lightify written by Mikael Magnusson. Also worth noting that the bridge/magic-box only exposes a single port so you can find the device on your VLAN by nmapping for port 4000: 
$ nmap -v 192.168.10.0/24 -p 4000
The device doesn't seem to allow any network setup at all - it only uses DHCP. So you might want to make sure it gets assigned a stable IP. Anyway I'm going to bed. When I do so I'll turn the lights off with my mobile phone. Neat. In the future I will look at more complex automation, and I think Z-wave is the way I'll go. Right now I'm in a rented flat so replacing wall-switches, etc, is something I can't do. But the systems I've looked at seem neat, and this current setup will keep me amused for several months!

Julian Andres Klode: APT 1.1.8 to 1.1.10 going faster

Not only do I keep incrementing version numbers faster than ever before, APT also keeps getting faster. But not only that, it also has some bugs fixed and the cache is now checked with a hash when opening. Important fix for 1.1.6 regression Since APT 1.1.6, APT uses the configured xz compression level. Unfortunately, the default was set to 9, which requires 674 MiB of RAM, compared to the 94 MiB required at level 6. This caused the test suite to fail on the Ubuntu autopkgtest servers, but I thought it was just some temporary hickup on their part, and so did not look into it for the 1.1.7, 1.1.8, and 1.1.9 releases. When the Ubuntu servers finally failed with 1.1.9 again (they only started building again on Monday it seems), I noticed something was wrong. Enter git bisect. I created a script that compiles the APT source code and runs a test with ulimit for virtual and resident memory set to 512 (that worked in 1.1.5), and let it ran, and thus found out the reason mentioned above. The solution: APT now defaults to level 6. New Features APT 1.1.8 introduces /usr/lib/apt/apt-helper cat-file which can be used to read files compressed by any compressor understood by APT. It is used in the recent apt-file experimental release, and serves to prepare us for a future in which files on the disk might be compressed with a different compressor (such as LZ4 for Contents files, this will improve rred speed on them by factor 7). David added a feature that enables servers to advertise that they do not want APT to download and use some Architecture: all contents when they include all in their list of architectures. This is to allow archives to drop Architecture: all packages from the architecture-specific content files, to avoid redundant data and (thus) improve the performance of apt-file. Buffered writes APT 1.1.9 introduces buffered writing for rred, reducing the runtime by about 50% on a slowish SSD, and maybe more on HDDs. The 1.1.9 release is a bit buggy and might mess up things when a write syscall is interrupted, this is fixed in 1.1.10. Cache generation improvements APT 1.1.9 and APT 1.1.10 improve the cache generation algorithms in several ways: Switching a lookup table from std::map to std::unordered_map, providing an inline isspace_ascii() function, and inlining the tolower_ascii() function which are tiny functions that are called a lot. APT 1.1.10 also switches the cache s hash function to the DJB hash function and increases the default hash table sizes to the smallest prime larger than 15000, namely 15013. This reduces the average bucket size from 6.5 to 4.5. We might increase this further in the future. Checksum for the cache, but no more syncs Prior to APT 1.1.10 writing the cache was a multi-part process:
  1. Write the the cache to a temporary file with the dirty bit set to true
  2. Call fsync() to sync the cache
  3. Write a new header with the dirty bit set to false
  4. Call fsync() to sync the new header
  5. (Rename the temporary file to the target name)
The last step was obviously not needed, as we could easily live with an intact cache that has its dirty field set to false, as we can just rebuild it. But what matters more is step 2. Synchronizing the entire 40 or 50 MB takes some time. On my HDD system, it consumed 56% of the entire cache generation time, and on my SSD system, it consumed 25% of the time. APT 1.1.10 does not sync the cache at all. It now embeds a hashsum (adler32 for performance reasons) in the cache. This helps ensure that no matter what parts of the cache are written in case of some failure somewhere, we can still detect a failure with reasonable confidence (and even more errors than before). This means that cache generation is now much faster for a lot of people. On the bad side, commands like apt-cache show that previously took maybe 10 ms to execute can now take about 80 ms. Please report back on your performance experience with 1.1.10 release, I m very interested to see if that works reasonably for other people. And if you have any other idea how to solve the issue, I d be interested to hear them (all data needs to be written before the header with dirty=0 is written, but we don t want to sync the data). Future work We seem to have a lot of temporary (?) std::string objects during the cache generation, accounting for about 10% of the run time. I m thinking of introducing a string_view class similar to the one proposed for C++17 and make use of that. I also thought about calling posix_fadvise() before starting to parse files, but the cache generation process does not seem to spend a lot of its time in system calls (even with all caches dropped before the run), so I don t think this will improve things. If anyone has some other suggestions or patches for performance stuff, let me know.
Filed under: Debian, Ubuntu

29 December 2015

Ben Hutchings: Comparing dracut and initramfs-tools

initramfs-tools version 0.121~rc2 and dracut version 044+3-2 have moved most of what was in their main binary packages into a 'core' package (dracut-core, initramfs-tools-core), leaving only automation hooks in the main package. The core packages can be coinstalled without conflicts and without creating any initramfs images at installation time. If you're been wondering how they compare but were afraid to break your system by overwriting your current working initramfs, now is the time to try the other option. Testing dracut, for initramfs-tools users
  1. Install dracut-core from unstable.
  2. Make an alternate initramfs image by running:
    dracut /boot/initrd.img-$(uname -r).dracut
  3. Add an entry to the boot loader configuration that uses your current kernel and the alternate initramfs image. (In GRUB, you can do this interactively at boot time.)
  4. Reboot through the new boot entry.
  5. In case of problems, check the dracut(8) and dracut.conf(5) manual pages.
By default, dracut creates an initramfs suitable for the current hardware, which will not boot if you move the disk to a system that requires different drivers. This is controlled by the hostonly configuration variable or --hostonly command-line option. Testing initramfs-tools, for dracut users
  1. Install initramfs-tools-core from experimental.
  2. Make an alternate initramfs image by running:
    mkinitramfs -o /boot/initrd.img-$(uname -r).i-t
  3. Add an entry to the boot loader configuration that uses your current kernel and the alternate initramfs image. (In GRUB, you can do this interactively at boot time.)
  4. Reboot through the new boot entry.
  5. In case of problems, check the mkinitramfs(8) and initramfs.conf(5) manual pages.
By default, initramfs-tools creates an initramfs suitable for most hardware, which will still boot if you move the disk to a system that requires different drivers. This is controlled by the MODULES configuration variable.

28 December 2015

Hideki Yamane: How about "grooming" outdated packages?

Lucas's good article: DEBIAN PACKAGES WITH /OUTDATED/ PACKAGING STYLE

Okay, then what's next? Action.

How about do "grooming" for those packages with contributors? Well experienced DDs become mentors and help to make those packages modern and put it to repo.


Packages become simple and modern, contributors can get packaging experience.
Isn't it a good idea? (Meow!)

Next.

Previous.