Search Results: "will"

1 June 2014

Antonio Terceiro: An introduction to the Debian Continuous Integration project

Debian is a big system. At the time of writing, looking at my local package list caches tells me that the unstable suite contains 21306 source packages, and 42867 binary packages on amd64. Between these 42867 binary packages, there is an unthinkable number of inter-package dependencies. For example the dependency graph of the ruby packages contains other 20-something packages. A new version of any of these packages can potentially break some functionality in the ruby package. And that dependency graph is very small. Looking at the dependency graph for, say, the rails package will make your eyes bleed. I tried it here, and GraphViz needed a PNG image with 7653 10003 pixels to draw it. It ain t pretty. Installing rails on a clean Debian system will pull in another 109 packages as part of the dependency chain. Again, as new versions of those packages are uploaded the archive, there is a probability that a backwards-incompatible change, or even a bug fix which was being worked around, might make some funcionality in rails stop working. Even if that probability is low for each package in the dependency chain, with enough packages the probability of any of them causing problems for rails is quite high. And still the rails dependency chain is not that big. libreoffice will pull in another 264 packages. gnome will pull in 1311 dependencies, and kde-full 1320 (!). With a system this big, problems will arrive, and that s a fact of life. As developers, what we can do is try to spot these problems as early as possible, and fixing them in time to make a solid release with the high quality Debian is known for. While automated testing is not the proverbial Silver Bullet of Software Engineering, it is an effective way of finding regressions. Back in 2006, Ian Jackson started the development of autopkgtest as a tool to test Debian packages in their installed form (as opposed to testing packages using their source tree). In 2011, the autopkgtest test suite format was proposed as a standard for the Debian project, in what we now know as the DEP-8 specification. Since then, some maintainers such as myself started experimenting with DEP-8 tests in their packages. There was an expectation in the air that someday, someone would run those tests for the entire archive, and that would be a precious source of QA information. Durign the holiday break last year, I decided to give it a shot. I initially called the codebase dep8. Later I renamed it to debci, since it could potentially also run other other types of test suites in the future. Since early January, ci.debian.net run an instance of debci for the Debian Project. The Debian continuous Integration will trigger tests at most 4 times a day, 3 hours after each dinstall run. It will update a local APT cache and look for packages that declare a DEP-8 test suite. Each package with a test suite will then have its test suite executed if there was any change in its dependency chain since the last test run. Existing test results are published at ci.debian.net every hour, and at the end of each batch a global status is updated. Maintainers can subscribe to a per package Atom feed to keep up with their package test results. People interested in the overall status can subscribe to a global Atom feed of events. Since the introduction of Debian CI in mid-January 2014, we have seen an amazing increase in the number of packages with test suites. We had little less than 200 packages with test suites back then, against around 350 now (early June 2014). The ratio of packages passing passing their test suite has also improved a lot, going from less than 50% to more than 75%. There is documentation available, including a FAQ for package maintainers with further information about how the system works, how to declare test suites in their packages and how to reproduce test runs locally. Also available is development information about debci itself, to those inclined to help improve the system. This is just the beginning. debci is under a good rate of development, and you can expect to see a constant flux of improvements. In special, I would like to mention a few people who are giving amazing contributions to the project:

31 May 2014

Floris Stoica-Marcu: GSoC2014 Muon and sudo [Week2]

Hello, This has been a terrific week and a real learning experience for me. I finished packaging both Muon and QApt for Debian; my first two real debian packages, with very different packaging styles! Sune (svuorela), Maximiliano (maxy) and others helped me allot to understand the tools and I thank them very much for their support. While the first part of this second week was all about trying to make a solid package, the second part was me figuring out Policykit and KAuth in order for us to let go (as much as possible) of Kdesudo. I have managed to do a complete turnaround from kdesudo through policykit and now I am working on making an add-repo helper in C++/Qt using KAuth to keep it in line with QApt, in order to have a more uniform codebase. Big thanks to Aleix (apol) that guided me through the different uses of policykit and Kauth. To keep it short, policykit will not be used directly except QApt, in every other case we ll use KAuth. I m very close to finishing the KAuth add-repo helper and that will add improved functionality, as it extends well in what I will be doing next: renouncing specific backend support; The helper will manage how backends add/remove repos, meaning that if you use PackageKit you will no longer be tied to Apt if you want to use Muon! Here is a screenshot of the configure-software button in muon-discovery (finally) working on Debian and launching software-properties-kde:
Muon Software properties And below are some screens of Muon in the wild, running on Debian:
Auth required through policykit

Auth required through policykit

Adding Sources

Adding Sources

I hope you enjoyed this update and stay tuned for the debian package which is currently under review. Floris.

30 May 2014

Julien Danjou: OpenStack Design Summit Juno, from a Ceilometer point of view

Last week was the OpenStack Design Summit in Atlanta, GA where we, developers, discussed and designed the new OpenStack release (Juno) coming up. I've been there mainly to discuss Ceilometer upcoming developments. The summit has been great. It was my third OpenStack design summit, and the first one not being a PTL, meaning it was a largely more relaxed summit for me! On Monday, we started by a 2.5 hours meeting with Ceilometer core developers and contributors about the Gnocchi experimental project that I've started a few weeks ago. It was a great and productive afternoon, and allowed me to introduce and cover this topic extensively, something that would not have been possible in the allocated session we had later in the week. Ceilometer had his design sessions running mainly during Wednesday. We noted a lot of things and commented during the sessions in our Etherpads instances. Here is a short summary of the sessions I've attended. Scaling the central agent I was in charge of the first session, and introduced the work that was done so far in the scaling of the central agent. Six months ago, during the Havana summit, I proposed to scale the central agent by distributing the tasks among several node, using a library to handle the group membership aspect of it. That led to the creation of the tooz library that we worked on at eNovance during the last 6 months. Now that we have this foundation available, Cyril Roelandt started to replace the Ceilometer alarming job repartition code by Taskflow and Tooz. Starting with the central agent is simpler and will be a first proof of concept to be used by the central agent then. We plan to get this merged for Juno. For the central agent, the same work needs to be done, but since it's a bit more complicated, it will be done after the alarming evaluators are converted. Test strategy The next session discussed the test strategy and how we could improve Ceilometer unit and functional testing. There is a lot in this area to be done, and this is going to be one of the main focus of the team in the upcoming weeks. Having Tempest tests run was a goal for Havana, and even if we made a lot of progress, we're still no there yet. Complex queries and per-user/project data collection This session, led by Ildik V ncsa, was about adding finer-grained configuration into the pipeline configuration to allow per-user and per-project data retrieval. This was not really controversial, though how to implement this exactly is still to be discussed, but the idea was well received. The other part of the session was about adding more in the complex queries feature provided by the v2 API. Rethinking Ceilometer as a Time-Series-as-a-Service This was my main session, the reason we met on Monday for a few hours, and one of the most promising session I hope of the week. It appears that the way Ceilometer designed its API and storage backends a long time ago is now a problem to scale the data storage. Also, the events API we introduced in the last release partially overlaps some of the functionality provided by the samples API that causes us scaling troubles. Therefore, I've started to rethink the Ceilometer API by building it as a time series read/write service, letting the audit part of our previous sample API to the event subsystem. After a few researches and experiments, I've designed a new project called Gnocchi, which provides exactly that functionality in a hopefully scalable way. Gnocchi is split in two parts: a time series API and its driver, and a resource indexing API with its own driver. Having two distinct driver sets allows it to use different technologies to store each data type in the best storage engine possible. The canonical driver for time series handling is based on Pandas and Swift. The canonical resource indexer driver is based on SQLAlchemy. The idea and project was well received and looked pretty exciting to most people. Our hope is to design a version 3 of the Ceilometer API around Gnocchi at some point during the Juno cycle, and have it ready as some sort of preview for the final release. Revisiting the Ceilometer data model This session led by Alexei Kornienko, kind of echoed the previous session, as it clearly also tried to address the Ceilometer scalability issue, but in a different way. Anyway, the SQL driver limitations have been discussed and Mehdi Abaakouk implemented some of the suggestions during the week, so we should very soon see more performances in Ceilometer with the current default storage driver. Ceilometer devops session We organized this session to get feedbacks from the devops community about deploying Ceilometer. It was very interesting, and the list of things we could improve is long, and I think will help us to drive our future efforts. SNMP inspectors This session, led by Lianhao Lu, discussed various details of the future of SNMP support in Ceilometer. Alarm and logs improvements This mixed session, led by Nejc Saje and Gordon Chung, was about possible improvements on the alarm evaluation system provided by Ceilometer, and making logging in Ceilometer more effective. Both half-sessions were interesting and led to several ideas on how to improve both systems. Conclusion Considering the current QA problems with Ceilometer, Eoghan Glynn, the new Project Technical Leader for Ceilometer, clearly indicated that this will be the main focus of the release cycle. Personally, I will be focused on working on Gnocchi, and will likely be joined by others in the next weeks. Our idea is to develop a complete solution with a high velocity in the next weeks, and then works on its integration with Ceilometer itself.

29 May 2014

Enrico Zini: wheezy-for-industrial-development

Wheezy for industrial software development I'm helping with setting up a wheezy-based toolchain for industrial automation. The basic requirements are: live-build, C++11, Qt 5.3, and a frozen internal wheezy mirror. debmirror A good part of a day's work was lost because of #749734 and possibly #628779. Mirror rebuild is still ongoing, and fingers crossed. This is Italy, and you can't simply download 21Gb of debs just to see how it goes. C++11 Stable toolchains for C++11 now exist and have gained fast adoption. It makes sense, since given what is in C++11 it is unthinkable to start a new C++ project with the old standard nowadays. C++11 is supported by g++ 4.8+ or clang 3.3+. None of them is available on wheezy or wheezy-backports. Backports exist of g++ 4.8 only for Ubuntu 12.04, but they are uninstallable on wheezy due at least to a different libc6. I tried rebuilding g++4.8 on wheezy but quickly gave up. clang 3.3 has a build dependency on g++ 4.8. LOL. However, LLVM provides an APT repository with their most recent compiler, and it works, too. C++11 problem solved! Qt 5.3 Qt 5.3 is needed because of the range of platforms it can target. There is no wheezy backport that I can find. I cannot simply get it from Qt's Download page and install it, since we need it packaged, to build live ISOs with it. I'm attempting to backport the packages from experimental to wheezy. Here are its build dependencies: libxcb-1.10 (needed by qt5) Building this is reasonably straightforward. libxkbcommon 0.4.0 (needed by qt5) The version from jessie builds fine on wheezy, provided you remove --fail-missing from the dh_install invocation. libicu 52.1 (needed by harfbuzz) The jessie packages build on wheezy, provided that mentions of clang are deleted from source/configure.ac, since it fails to build with clang 3.5 (the one currently available for wheezy on llvm.org). libharfbuzz-dev Backporting this is a bloodbath: the Debian packages from jessie depend on a forest of gobject hipsterisms of doom, all unavailable on wheezy. I gave up. qt 5.3 qtbase-opensource-src-5.3.0+dfsg can be made to build with an embedded version of harfbuzz, with just this change:
diff -Naur a/debian/control a/debian/control
--- a/debian/control    2014-05-20 18:48:27.000000000 +0200
+++ b/debian/control    2014-05-29 17:45:31.037215786 +0200
@@ -28,7 +28,6 @@
                libgstreamer-plugins-base0.10-dev,
                libgstreamer0.10-dev,
                libgtk2.0-dev,
-               libharfbuzz-dev,
                libicu-dev,
                libjpeg-dev,
                libmysqlclient-dev,
diff -Naur a/debian/rules b/debian/rules
--- a/debian/rules  2014-05-18 01:56:37.000000000 +0200
+++ b/debian/rules  2014-05-29 17:45:25.738634371 +0200
@@ -108,7 +108,6 @@
                -plugin-sql-tds \
                -system-sqlite \
                -platform $(platform_arg) \
-               -system-harfbuzz \
                -system-zlib \
                -system-libpng \
                -system-libjpeg \
(thanks Lisandro Dami n Nicanor P rez Meyer for helping me there!) There are probably going to be further steps in the Qt5 toolchain. Actually, let's try prebuilt binaries The next day with a fresh mind we realised that it is preferable to reduce our tampering with the original wheezy to a minimum. Our current plan is to use wheezy's original Qt and Qt-using packages, and use Qt's prebuilt binaries in /opt for all our custom software. We run Qt's installer, tarred the result, and wrapped it in a Debian package like this:
$ cat debian/rules
#!/usr/bin/make -f
QT_VERSION = 5.3
%:
    dh $@
override_dh_auto_build:
    dh_auto_build
    sed -re 's/@QT_VERSION@/$(QT_VERSION)/g' debian-rules.inc.in > debian-rules.inc
override_dh_auto_install:
    dh_auto_install
    # Download and untar the prebuild Qt5 binaries
    install -d -o root -g root -m 0755 debian/our-qt5-sdk/opt/Qt
    curl http://localserver/Qt$(QT_VERSION).tar.xz   xz -d   tar -C debian/our-qt5-sdk/opt -xf -
    # Move the runtime part to our-qt5
    install -d -o root -g root -m 0755 debian/our-qt5/opt/Qt
    mv debian/our-qt5-sdk/opt/Qt/$(QT_VERSION) debian/our-qt5/opt/Qt/
    # Makes dpkg-shlibdeps work on packages built with Qt from /opt
    # Hack. Don't try this at home. Don't ever do this unless you
    # know what you are doing. This voids your warranty. If you
    # know what you are doing, you won't do this.
    find debian/our-qt5/opt/Qt/$(QT_VERSION)/gcc_64/lib -maxdepth 1 -type f -name "lib*.so*" \
          sed -re 's,^.+/(lib[^.]+)\.so.+$$,\1 5 our-qt5 (>= $(QT_VERSION)),' > debian/our-qt5.shlibs
$ cat debian-rules.inc.in
export PATH := /opt/Qt/@QT_VERSION@/gcc_64/bin:$(PATH)
export QMAKESPEC=/opt/Qt/@QT_VERSION@/gcc_64/mkspecs/linux-clang/
To build one of our packages using Qt5.3 and clang, we just add this to its debian/rules:
include /usr/share/our-qt5/debian-rules.inc
Wrap up We got the dependencies sorted. Hopefully the mirror will rebuild itself tonight and tomorrow we can resume working on our custom live system.

Russell Coker: Links May 2014

Charmian Gooch gave an interesting TED talk about her efforts to fight organised crime and corruption by prohibiting anonymous companies [1]. The idea of a company is to protect the owner from unlimited liability not to protect them from law enforcement. Dr Nerdlove has an insightful article about sexual harassment in geek culture [2]. Rebecca Rose wrote an insightful article for Jezebel about the worse bullying advice ever, her summary is that it should be called Ways We Can Get You Goddamn Kids to Act So We Never Have to Deal With Your Problems Ever [3]. In the wake of the Heardbleed bug Imperial Violet wrote an informative article explaining why revocation checks isn t the solution [4]. Martin Lukacs and Shiri Pasternak wrote an insightful article about the Canadian government s attempts to stop Canadian aboriginies from exercising their legal rights [5]. I bet that the Australian government is doing the same things. Nelson Groom wrote an interesting interview with Norrie May Welby, the first recognised agender person in Australia [6]. Marriage equality is bound to happen soon in Australia, now that the government officially recognises non-binary gender people it can t refuse them the right to get married and therefore the straight/gay marriage distinction isn t relevant. BDA Technology has an interesting article on choosing fonts to make text more readable for dyslexic people [7]. Eamon Waterford wrote an informative article for the ABC about how early-intervention social policies save significant amounts of tax money [8]. Conservatives claim to want to save money and try to cut such programs which costs everyone in the long term. Greta Christina wrote an insightful article for Salon about why religious people want atheists to lie and pretend to believe [9]. Chris Mooney wrote an informative URL about a machine to test whether someone is liberal or conservative [10]. Paul Rosenberg wrote an informative article on the link between conservatism, evil, and psychopathy [11]. The next logical step is to treat conservatism as a mental health problem. Sociological Images has an interesting article by Jay Livingston about the hypocrisy of conservative tax policies [12]. Scientific American reprinted an article from The Conversation by Elaine McKewon about the climate deniers who intimidated a journal into retracting an article about their belief in conspiracy theories [13]. It seems obvious that the climate deniers are the ones who conspire. Ben Caldecott wrote an interesting article for the ABC about the fossil fuel divestment campaign [14]. It seems that this is getting some success already, as renewable energy will soon be cheaper than coal power this could kill off coal. Mark Taylor wrote an interesting blog post titled Observations of an Internet Middleman about the operations of Level3 and Internet peering [15]. He explains how monopoly Telcos throttle their customers Internet access. Matt Savoy wrote an informative and disturbing article about the fact that US cops are twice as likely to beat their wives as the general population [16]. Apparently the police hierarchy aren t interested in prosecuting such crimes. Paul Rosenberg wrote an insightful article about mythos vs logos and the conservative approach to relity [17]. One interesting point he makes is that white men (and members of privileged groups generally) fear a loss of status more than more realistic concerns (such as global warming). Nick wrote an interesting blog post about using GPG encrypted email on an Android phone [18]. I should get this going on my phone.

28 May 2014

John Goerzen: Review: The Wheel of Time

Five years ago, I opened The Eye of the World, the first book in the Wheel of Time series, and read:
Lord of the Morning, he said, I have come for you.

On the island, the air shimmered and coalesced. The black-clad man stood staring at the fiery mountain rising out of the plain. His face twisted in rage and contempt. You cannot escape so easily, Dragon. It is not done between us. It will not be done until the end of time. Then he was gone, and the mountain and the island stood alone. Waiting.
And then, as often, Robert Jordan began books citing some of the prophecy he invented:
And it came to pass in those days, as it had come before and would come again, that the Dark lay heavy on the land and weighed down the hearts of men, and the green things failed, and hope died. And men cried out to the Creator, saying, O Light of the Heavens, Light of the World, let the Promised One be born of the mountain, according to the prophecies, as he was in ages past and will be in ages to come. Let the Prince of the Morning sing to the land that green things will grow and the valleys give forth lambs. Let the arm of the Lord of the Dawn shelter us from the Dark, and the great sword of justice defend us. Let the Dragon ride again on the winds of time. from Charal Drianaan to Calamon,
The Cycle of the Dragon.
Author unknown, the Fourth Age
And then, the grand opening to each of the books in the 11,000-page series, something like this:
The Wheel of Time turns, and Ages come and pass, leaving memories that become legend. Legend fades to myth, and even myth is long forgotten when the Age that gave it birth comes again. In one Age, called the Third Age by some, an Age yet to come, an Age long past, a wind rose in the Mountains of Mist. The wind was not the beginning. There are neither beginnings nor endings to the turning of the Wheel of Time. But it was a beginning. Born below the ever cloud-capped peaks that gave the mountains their name, the wind blew east, out across the Sand Hills, once the shore of a great ocean, before the Breaking of the World. Down it flailed into the Two Rivers, into the tangled forest called the Westwood, and beat at two men walking with a cart and horse down the rock-strewn track called the Quarry Road.
And yesterday, 11,000 pages later, I have completed the series. Really, how could I have put down a book with such a majestic beginning as that? It hinted at the epic that was to come, the amazing literature I was going to read. I didn t read it straight through; I enjoyed books like War and Peace along the way. But what a trip the Wheel of Time series was. The characters were so detailed, so deep, the setting so elaborate, the plots so grand and also so small everyone from servants to queens were well-represented in the books. Not every section was a page-turner; in fact, around the 4th or 5th book, the series started to plod for a thousand pages or two, reading more like a history book than a novel. Sometimes the prejudices and stereotypes of the characters, particularly the sexism that both the men and women displayed, were frustrating to read. I never quite figured out of this was brilliant writing (making the reader feel frustrated at the plodding pace of the book when it was describing characters being frustrated at the plodding pace of events), or if it could have been trimmed down by 2/3 in some places. But really, the number of brilliant endings, laugh-out-loud situations, and page-turning suspense that somehow managed to build over hundreds, and then even thousands of pages, outweighed the flaws.
Politics is a foggy mire full of snakes. - The Eye of the World
Ah. Furlan washed his hands in one direction, then rubbed them in the other. Ah. Forgive me, Master Andra, but Lady Alys is a fierce sounding woman. Only with those who displease her, Lan said blandly. Her bite is far worse than her bark. Ah. Ah. Ah. Your rooms are this way. . . - The Dragon Reborn
One of the marks of truly good writing is that I leave a book or series with a sense of sadness or nostalgia, leaving characters that seem almost like friends. I felt that after finishing the last book in the series, A Memory of Light. The ending was deep, and satisfying, but ultimately left me wanting even more even more than 11,000 pages, somehow.
Elienda and Briain woke from the dream. She might have been speaking of the weather rather than two deaths among women she knew. We all must wake eventually. - Knife of Dreams
I was often touched by the way the Aiel referred to each other. A desert people, their fond farewall was may you always find water and shade. A fitting metaphor for anyone s life, I think. And they referred to their spouse as shade of my heart. Jordan touched on quite a few themes of religion and philosophy. In that quote above, he has the Aiel referring to death as waking from a dream. An intriguing metaphor, isn t it? He even writes conflicts of philosophy, religion, and culture into the series. I was particularly impressed with his treatment of theodicy and the problem of evil (the argument and religious response relating to the presence of evil in the world in the face of a loving God.) It is the most impressive treatment I have seen in fiction, and does a better job of advancing the theodistic argument than many philosophers have, I suspect. It is hard to really go into that one here because of the length and potential for spoilers. I can, however, comment on his treatment of existential nihilism the theory that life and the universe is without meaning, and the sometimes-related comment that what does anything matter then anyway? Here are a couple of quotes:
In Maradon, I saw what had been done to men who followed me. I saw Light in them, Min. Defying the Dark One no matter the length of his shadow. We will live, that defiance said. We will love and we will hope. And I saw him trying so hard to destroy that. He knows that if he could break them, it would mean something. Something much more than Maradon. Breaking the spirit of men he thirsts for that. He struck far harder than he otherwise would have because he wanted to break my spirit. His voice grew softer and he opened his eyes, looking down at her. And so I stood against him. - Towers of Midnight
What if I think it s all meaningless? he demanded with the loud voice of a king. What if I don t want it to keep turning? We live our lives by the blood of others! And those others become forgotten. What good is it if everything we know will fade? Great deeds or great tragedies, neither means anything! They will become legends, then those legends will be forgotten, then it will all start over again!

Why? Rand thought with wonder. Because each time we live, we get to love again.

It felt relaxing to stare out at that distant light, so welcoming and noble. Storms will soon come, it seemed to say. But for now, I am here. I am here. - The Gathering Storm
I unfortunately can t put in in its proper context due to spoilers, but it is a pivotal turning point in the series, relating to the preservation of the world. By giving what we might normally call a characteristic or behavior evil an almost-human voice and character, it can be discussed in interesting ways.
We are always more afraid than we wish to be, but we can always be braver than we expect. Hold on to your heart, and the Aes Sedai cannot harm what is really you, your heart. They are not nearly so far above us as we believed. May you always find water and shade, Egwene. And always remember your heart. - Lord of Chaos
Eben is dead. Would you want to forget your pain if you lost that hulking giant of yours? Have your feelings for him cut away like some spoiled chunk of flesh in an otherwise good roast? - The Gathering Storm
These quotes are examples of characters in the series dealing with difficult situations. In one, a character is yelled at for offering to use her special powers to numb the pain of another person over a death. It brings up a question that those of us that have known pain might find interesting: would we really have wanted our pain erased? And suggests perhaps not. That there is a benefit to it. That we get through it by embracing it by embracing it while still being true to our heart.
He had four rules concerning action and information. Never make a plan without knowing as much as you can of the enemy. Never be afraid to change your plans when you receive new information. Never believe you know everything. And never wait to know everything. - Lord of Chaos
Sometimes the series treats us to logical thinkers, whether they are generals or innkeepers, and gives us little tidbits like this one.
Androl brushed off his hands, smiling. Children were so adaptable. Before them, centuries of tradition, terror and superstition could melt away like butter left too long in the sun. - Towers of Midnight
And sometimes, things to chew over. Isn t that an interesting comment on prejudice? Children aren t born prejudiced; we can see how easily they make friends with anyone. It is sad that we teach them to be, isn t it? Wheel of Time is a fine story, expertly woven. But it is not just a story. It is true, in the sense that it illuminates truths, just like Romeo and Juliet can. In it, I see lessons and echoes of the civil rights era, comments on what makes good leaders, characters wrestling with decisions without adequate data, people putting on a mask of emotions, sacrifice, and the overall theme of the entire series: the power of love and compassion. The inspiration of the series for me well, it is there, but it may take a few more days to figure it out. I think it has something to do with this: The wind rose high and free, to soar in an open sky with no clouds.

Joachim Breitner: Predicting the livetime of a Hackage package

The Debian Haskell Group has no proper policy about when to update a certain package to a new version from Hackage. So far, we upgrade when one of us personally needs something, when someone nudges us, if its required as a dependency or just when we feel like it. I was thinking about ways to improve that. One thing that we should definitely do is to upgrade to new versions that differ only in the forth version number, e.g. from 1.2.3.4 to 1.2.3.5 these are most likely bug fix uploads and have little risk of breaking things. (Unless they are only changes to the .cabal file, then they might not be an improvement for our users or us.) I plan to code that soon. But what else can we do? Ideally we d package versions that will be the newest version for a long time, and not versions that are going to be replaced the next day. Unfortunately, deciding that requires visionary powers. But maybe there is a correlation between the upload history and the lifetime of a new package? Maybe there are patterns, e.g. the first upload after a long time tends to be replaced fast, but the third package in a short succession has a high chance to live long? Can we predict the livetime of a freshly uploaded package? So after a bit of hacking I got this graphic:
It needs a bit explanation: Both axis are time differences, the picture is one year by one year. For every release of which we know the lifetime (i.e. there has been an upload later), we draw its history on a vertical line. The horizontal position of the line corresponds to the lifetime of the release: A package that was superseded immediately later (which I sometimes do after spotting annoying typos) would appear on the far left, a package that is stable for one year on the far right. The history itself is a series of dots, representing the previous uploads of that package, with their distance from the lower edge indicating the time that has passed since then. So if a package had two updates half a year ago, and then no update for another half year, it would contribute two dots right above each other in the middle of the picture. Unfortunately, the picture does not yield any insight, besides that short-lived packages are more frequent than long-lived packages. So I tried another view:
I grouped uploads by the livetime of their two preceding uploads. For each such groups, the circle indicates the average livetime. The darkness indicates the absolute number in the sample. So we see a correlation between the packages livetime and the livetime of its predecessor, which is also not very surprising. We can state some hypothesis, like: A package replacing a very old package is likely to be long-lived if its pre-predecessor lived very long or very shortly, but less so if that lived for a few months. But I m not sure of the significance of these. I could have also taken into account which uploads are major and which are minor, by looking at the version number, but I didn t. What s the conclusion? Nothing much. I got some funny looking graphs. And there needs to be a way to render pictures like the first within the axes of a Chart diagram. I put the (somewhat hackish) code online feel free to play with it.

Tanguy Ortolo: GNU/Linux graphic sessions: allowing computer suspend and disabling a monitor

Allowing computer suspend Major desktop environments such as Xfce or KDE have a built-in computer suspend feature, but when you use a lighter alternative, things are a bit more complicated, because basically: only root can suspend the computer. Possible solutions include: With recent updates of the related Debian packages no idea of which one exactly the latter solution may not work any more, in which case it will only return the following error:
Error org.freedesktop.UPower.GeneralError: not authorized
It appears that this error is linked to ConsoleKit, a part of all this modern *Kit gizmo pile. If you are in this case, try prefixing your session launcher with the undocumented dark magic call ck-launch-session. For instance, this is what I have in my .xsession to launch my window manager i3:
exec ck-launch-session i3
Note: I do not know what ck-launch-session does exactly, why it is needed, and I do not want to know. To me, all that WhatsitKit pile is just some opaque, under-documented as in: no man page crap, that no one but their author really understand, designed to solve theoretical problems no one really cares about like: how to allow locally connected users to use the sound card while forbidding it to remote users while creating new issues such as this one. This stuff is too complex and under-documented for me to dive into it, so if it does not work out of the box, it is just some crap that gets in my way to using my computer as I wish. Disabling a monitor In some configurations, you have two monitors and want to disable one. For instance, in addition to my LCD monitor, I have a projector which I only use for movies. According to xorg.conf's man page, it can be disabled this way:
Section "Device"
    Identifier  "Internal graphic card"
    Option      "Monitor-DVI"   "LCD Monitor"
    Option      "Monitor-VGA"   "Projector"
EndSection
Section "Monitor"
    Identifier  "LCD Monitor"
EndSection
Section "Monitor"
    Identifier  "Projector"
    Option      "Enable"    "false"
EndSection
Except that does not work, because contrary to what the man page says the real option to use is not Enable but Disable! So here is the correct configuration to disable that monitor at start-up:
Section "Device"
    Identifier  "Internal graphic card"
    Option      "Monitor-DVI"   "LCD Monitor"
    Option      "Monitor-VGA"   "Projector"
EndSection
Section "Monitor"
    Identifier  "LCD Monitor"
EndSection
Section "Monitor"
    Identifier  "Projector"
    Option      "Disable"   "true"
EndSection
Note: yes, I will send a bug report with a patch against xorg.conf's man page.

27 May 2014

Jon Dowland: 2012 In Review

2013 is nearly all finished up and so I thought I'd spend a little time writing up what was noteable in the last twelve months. When I did so I found an unfinished draft from the year before. It would be a shame for it to go to waste, so here it is. 2012 was an interesting year in many respects with personal highs and lows. Every year I see a lots of "round-up"-style blog posts on the web, titled things like "2012 in music", which attempt to summarize the highlights of the year in that particular context. Here's JWZ's effort, for example. Often they are prefixed with statements like "2012 was a strong year for music" or whatever. For me, 2012 was not a particularly great year. I discovered quite a lot of stuff that I love that was new to me, but not new in any other sense. In Music, there were a bunch of come-back albums that made the headlines. I picked up both of Orbital's Wonky and Brian Eno's Lux (debatably a comeback: his first ambient record since 1983, his first solo effort since 2005, but his fourth collaborative effort on Warp in the naughties). I've enjoyed them both, but I've already forgotten Wonky and I still haven't fully embraced Lux (and On Land has not been knocked from the top spot when I want to listen to ambience.) There was also Throbbing Gristle's (or X-TG) final effort, a semi/post-TG, partly posthumous double-album swan song effort which, even more than Lux, I still haven't fully digested. In all honesty I think it was eclipsed by the surprise one-off release of a live recording of a TG side project featuring Nik Void of Factory Floor: Carter Tutti Void's Transverse, which is excellent. Ostensibly a four-track release, there's a studio excerpt V4 studio (Slap 1) which is available from (at least) Amazon. There's also a much more obscure fifth "unreleased" track cruX which I managed to "buy" from one of the web shops for zero cost. The other big musical surprise for me last year was Beth Jeans Houghton and the Hooves of Destiny: Yours Truly, Cellophane Nose. I knew nothing of BJH, although it turns out I've heard some of her singles repeatedly on Radio 6, but her band's guitarist Ed Blazey and his partner lived in the flat below me briefly. In that time I managed to get to the pub with him just once, but he kindly gave me a copy of their album on 12" afterwards. It reminds me a bit of Goldfrapp circa "Seventh Tree": I really like it and I'm looking forward to whatever they do next. Reznor's How To Destroy Angels squeezed out An Omen EP which failed to set my world on fire as a coherent collection, despite a few strong songs individually. In movies, sadly once again I'd say most of the things I recall seeing would be "also rans". Prometheus was a disappointment, although I will probably rewatch it in 2D at least once. The final Batman was fun although not groundbreaking to me and it didn't surpass Ledger's efforts in The Dark Knight. Inception remains my favourite Nolan by a long shot. Looper is perhaps the stand-out, not least because it came from nowhere and I managed to avoid any hype. In games, I moaned about having moaning about too many games, most of which are much older than 2012. I started Borderlands 2 after enjoying Borderlands (disqualified on age grounds) but to this day haven't persued it much further. I mostly played the two similar meta-games: The Playstation Plus download free games in a fixed time period and the more sporadic but bountiful humble bundle whack-a-mole. More on these another time. In reading, as is typical I mostly read stuff that was not written in 2012. Of that which was, Charles Stross's The Apocalypse Codex was an improvement over The Fuller Memorandum which I did not enjoy much, but in general I'm finding I much prefer Stross's older work to his newer; David Byrne's How Music Works was my first (and currently last) Google Books ebook purchase, and I read it entirely on a Nexus 7. I thoroughly enjoyed the book but the experience has not made a convert of me away from paper. He leans heavily on his own experiences which is inevitable but fortunately they are wide and numerous. Iain Banks' Stonemouth was an enjoyable romp around a fictional Scottish town (one which, I am reliably informed, is incredibly realistical rendered). One of his "mainstream" novels, It avoided a particular plot pattern that I've grown to dread with Banks, much to my suprise (and pleasure). Finally, the stand-out pleasant surprise novel of the year was Pratchett and Baxter's The Long Earth. With a plot device not unlike Banks' Transition or Stross's Family Trade series, the pair managed to write a journey-book capturing the sense-of-wonder that these multiverse plots are good for. (Or perhaps I have a weakness for them). It's hard to find the lines between Baxter and Pratchett's writing, but the debatably-reincarnated Tibetan Monk-cum-Artificial Intelligence 'Lobsang' must surely be Pratchett's. Pratchett managed to squeeze out another non-Discworld novel (Dodger) as well as a long-overdue short story collection, although I haven't read either of them yet. On to 2013's write-up...

Elena 'valhalla' Grandi: On using pgp.mit.edu

Dear lazyweb: does anybody know if the suggestion not to use pgp.mit.edu as a keyserver is still valid?

The only source that claims that it should not be used seems to be on OpenPGP Best Practices, but even there there are no explanation on why it shouldn't be used today (it does mention some old problems, but it also admits that they have been solved) or other sites that give that article as their source.

I don't use it as my primary keyserver, and there are others that work, but blacklisting a server without giving actual reasons sounds wrong to me, so I'd like to hear if some of those reasons are actually listed elsewhere.

P.S. if you don't have a friendica account to comment, this post will also appear on my pump.io account, or you can send me an email to the address listed on my homepage.

Edit: direct link to the pump.io post

26 May 2014

Clint Adams: Starring Danny Pudi as someone from Tandaloor

For ages I've been advocating for the dissolution of the tech-ctte, one of the biggest bugs in the Constitution. However, for the longest time I was unable to suggest a suitable replacement; there will always be people whose characters are so flawed that they will appeal to authority as a matter of first recourse, and those people need some sort of forum within which to waste time. Then the circus of #727708 came about, possibly the only useful thing the tech-ctte has ever done. Suddenly it became clear to me what would be a drastic improvement: replace the entire shitshow with a close analog of the Gowachin legal system from Frank Herbert's ConSentiency universe. Sure, it wouldn't fix the structural problems of a self-selecting group, or the related problem of the wider community not understanding such basic concepts as nomocracy, conflicts of interest, separation of duties, separation of powers, accountability, responsibility, independence, oversight, due diligence, corruption, transparency, and egomania, but it would still be better in nearly every aspect. Or maybe term-limit band-aids will magically fix everything.

Matthias Klumpp: Want AppStream metadata for your KDE project? Act now!

And again, another KDE-AppStream post ;-) If you want to know more about AppStream metadata and why adding it to your project is a good idea, you might be interested in this blogpost (and several previous ones I wrote). Originally, my plan was to directly push metadata to most KDE projects. The problem is that there is no way to reach all maintainers and have them opt-out for getting metadata pushed to their repositories. There is also no technical policy for a KDE project, since KDE is really only about the community right now, and there are no technical criteria a project under the KDE umbrella has to fulfill (at least to my knowledge, in theory, even GTK+ projects are perfectly fine within KDE). Since I feel very uncomfortable in touching other people s repositories without sending them a note first, I think the best way forward is an opt-in approach. So, if you want your KDE project to ship metadata, follow these simple steps: 1. Check if there already is metadata for your project That s right we already have some metadata available. Checkout the kde-appstream-metadata-templates repository at Github. You can take the XML file from there, if you want. Just make sure that there are no invalid tags in the description field (no <a/> nodes allowed, for example the content is not HTML!), check if you have an SPDX-compliant <project_license/> tag, check if the provided public interfaces in the <provides/> tag match your project and also test if the URLs work. Then you can copy the modified AppStream metadata to your project. 2. Write new metadata How to write new metadata is described in detail at this TechBase Wiki page. Just follow the instructions. In case you need help or want me to push the metadata to your project if you don t have the time, you can also write me an email: matthias [ AT ] tenstral . net or alternatively file a bug against the Github project linked above. Don t forget to have CMake install your shiny new metadata into /usr/share/appdata/. All metadata you add to your project will automatically get translated by the KDE l10n scripts, no further action is required. So far, projects like Ark, K3b and Calligra are shipping metadata, and the number of AppStream-aware projects in KDE is growing constantly, which greatly improves their visibility in software centers, and will help distributions a lot in organizing installed software. If you have further questions, please ask! .-)

Joachim Breitner: Does list fusion work?

I m writing this in the lunch break of TFP in Soesterberg. The invited talk this morning was by Geoffrey Mainland, and he talked about the difficulties of (informal) reasoning about performance in high-level languages like Haskell, especially with fancy stuff in the compiler like fusion. So I couldn t help but think about a (very small) help here. Consider the the two very similar expressions foldl (+) 0 [0..1000] and foldr (+) 0 [0..1000]. Which of these fuse away the list? Hopefully both, but hard to predict. So with my list-fusion-probe library, you can write
import Data.List.Fusion.Probe (fuseThis)
main = print $ foldr (+) 0 (fuseThis [0..1001])
and find out. If you compile this (with -O!), it will print
500500
If you change the foldr to foldl, it will print
Test: fuseThis: List did not fuse
So you can see that the function fuseThis :: [a] -> [a] does nothing if the list gets fused away, but causes a run-time error if not. It allows you to annotate your code with your assumptions of list fusion, and get shouted at if your assumptions are wrong. It wouldn t be hard to have the compiler give a warning or error message at compile time; we d just need to introduce a special function abortCompilation that, when found in the code during compilation, does just that. Note that you ll have trouble reproducing the above in GHC HEAD, where foldl does fuse (which is what I m going to talk about tomorrow here).

24 May 2014

Wouter Verhelst: kFreeBSD and ZFS

At one particular customer, increasing storage needs require that we add a large file server which should be as fast as possible. In that context, I'm currently evaluating using Debian GNU/kFreeBSD ("Debian," from now on) on ZFS. Not being hampered by a large degree of knowledge on the subject, here's a blog post with things I've learned so far. For context, the system on which we'll be installing will have 64 gigs of RAM, about two dozen SAS disks connected to a RAID controller of which I'm reasonably sure (though not 100% so) that it supports JBOD mode, and a PCIe-attached solid state storage device of several terabytes. If it turns out the RAID controller doesn't support JBOD mode, I'll probably create as many RAID0 devices as possible and hand those over to ZFS as "hard disks", instead; this wouldn't be an ideal solution, but I don't have much of a choice of hard disk attachment, unfortunately. That being said, the system on which I'm testing right now is the latest incantation of samba, an HP ProLiant ML115 server. A system with far more modest hardware specifications, it has 2G of RAM (which isn't really enough for ZFS), a single 160G SATA hard disk, and a dual-core AMD processor of about half a decade ago. And oh yes, no SSD. So for evaluating ZFS performance it's not ideal, but for evaluating Debian and ZFS features, it will do. Anyway, what I've found so far on samba: We'll see what else I learn.

John Goerzen: End of a year

AAAAHHH! LOOK OUT! A SHARK! Such were the cries a few first graders raised Monday. Laura and I had gone to school to eat lunch with Jacob this was his last week of first grade. We stayed for recess after lunch, and Jacob and his friend told us to pretend to be a shark lurking underneath the bridges and things of the playground. I was told that adults are too old to climb up here. So Laura and I started chasing Jacob and his friend around the playground equipment. And, much to their delight, they were usually able to evade us. And, to their even greater delight, sometimes I could reach up and brush a shoe or leg. Pretty soon Laura had enough of the game, but about then other kids started to notice what was going on. Shark-taunting began ( nana-booboo! ), and pretty soon half a dozen first-graders were running away from the shark! Eventually, it was time to go in, and it had been a great time. Jacob had been proud to have Laura and me there to eat lunch with him. He got to be the first in his class to go to lunch, happily slipping into the tour guide role he loves. Sometimes he would tell us in exacting detail what would happen. Other times, I d ask something like how will we know when lunch is over, and he d get a sly smile and say, You will see! The next day was Jacob s last at school, and Oliver had also recently finished preschool. So Laura and I planned a graduation party . We came up with some ideas of things they might like to do, and let them choose. Thankfully they both agreed on a choice: water slide. My parents gave them water slides one time, and we got one out, hooked it up to the hose, and let them have fun. And boy, did they ever! IMG_3222 And after the better part of an hour running with delight on the water slide, they decided to turn another slide into a water slide. So I helped them rig up a hose on our slide, and they d slide down it, and splash into a giant mud puddle at the bottom. Or, as the case may be, just stand in the mud, enjoying it. IMG_3275 We then went out to eat at one of their favorite restaurants and even had time to play in a park. Yes, a wonderful celebration. All four of us were wearing smiles that day. It reminded me of Jacob s school year, but also of the past year it was about a year ago that Jacob and Oliver first had the chance to really meet Laura. I have many photos and stories from the past year that I ll be sharing in the next little while. What an incredible year it has been!

23 May 2014

Floris Stoica-Marcu: GSoC2014 Debianizing Muon [Week 1]

Hello, My name is Floris (FlowRiser) and during GSoC I will be working at the project Getting Muon ready . I am guided by two awesome mentors: Sune Vuorela (svuorela) and Aleix Pol (apol). Essentially what we are trying to accomplish is not only port Muon to Debian, but also make it part of Debian; making users that use and love Debian confortable with it. Some of our goals are: integrating apt-listbugs into muon, making installing updates easier and contributing to both KDE and Debian at the same time. The exact plan (week by week), subject to change is here: https://wiki.debian.org/SummerOfCode2014/StudentApplications/FlorisAndreiStoicaMarcu. This week, we focused on getting the core functionality of Muon for Debian; Browsing packages, installing/updating, getting update notifications; Today I will also be making some packaging work, so that other people can begin testing Muon. The exact changes so far:
*changes to the .desktop files parsing, so you can browse packages as expected
*changes to the way more than one backend interract
*created a new releasechecker script for future use Plans for next week include:
*fixing kdesudo use
*fixing use of specific backend resources Cheers,
Floris.

Emanuele Rocca: A (very) brief history of Australia

This post is mostly a sum-up of the Wikipedia page History of Australia, with some content taken from History of the British Empire. Both texts are available under the Creative Commons Attribution-ShareAlike License. I do not seem to be able to learn about a new topic without taking notes: in this case I have decided to publish my work, hoping that someone will find it useful. Some very important themes such as the Gold Rush and Australian History during the World Wars have been impudently ignored.
Indigenous Australians The ancestors of Indigenous Australians are believed to have arrived in Australia 40,000 to 60,000 years ago, and possibly as early as 70,000 years ago. By 1788, the population of Australia existed as 250 individual nations, many of which were in alliance with one another, and within each nation there existed several clans, from as few as five or six to as many as 30 or 40. Each nation had its own language and a few had multiple, thus over 250 languages existed, around 200 of which are now extinct. Permanent European settlers arrived at Sydney in 1788 and came to control most of the continent by end of the 19th century. Bastions of largely unaltered Aboriginal societies survived, particularly in Northern and Western Australia into the 20th century, until finally, a group of Pintupi people of the Gibson Desert became the last people to be contacted by outsider ways in 1984.
European explorers Terra Australis (Latin for South Land) is one of the names given to a hypothetical continent which appeared on European maps between the 15th and 18th centuries. Although the landmass was drawn onto maps, Terra Australis was not based on any actual surveying of such a landmass but rather based on the hypothesis that continents in the Northern Hemisphere should be balanced by land in the south. The first documented European landing in Australia was made in 1606 by a Dutch ship led by Willem Janszoon. Hence the ancient name "Nova Hollandia". The same year, a Spanish expedition had landed in the New Hebrides and, believing them to be the fabled southern continent, named the land: "Terra Austral del Espiritu Santo". Hence the current name "Australia". Although various proposals for colonisation were made, notably by Pierre Purry from 1717 to 1744, none was officially attempted. Indigenous Australians were less able to trade with Europeans than were the peoples of India, the East Indies, China, and Japan. The Dutch East India Company concluded that there was "no good to be done there". In 1769, Lieutenant James Cook tried to locate the supposed Southern Continent. This continent was not found, and Cook decided to survey the east coast of New Holland, the only major part of that continent that had not been charted by Dutch navigators. Cook charted and took possession of the east coast of New Holland. He noted the following in his journal:
"I can land no more upon this Eastern coast of New Holland, and
 on the Western side I can make no new discovery the honour of
 which belongs to the Dutch Navigators and as such they may lay
 Claim to it as their property, but the Eastern Coast from the
 Latitude of 38 South down to this place I am confident was never
 seen or viseted by any European before us and therefore by the
 same Rule belongs to great Brittan."
Colonisation The American Revolutionary War (1775-1783) saw Great Britain lose most of its North American colonies and consider establishing replacement territories. The British colony of New South Wales was established with the arrival of the First Fleet of 11 vessels in January 1788. It consisted of over a thousand settlers, including 778 convicts (192 women and 586 men). A few days after arrival at Botany Bay the fleet moved to the more suitable Port Jackson where a settlement was established at Sydney Cove on 26 January 1788. This date later became Australia's national day, Australia Day. Between 1788 and 1868, approximately 161,700 convicts (of whom 25,000 were women) were transported to the Australian colonies of New South Wales, Van Diemen's land and Western Australia. Early colonial administrations were anxious to address the gender imbalance in the population brought about by the importation of large numbers of convict men. In 1835, the British Colonial Office issued the Proclamation of Governor Bourke, implementing the legal doctrine of terra nullius upon which British settlement was based, reinforcing the notion that the land belonged to no one prior to the British Crown taking possession of it and quashing any likelihood of treaties with Aboriginal peoples, including that signed by John Batman. Its publication meant that from then, all people found occupying land without the authority of the government would be considered illegal trespassers. A group in Britain led by Edward Gibbon Wakefield sought to start a colony based on free settlement and political and religious freedoms, rather than convict labour. The South Australia Act [1834], passed by the British Government which established the colony reflected these desires and included a promise of representative government when the population reached 50,000 people. Significantly, the Letters Patent enabling the South Australia Act 1834 included a guarantee of the rights of 'any Aboriginal Natives' and their descendants to lands they 'now actually occupied or enjoyed'. In 1836, two ships of the South Australia Land Company left to establish the first settlement on Kangaroo Island. The foundation of South Australia is now generally commemorated as Governor John Hindmarsh's Proclamation of the new Province at Glenelg, on the mainland, on 28 December 1836. By 1851 the colony was experimenting with a partially elected council.
Development of Australian democracy Traditional Aboriginal society had been governed by councils of elders and a corporate decision making process, but the first European-style governments established after 1788 were autocratic and run by appointed governors. The reformist attorney general, John Plunkett, sought to apply Enlightenment principles to governance in the colony, pursuing the establishment of equality before the law. Plunkett twice charged the colonist perpetrators of the Myall Creek massacre of Aborigines with murder, resulting in a conviction and his landmark Church Act of 1836 disestablished the Church of England and established legal equality between Anglicans, Catholics, Presbyterians and later Methodists. In 1840, the Adelaide City Council and the Sydney City Council were established. Men who possessed 1,000 pounds worth of property were able to stand for election and wealthy landowners were permitted up to four votes each in elections. Australia's first parliamentary elections were conducted for the New South Wales Legislative Council in 1843, again with voting rights (for males only) tied to property ownership or financial capacity. Voter rights were extended further in New South Wales in 1850 and elections for legislative councils were held in the colonies of Victoria, South Australia and Tasmania. Women became eligible to vote for the Parliament of South Australia in 1895. This was the first legislation in the world permitting women also to stand for election to political office and, in 1897, Catherine Helen Spence became the first female political candidate for political office, unsuccessfully standing for election as a delegate to the Federal Convention on Australian Federation. Western Australia granted voting rights to women in 1899. Early federal parliamentary reform and judicial interpretation sought to limit Aboriginal voting in practice, a situation which endured until rights activists began campaigning in the 1940s.
Road to independence Despite suspicion from some sections of the colonial community (especially in smaller colonies) about the value of nationhood, improvements in inter-colonial transport and communication, including the linking of Perth to the south eastern cities by telegraph in 1877, helped break down inter-colonial rivalries. New South Wales Premier Henry Parkes addressed a rural audience in his 1889 Tenterfield Oration, stating that the time had come to form a national executive government:
"Australia [now has] a population of three and a half millions,
 and the American people numbered only between three and four
 millions when they formed the great commonwealth of the United
 States. The numbers were about the same, and surely what the
 Americans had done by war, the Australians could bring about in
 peace, without breaking the ties that held them to the mother
 country."
Though Parkes would not live to see it, his vision would be achieved within a little over a decade, and he is remembered as the "father of federation". The Commonwealth of Australia came into being when the Federal Constitution was proclaimed by the Governor-General, Lord Hopetoun, on 1 January 1901. Australia took part in WWI. The contributions of Australian and New Zealand troops during the 1915 Gallipoli Campaign against the Ottoman Empire had a great impact on the national consciousness at home, and marked a watershed in the transition of Australia and New Zealand from colonies to nations in their own right. The countries continue to commemorate this occasion on ANZAC Day. Australia achieved independent Sovereign Nation status after World War I, under the Statute of Westminster, which defined Dominions of the British empire in the following way:
"They are autonomous Communities within the British Empire, equal
 in status, in no way subordinate one to another in any aspect of
 their domestic or external affairs, though united by a common
 allegiance to the Crown, and freely associated as members of the
 British Commonwealth of Nations."
The parliaments of Canada, Australia, New Zealand, the Union of South Africa, the Irish Free State and Newfoundland (currently part of Canada) were now independent of British legislative control, they could nullify British laws and Britain could no longer pass laws for them without their consent. The Australia Act 1986 removed any remaining links between the British Parliament and the Australian states.

22 May 2014

Mart&iacute;n Ferrari: Yakker, part 4: the client application

This is the fourth post in a series of posts (part 1, part 2, part 3) describing a secure alternative to applications like WhatsApp. I started with the following statement:
I believe it is possible to build a system with the simplicity and functionality of WhatsApp or Viber, which provides end-to-end encryption, is built on free software and open protocols, that supports federation and is almost decentralised, and that would allow interested companies to turn a profit without compromising any of these principles.
Now that most of the infrastructure has been described, in this post I will talk about the user-visible part: a mobile SIP client specially tailored for this architecture.

Features For Yakker to be successful, an application that is visually attractive and simple to use, while providing excellent call quality and stability, is critical. This idea is useless if only geeks adopt it: I want my parents to use it, I want my non-techie friends to use it. I want to tell them to switch from any of the other applications, not only because it is more secure, open and community-based, but also because it is better. For Android, there is already some excellent free applications that could be used as a starting point: Lumicall, CSipSimple and Linphone. I haven't tried it yet, but Linphone has a port to iOS too. Apart from the quality considerations, the following features must be added:
  • Certificate creation and proper storage in the mobile device.
  • An account creation wizard that interacts with the directory service and the account providers. Lumicall currently does something like this already, but only for their own ENUM and SIP server only.
  • Proper DNSSEC validating resolver to securely get the callee's certificate and ENUM record.
  • Optionally, use the directory service API to query records instead of DNS, to enhance privacy.
  • Periodic verification of the user's own ENUM and certificate records in DNS.
  • Use of those certificates to set up the SRTP stream, instead of unauthenticated encryption. Text messages must be encrypted in the same way, but included in the SIP message. SIPs encryption is not enough, as the proxies and service providers can read them.
  • Integration with the system's address book.
  • ENUM lookup, and SIP SRV lookup to detect phones with an associated SIP account.
  • When the called party does not have a SIP account, the client must offer to call using the PSTN gateway at service provider, if it provides one, and to call using the GSM network. It needs to be fast and reliable, so people can use it as the default texting and calling application.
  • Provide an interface to query account's balance, calling rates, and to buy credit. Possibly, offer this by opening a web browser, but negotiating authentication first, so the user does not need to enter an user name or password.
  • Capabilities to migrate to a different service provider.
  • A secure way to share the key pair with another trusted device.
  • A way to import a key pair created by the user manually.
I am probably missing a bunch of other capabilities that need to be implemented. It is a lot of work. Some of these features would only be useful for participants of the Yakker network, but there are others that could be useful for every SIP user, and therefore, implemented first. For example, SIP accounts that are not associated with a phone number could publish certificates under the same domain, and have the client use them to have secure communications with legacy infrastructure.

Funding I am aware that this amount of work is not going to happen overnight. In fact, without a bunch of people from the community interested in the project, it is never going to happen. The good news is that I think that companies might be interested in investing in this. A company could create their own branded version of the client, and use advertisements or service provider preference (the provider the user gets unless they choose one manually) to generate income. As long as the code remains free, and the client is compatible with the whole system, there could many competing clients out there. Their own promotion schemes would work for the benefit of the whole system, by bringing more users.

Risks The biggest threat would be of one client monopolising the network, and then changing the protocols to make the system a walled garden. It has already happened with GTalk, so there is precedent for this. I don't think there is any way to stop a big company from doing this, but one strategy to mitigate the risk a bit would be to create a brand (Yakker or whatever this ends up being called) and have it managed by a trusted community organisation, which can revoke the right to use the brand when a party is not behaving.

Steve Kemp: So firefox is dead to me now, sadly.

I've somehow managed to break firefox:
Random downloads fail
They appear in the download manager with "failed" next to them. Copying and pasting the URL and fetching via wget works.
Random extensions fail to install.
"Ghostery could not be installed because firefox cannot modify the needed file".
Moving ~/.cache/mozilla and ~/.mozilla out of the way don't help. Installing "tree style tabs" fails with no particular error. The actual error reads "The extension couldn't be installed because firefox couldn't modify the needed file". Googling didn't help, because it says "Create a new profile", which doesn't help, or "Disable extensions", which doesn't apply since none are present in the new profile. Running strace doesn't reveal any obvious EACCES, EPERM, or ENOENT errors so I'm struggling to spot an obvious problem. Downloading a binary firefox to /opt/firefox fails in the same way. Logging out of my desktop fails to make any difference. Annoying. The only thing I can say is:
shelob ~ $ dpkg --list   egrep '(xul icewea)'
ii  iceweasel     29.0.1-1~bpo70+1    ...
ii  xulrunner-29  29.0.1-1~bpo70+1    ...
For the moment I'm hating the use of chromium, but it will suffice until I can try to dig deeper.

21 May 2014

Russell Coker: Why I Use BTRFS

I ve just had to do yet another backup/format/restore operation on my workstation due to a BTRFS corruption problem, but as usual I didn t lose any data. The BTRFS data integrity features work reasonably well even when the filesystem gets into a state where the kernel will only accept a read-only mount. Given that the BTRFS tag on my blog is mostly about problems with BTRFS I think it s time that I explain why I use it in spite of the problems before people start to worry about my sanity or competence. The first thing to note is that BTRFS is fairly resiliant toward errors when mounted in read-only mode. When mounting a filesystem read-write there are a number of ways in which things can break which are often due to kernel code not being able to handle corrupt metadata I don t know how much of this is inherent to the design of BTRFS and how much is simply missing features in filesystem error handling. Some of the errors that I have had weren t entirely the fault of BTRFS, I twice had to do a backup/format/restore of my workstation due to a faulty DIMM corrupting memory (that has the potential to mess up any filesystem), but I still didn t lose any data AFAIK. The next thing to note is that I don t use BTRFS when doing paid sysadmin work. ZFS is a solid and reliable filesystem and it is working really well for my clients while BTRFS has too many issues at the moment. As an aside I m not interested in any comments about the ZFS license situation from anyone who s not officially representing Oracle. I also don t use BTRFS on systems that I can t access easily. The servers I have running BTRFS are all within an hour s drive from home, while driving for an hour on account of a kernel or filesystem error is really annoying it s not as bad as dealing with a remote server where I have no direct access. Reasons to Use BTRFS The benefits of BTRFS right now are snapshots (which are good for a first-line backup) and the basic data integrity features. I ve found these features to work well in real use. According to the comparison of Filesystems Wikipedia page ZFS and BTRFS are the only general purpose filesystems (IE for disks not tapes, NVRAM, or a cluster) that support checksums for all data and compression. Given that ZFS license issues will never allow it to be included in the Linux kernel tree it seems clear that BTRFS is the next significant filesystem for Linux. More testing of BTRFS is a good thing, while there are a number of known problems that the developers are working on it seems that more testing is needed now to find corner cases. Also we need a lot of testing to find bugs related to interactions with other software. I ve recently filed bug reports against the Debian installer because it can t install to a BTRFS RAID-1 (fortunately BTRFS supports changing to RAID-1 after installation) and because it doesn t support formatting an existing BTRFS filesystem (the mkfs program needs a -f option in that case). I also sent in a patch for the magic database used by file(1) to provide more information on BTRFS filesystems (which is in Debian/testing but not Debian/Wheezy). These are the sorts of things you encounter when routinely using software that you don t necessarily notice in basic testing. As an aside the Debian installation process failed at the GRUB step when I manually balanced a filesystem to use RAID-1 while the Debian installation was in progress. I didn t file a bug report because the best advice is to not mess with filesystems while the installer is running. I ll do a lot more testing of this when the Debian installer supports a BTRFS RAID-1 installation. A final thing that we need to work on is developing sysadmin best practices and scripts for managing BTRFS filesystems. I ve done some work on scripts to create snapshots for online backups but there are issues of managing free space etc. But working out how to best manage a new filesystem is something that takes years because there are many corner cases you may only encounter after a system has been running for a long time. So I really wouldn t want to be in the situation of using a new filesystem on an important server without having practice running it on less important systems, I did that with ZFS and now have a hacky first install that I have to support for years.

Next.

Previous.