Search Results: "beh"

12 June 2013

Daniel Pocock: When conspiracy theories come true

In a blog attempting to join the dots between Australia's intelligence services and the US PRISM program, I speculated that Australia has had extensive access to data harvested by PRISM. My hunch has just been exposed rather dramatically today: Top secret data center under construction to house all our secrets. "There has been no discussion of the project in Senate estimates committee hearings, and the public reports of Parliament's joint committee on intelligence and security make no reference to it." The news story also suggests that the relationship is bi-directional, with data harvested from Australian ISPs cross referenced against US databases from Facebook and Google. From a human rights perspective, the implications of this data sharing are much more dramatic than the NSA just snooping on US citizens within the US. While Australia often tries to portray the image that it is a safe and happy country, it has a dark side. Police are kept busy investigating politicians and Government officials: Prime Minister and her associates connected to a union slush fund, the central bank's banknote printing agency's chief executive and 8 colleagues on conspiracy charges and even a senator on good old fashioned shoplifting and assault charges Exporting PRISM data to a human rights backwater As an Australian, my own rights and those of my family are more heavily protected and respected living in Europe than if I lived in my country of birth. Australia does not have a bill of rights like the US (where PRISM data is harvested) or the European Convention of Human Rights. Once US data is exported to Australia, it can be used to support sinister programs like the indefinite detention of coloured people in sinister desert camps. Just this week, the Government left over 50 bodies of poor refugees for the sharks to eat. For someone with such a lack of a conscience, can you imagine them thinking twice about using PRISM data for unscrupulous purposes? Is US PRISM data at greater risk? Despite the stench of corruption and lip service to human rights, is there any hard evidence that PRISM data will be at greater risk if allowed into the hands of Australians? Death by SMS One former Australian official (having dual-citizenship with Sri Lanka) is accused of facilitating war crimes in the Sri Lankan civil war. At present he is Sri Lanka's ambassador at the UN, giving him the benefit of diplomatic immunity. When he was working for Australia, how much data did he have access to? Isn't it ludicrous that in the name of stopping terrorism, data may have been collected and handed to war criminals on a silver platter? Behind enemy lines Like James Bond in reverse, an Australian official was exposed unofficially engaging in a relationship with a Vietnamese spy colonel. In addition to the $20 million in bribery to win a bank note printing contract, is data being used as a tool to leverage trade negotiations, officially or otherwise? If an Australian official with top-secret clearance ends up under the control of a foreign agent, what is the risk that they would be exploited to obtain PRISM data? Defence minister resignation in Chinese spy scandal In 2009 Australia's defence minister was identified having close links to a Chinese-Australian business woman including all the usual little luxuries: first class flights, apartment rental and joint business deals with family members. He subsequently resigned. Little details were made public. How much PRISM data was directly available to political officials like the defence minister? Disappearing Prime Ministers The disappearance of a serving Prime Minister, Harold Holt, during the cold war remains an unsolved mystery to this day. People continue to ask was he swept out to sea or voluntarily whisked away in a Chinese or Soviet nuclear submarine. While Australia has traditionally had strong ties to Britain and the US, there have been exceptions to the rule. Fortunately this all happened before PRISM and Harold Holt red submarine rumours are no more well-founded than the UFO theories of his abduction. Where did all the data go? As the music industry has found out the hard way, digital data can be copied at will with the originator having no control. The revelations that Britain and Australia are in on the PRISM shenanigans only confirms that this data is seen as a new form of currency for exchange in geo-political posturing. While used weapons take decades to work their way down the chain, data can spread at the speed of light.

11 June 2013

Daniel Pocock: Interrupt-free computing

On debian-devel, there has been a discussion about the security issues of "spontaneously" appearing popups demanding the root password to make immediate security updates. There is a much more general issue related to this: computing without interruptions. Most of us have probably seen some friend or acquaintance with a (usually non-Linux) PC that is constantly beeping and flashing with chat notifications, new email popups, Adobe update this, Java update that, etc. In one recent case I came across somebody who had experienced a dramatic drop in his productivity as a consequence - giving him a laptop with a freshly installed copy of Linux made a dramatic difference to his work. I can already hear people insisting that security trumps everything (which isn't an original argument either) and that popups can't be avoided. A search on the web for "computing without interruptions" reveals users have a particular distaste for these things appearing while watching a video. Websites responding to that complaint fill the search results. With many types of interactive real-time content (video, WebRTC phone/video calls and so on) deployed within browsers, it is even more important for UI designers to contemplate when it is not appropriate to interrupt a user and to do everything possible to avoid interrupting the user. Preparing for disaster On the other hand, just ignoring security updates and not telling the user their disk is filling until 0 bytes remain available could only shift the problem down the road (from constant annoyance to periodic crisis). That said, sometimes you can still fill the disk very suddenly (especially with fast SSDs) and rather than relying on popups to keep users away from the precipice, applications (particularly the core desktop and daemon processes) could be tested more regularly to ensure they remain resilient in full disk situations. Managing information overload Popups are just part of a wider problem of information overload. There are emails too: some applications, such as Drupal, will send daily or weekly emails to a user if their system is not up to date. For many virtual-hosted sites, this starts to resemble a small flood. There is a flaw in this design: applications are competing for attention by sending more and more emails and popups or making them more annoying (e.g. the security updates in Debian 6 were ignorable popups in the top right-hand corner of the screen, Debian 7's Gnome Classic mode displays a big password prompt in the middle of the screen). The solution would be to develop a mechanism for unifying, de-duplicating and then prioritising these information/event flows. Some fault alerting systems already do this for their own events - these are niche solutions that aren't always applicable to the average PC-owner, although the principles are well tested. Some email organisation tools have similar features, but only for email. I'm not currently aware of any solution that synthesizes such an experience for all possible information sources. Setting priorities One well-read work on this subject in the business world is The 7 Habits of Highly Effective People (Stephen R. Covey, 1989). Of particular interest for the problem at hand is the priority matrix (borrowed from the Eisenhower Method): The left column, Urgent items, typically must be executed by a certain date (e.g. buying a gift before a birthday or installing a new SSL certificate before the old one expires). A security update or Acrobat reader update does not have this same characteristic. Under this model:
Urgent Non-urgent
Important Replace SSL certificate
Buy birthday gift
Security update
Run backup job
Not important Register for conference before deadline for free gift Non-security update for Acrobat reader
Covey even released an Outlook plugin, Plan Plus to help people organise their tasks (and their lives) using his methodology. Unfortunately it is closed-source software with a terrible set of ratings on Amazon - this review from a customer stands out: "My take is that Franklin does not consider robust software nor customer support to be either Urgent or Important." Could this be replicated more successfully with an open-source plugin for Mozilla Lightning or a similar productivity tool, and could the concept be extended across the range of data sources, including email, calendar items, system notifications and more to provide a unified approach to both the computing platform and general productivity (real-life) time management? Would this help solve the same problem in a more effective manner? In other words, would such an effort to help users integrate the demands of technology with the other demands of life make them more likely to keep their systems up to date? The wider community experience Going beyond the desktop/user experience, could this model be extended to automatically integrate external tasks, such as handling bug reports, moderating mailing lists and other slightly tedious things that have to be given regular attention to keep the free-software world moving along smoothly? Managing down-time For people who work in computing, there is almost no down-time any more. Even when on holiday, checking in for a flight might involve navigating through a buggy wifi access control system and an annoying set of advertisements from your low-cost airline as you try to print a boarding pass. These things often trigger thoughts about similar issues on client projects. Glancing at your email to find the booking number could awaken thoughts of a whole lot of projects you had tried to put out of your mind for a week. This is another area where excessive popups and emails can only compound the problem. Who really wants to download and install security updates while on holiday using an intermittent wifi connection? Managing all these events through a common mechanism may also finally make it possible to have an "ordinary user" experience with your PC. In practice, this might mean being able to view information/events through a time-of-day filter or "holiday mode" - and only on demand. A worthy design goal? Would any free software operating system make it a design goal to give their users a 100% interrupt-free experience? Of course there would still be things like chat notifications - but those would only be possible when a user has signed-in to a chat application. The distinction for the interrupt-free experience would only need to apply to default system behavior and not to every application.

8 June 2013

Steve Kemp: Migrations and movements

Recently I wanted to cleanup my "main" remote machine. It is a system I've had for many years, which started off as a i386 KVM-guest and was later migrated in place to an AMD64 installation. These days the host runs: The plan was originally to move the "mail stuff" to a new (wheezy) guest. I aborted that after discovering that the mutt-patched package has (IMHO) regressed. So today I spun up a new virtual machine, and configured it to host websites. Thus far I've migrated steve.org.uk, and lumail.org to the new host. Both simple sites that are built via my static-site-generator. Migration mostly involved configuring the proxy and the thttpd instances - then using rsync to migrate the content. I've renamed my current host, which was previously www.steve.org.uk and is now ssh.steve.org.uk, and pushed DNS changes. If all is smooth and happy I'll slowly migrate the rest of the sites. Fingers crossed this will be painless and I'll have a clean split between "login + mail" and "websites".

28 May 2013

Daniel Pocock: Google's XMPP users left high and dry

A few weeks ago, I was alarmed to discover Empathy's dependency on the Google version of XMPP, particularly for NAT traversal It was surprising to see those fears confirmed so quickly with Google moving away from open XMPP connectivity, particularly for multimedia protocols such as voice and video, leaving open source clients like Empathy who put their trust in Google high-and-dry. With Empathy being the default chat client on the Gnome desktop of Debian and Fedora, this is quite a serious situation. Fortunately, the latest versions of Debian 7 (wheezy) and Fedora 19 (Schrodinger's Cat (beta)) contain their best ever support for VoIP and RTC federation on the server side. Still under construction, the RTCquickStart.org guide tries to make it easier than ever to put the necessary packages to use - anybody who already runs their own mail, DNS or web server can become their own VoIP/SIP/XMPP provider in less than an hour. One particular point to note is that TURN servers work equally well for both SIP and XMPP users. Getting close to 100% support for VoIP users behind NAT requires a TURN server. It is possible to mix and match the reSIProcate TURN server, reTurn with any popular SIP and Jabber servers on Debian, Ubuntu or Fedora. Also look at the Turnserver.org project and RFC 5766 TURN server project I'd strongly encourage those who are concerned about this latest development to come and join the discussion in the Free RTC mailing list sponsored by the Free Software Foundation Europe

Russ Allbery: Collected haul

I've been slow lately in writing these up (and, for that matter, in doing most other things related to reading; things have been rather busy lately). This is a bunch of here-and-there purchases over the last few months, including Powell's Indiespensible shipments. Sandra Barret Face of the Enemy (sff)
Anne Bishop Written in Red (sff)
Lois McMaster Bujold Captain Vorpatril's Alliance (sff)
Cary Caffrey The Girls from Alcyone (sff)
Jenni Fagan The Panopticon (mainstream)
Niels Ferguson, et al. Cryptography Engineering (non-fiction)
Jen Kirchner The Fourth Channel (sff)
Anothony Marra A Constellation of Vital Phenomena (mainstream)
Steve McConnell Code Complete (non-fiction)
Seanan McGuire Velveteen vs. The Junior Super-Patriots (sff)
Patrick Nielsen Hayden, et al. (ed.) Some of the Best from Tor.com: 2012 (sff anthology)
Lisa O'Donnell The Death of Bees (mainstream)
Susan Palwich Flying in Place (sff)
Kim Stanley Robinson 2312 (sff)
John Scalzi Redshirts (sff)
Ian Tregillis Bitter Seeds (sff)
Leon Trotsky The History of the Russian Revolution (non-fiction)
Simon Van Booy The Illusion of Separateness (mainstream)
Chris Anne Wolfe Shadows of Aggar (sff)
Barbara Ann Wright The Pyramid Waltz (sff) That's a lot of stuff. It includes a couple of non-fiction O'Reilly books from sales, a few months of Powell's Indiespensible subscriptions, a variety of books I picked up after a discussion of good lesbian fiction on Tor.com (romance without the obnoxious gender tropes, or at least as many of them), and the rest of the Hugo nominees for the year. I got a ton of reading done earlier this month. I wish I could say the same thing about reviews, but I only wrote a few. That's something that I want to try to catch up on soon, so there will probably be a flurry of those posted soon. I've already read Blackout and Redshirts of this year's nominees (a review of the latter is coming), so at least I'm not too far behind on the reading. Throne of the Crescent Moon is in progress now.

24 May 2013

Wouter Verhelst: Not switching to 3.0 (quilt) just yet

I had a look at the 3.0 source formats just now. I've never been very fond of that, mainly because of the fact that it wants to store individual patches for no particularly good reason1. However, recently I found out about the --single-debian-patch option to dpkg-source, which makes the 3.0 (quilt) format behave somewhat more like the non-native 1.0 format, which is good. Unfortunately, "somewhat more" does not mean "entirely". For some weird reason, after converting the source to a 3.0 source package and running "dpkg-buildpackage", I find that suddenly my source tree is devoid of the patches that I'd applied to it. I'm sure there's another option hidden somewhere to make it not do that either, but I was quickly annoyed about it enough that I didn't care anymore. I just removed the debian/source directory entirely again, instead. Maybe I'll look into this again at some future point, but it's pretty unlikely. 1I have all my packages stored in git with a proper Vcs-Git: header; if people really want to look at individual patches, they can just use debcheckout, kthxbye.

Martin Pitt: umockdev 0.2.2 released

I did a 0.2.2 maintenance release for umockdev to fix building with Vala 0.16.1, gcc 4.8 (the changed sizeof behaviour caused segfaults), and current udev releases (umockdev-record stumbled over the new link priority fields of udevadm). There are also a couple of bug fixes, but no new features.

23 May 2013

Clint Adams: Oh, free software

I want a CLI WebDAV client that's better than cadaver or hdav. I want a program that can sync an .ics file with a CalDAV server, by dividing it up into events and individually synchronizing each of those. I don't have a clue how deletions would be handled, but that would be nice too. Then I want a program that can synchronize the .ics file with org-mode files. I want a SIP client that works as well as Twinkle but has the architecture of SFLphone or is a library upon which an arbitrary UI can be constructed. I want an HTML-rendering library that has callbacks or hooks for security- and privacy-relevant things like cookies and SSL certificates. I want at least one browser built on this library. I want it to support vi-like keybindings. I want an HTTP(S) proxy that can be dynamically-configured per-client or per-connection through a standardized protocol that web browsers or their plugins can speak. I want it to be able to handle all the relevant things covered by AdBlock Plus, RequestPolicy, and NoScript. I want HTTPS authentication through Monkeysphere and mod_gnutls. I want a git-annex backend for Ogg Vorbis files that treat the audio streams independently of the metadata yet stores them together in the same file so that everything behaves as usual but the annex doesn't bloat by 400Go after I run beets. I want a file transfer queuing system that can work over any sort of transport mechanism, direct or asynchronous, that handles partial transfers and throttling, and is generally magical. I want all kinds of accounting software improvements. I want sane PBX software. I want an OpenStack that doesn't use libvirt for KVM. I want backup software that behaves some weird hybrid of BoxBackup and Dirvish. I want a peer-to-peer card- and board-game platform that uses cryptographic assurance. I want everyone to use YAML instead of XML. I want a phone that's not running a doomed operating system. I want lots of other stuff.

22 May 2013

Lisandro Damián Nicanor Pérez Meyer: Debian/Ubuntu packages caching and mobile workstations

Not so long ago I read Dmitrijs' blog post on how to configure apt-cacher-ng to advertise it's service using avahi. As I normally use my laptop in my home and at work, and both networks have apt-cacher-ng running, I decided to give it a try.

I have been administering apt-cacher-ng for three networks so far, and I really find it a useful tool. Then, thanks to the aforementioned blog post, I discovered squid-deb-proxy. I don't use squid, so it's not for my normal use case, but some people will surely find it interesting.

But I found it's client package to be really interesting. It will discover any service providing _apt_proxy._tcp through avahi and let apt use it. But then the package wasn't available in Debian. So, I contacted Michael Vogt to see if he was interested in putting at least the client in Debian's archive. He took the opportunity to upload the full squid-deb-proxy, so thanks a lot Michael :-)

I then filled a wishlist bug against apt-cacher-ng to provide the avahi configuration for publishing the service, which Eduard included in the last version of it. So thanks a lot Eduard too!

tl;dr
You know only need apt-cacher-ng >= 0.7.13-1 and avahi-daemon installed on your server and your mobile users just need squid-deb-proxy-client. Then the proxy autoconfiguration for apt will just work.

One again, thanks a lot to the respective maintainers for allowing this into Jessie :-)

Gotchas
Yes, there are still some rough edges. On one of the networks I'm behind a proxy. While configuring my machine to use apt-cacher-ng's service as a proxy trough apt.conf, apt-listbugs would just work. But now, using the service as discovered by squid-deb-proxy-client, apt-listbugs just times out. Maybe I need to fill some other bug yet...

17 May 2013

Rob Bradford: GNOME in Moblin: Myzone

Howdy, i m sure most people are aware of the recent release of Moblin 2.0; a user experience for netbooks. I m going to write a few blog posts about how the Moblin user experience is built on the awesome technologies in the GNOME platform. So first up, let s look at the Myzone, we re starting here since this is the first thing I really worked on in the Moblin UX and i ve been able to see it through from early ideas to the 2.0 and 2.1 releases. So, deep breath, the idea behind the Myzone is to provide a springboard to things that matter to you most: your recent files and web pages you ve visited, your upcoming events and things you need to do, things that are happening on social web services and your favourite applications. Now then, that s the theory, how does it work:

14 May 2013

Matthew Palmer: A Modest Vocabulary Proposal

I would like to suggest that the word unprofessional be struck from the dictionary and anyone who uses it struck with a dictionary. It is a word which conveys no useful information or proposal for action, and is thus nothing but meaningless noise. The purpose of communication is to adjust another person s process of cognition. I ve heard it said that all communication is persuasion , which is quite true you re trying to persuade someone to change what they think. We can consider the intention and effectiveness of an attempt to communicate in this light. What is someone trying to achieve when they label a person or behaviour unprofessional ? If we re being charitable, we would probably say that they re trying to highlight that something is bad, or could be better. However, just stamping our foot and saying bad! isn t enough it s also important to provide some information that the recipient can act upon. The problem with the word unprofessional is that it really isn t specific enough on the subject of what is wrong . Have you ever had someone say something like, your behaviour yesterday was really unprofessional ? They re assuming you know what they re talking about and you might well have a reasonable guess but what if you guess wrong? Should you never do anything you did yesterday, just in case that particular thing was unprofessional? When I ve caught myself thinking, that was unprofessional , of my own behaviour, or someone else s, I think about what caused me to think that. Once I drill down into it, I usually come to the conclusion that what I really meant was, I don t like that . Since I m not paid to like things, that s pretty much irrelevant as a reason to tell someone not to do something. On the occasions when I come up with something more concrete, it is invariably a more useful expression than unprofessional . Things like, it frustrates the customer , or it pisses off the person sitting in the next cube are a much better expression of why something is bad than unprofessional . I d encourage everyone to keep a careful watch over themselves and those around them for use of the word. When you catch yourself saying it (or thinking it), examine your motives more closely. Whatever the more specific adjective is, use that instead. If it just comes down to I don t like that , at the very least say that to the person you re talking to. Don t try and hang anything grandiose on your personal prejudices. You might come off as being petty, but at least you ll be honest.

9 May 2013

Benjamin Mako Hill: The Remixing Dilemma: The Trade-off Between Generativity and Originality

This post was written with Andr s Monroy-Hern ndez. It is a summary of a paper just published in American Behavioral Scientist. You can also read the full paper: The remixing dilemma: The trade-off between generativity and originality. It is part of a series of papers I have written with Monroy-Hern ndez using data from Scratch. You can find the others on my academic website.
Remixing the reworking and recombination of existing creative artifacts represents a widespread, important, and controversial form of social creativity online. Proponents of remix culture often speak of remixing in terms of rich ecosystems where creative works are novel and highly generative. However, examples like this can be difficult to find. Although there is a steady stream of media being shared freely on the web, only a tiny fraction of these projects are remixed even once. On top of this, many remixes are not very different from the works they are built upon. Why is some content more attractive to remixers? Why are some projects remixed in deeper and more transformative ways?
Remix Diagram
We try to shed light on both of these questions using data from Scratch a large online remixing community. Although we find support for several popular theories, we also present evidence in support of a persistent trade-off that has broad practical and theoretical implications. In what we call the remixing dilemma, we suggest that characteristics of projects that are associated with higher rates of remixing are also associated with simpler and less transformative types of derivatives.

Our study is focused on two interrelated research questions. First, we ask why some projects shared in remixing communities are more or less generative than others. Generativity a term we borrow from Jonathan Zittrain describes creative works that are likely to inspire follow-on work. Several scholars have offered suggestions for why some creative works might be more generative than others. We focus on three central theories:
  1. Projects that are moderately complicated are more generative. The free and open source software motto release early and release often suggests that simple projects will offer more obvious opportunities for contribution than more polished projects. That said, projects that are extremely simple (e.g., completely blank slates) may also uninspiring to would-be contributors.
  2. Projects by prominent creators are more generative. The reasoning for this claim comes from the suggestion that remixing can act as a form of cultural conversation and that the work of popular creators can act like a common medium or language.
  3. Projects that are remixes themselves are more generative. The reasoning for this final claim comes from the idea that remixing thrives through the accumulation of contributions from groups of people building on each other s work.
Our second question focuses on the originality of remixes and asks when more or less transformative remixing occurs. For example, highly generative projects may be less exciting if the projects produced based on them are all near-identical copies of antecedent projects. For a series of reasons including the fact that increased generativity might come by attracting less interested, skilled, or motivated individuals we suggest that each of the factors associated with generativity will also be associated with less original forms of remixing. We call this trade-off the remixing dilemma. We answer both of our research questions using a detailed dataset from Scratch, where young people build, share, and collaborate on interactive animations and video games. The community was built to support users of the Scratch programming environment, a desktop application with functionality similar to Flash created by the Lifelong Kindergarten Group at the MIT Media Lab. Scratch is designed to allow users to build projects by integrating images, music, sound, and other media with programming code. Scratch is used by more than a million users, most of them under 18 years old. To test our three theories about generativity, we measure whether or not, as well as how many times, Scratch projects were remixed in a dataset that includes every shared project. Although Scratch is designed as a remixing community, only around one tenth of all Scratch projects are ever remixed. Because more popular projects are remixed more frequently simply because of exposure, we control for the number of times each project is viewed. Our analysis shows at least some support for all three theories of generativity described above. (1) Projects with moderate amounts of code are remixed more often than either very simple or very complex projects. (2) Projects by more prominent creators are more generative. (3) Remixes are more likely to attract remixers than de novo projects. To test our theory that there is a trade-off between generativity and originality, we build a dataset that includes every Scratch remix and its antecedent. For each pair, we construct a measure of originality by comparing the remix to its antecedent and computing an edit distance (a concept we borrow from software engineering) to determine how much the projects differ. We find strong evidence of a trade-off: (1) Projects of moderate complexity are remixed more lightly than more complicated projects. (2) Projects by more prominent creators tend to be remixed in less transformative ways. (3) Cumulative remixing tends to be associated with shallower and less transformative derivatives. That said, our support for (1) is qualified in that we do not find evidence of the increased originality for the simplest projects as our theory predicted.
Two plots of estimated values for prototypical projects. Panel 1 (left) display predicted probabilities of being remixed. Panel 2 (right) display predicted edit distances. Both panels show predicted values for both remixes and de novo projects from 0 to 1,204 blocks (99th percentile).

Two plots of estimated values for prototypical projects. Panel 1 (left) displays predicted probabilities of being remixed. Panel 2 (right) displays predicted edit distances. Both panels show predicted values for both remixes and de novo projects from 0 to 1,204 blocks (99th percentile).

We feel that our results raise difficult but important challenges, especially for the designers of social media systems. For example, many social media sites track and display user prominence with leaderboards or lists of aggregate views. This technique may lead to increased generativity by emphasizing and highlighting creator prominence. That said, it may also lead to a decrease in originality of the remixes elicited. Our results regarding the relationship of complexity to generativity and originality of remixes suggest that supporting increased complexity, at least for most projects, may have fewer drawbacks. As supporters and advocates of remixing, we feel that although highly generative works that lead to highly original derivatives may be rare and difficult for system designers to support, understanding remixing dynamics and encouraging these rare projects remain a worthwhile and important goal. Benjamin Mako Hill, Massachusetts Institute of Technology
Andr s Monroy-Hern ndez, Microsoft Research
For more, see our full paper, The remixing dilemma: The trade-off between generativity and originality. Published in American Behavioral Scientist. 57-5, Pp. 643 663. (Official Link, Pay-Walled ).

8 May 2013

Joey Hess: faster dh

With wheezy released, the floodgates are opened on a lot of debhelper changes that have been piling up. Most of these should be pretty minor, but I released one yesterday that will affect all users of dh. Hopefully in a good way. I made dh smarter about selecting which debhelper commands it runs. It can tell when a package does not use the stuff done by a particular command, and skips running the command entirely. So the debian/rules binary of a package using dh will now often look like this:
dh binary
   dh_testroot
   dh_prep
   dh_auto_install
   dh_installdocs
   dh_installchangelogs
   dh_perl
   dh_link
   dh_compress
   dh_fixperms
   dh_installdeb
   dh_gencontrol
   dh_md5sums
   dh_builddeb
Which is pretty close to the optimal hand-crafted debian/rules file (and just about as fast, too). But with the benefit that if you later add, say, cron job files, dh_installcron will automatically start being run too. Hopefully this will not result in any behavior changes, other than packages building faster and with less noise. If there is a bug it'll probably be something missing in the specification of when a command needs to be run. Beyond speed, I hope that this will help to lower the bar to adding new commands to debhelper, and to the default dh sequences. Before, every such new command slowed things down and was annoying. Now more special-purpose commands won't get in the way of packages that don't need them.
The way this works is that debhelper commands can include a "PROMISE" directive. An example from dh_installexamples
# PROMISE: DH NOOP WITHOUT examples
Mostly this specifies the files in debian/ that are used by the command, and whose presence triggers the command to run. There is also a syntax to specify items that can be present in the package build directory to trigger the command to run. (Unfortunatly, dh_perl can't use this. There's no good way to specify when dh_perl needs to run, short of doing nearly as much work as dh_perl would do when run. Oh well.) Note that third-party dh_ commands can include these directives too, if that makes sense.
I'm happy how this turned out, but I could be happier about the implementation. The PROMISE directives need to be maintained along with the code of the command. If another config file is added, they obviously must be updated. Other changes to a command can invalidate the PROMISE directive, and cause unexpected bugs. What would be ideal is to not repeat the inputs of the command in these directives, but instead write the command such that its inputs can be automatically extracted. I played around with some code like this:
$behavior = main_behavior("docs tmp(usr/share/doc/)", sub  
       my $package=shift;
       my $docs=shift;
       my $docdir=shift;
       install($docs, $docdir);
 );
$behavior->($package);
But refactoring all debhelper commands to be written in this style would be a big job. And I was not happy enough with the flexability and expressiveness of this to continue with it. I can however, dream about what this would look like if debhelper were written in Haskell. Then I would have a Debhelper a monad, within which each command executes.
main = runDebhelperIO installDocs
installDocs :: Monad a => Debhelper a
installDocs = do
    docs <- configFile "docs"
    docdir <- tmpDir "usr/share/doc"
    lift $ install docs docdir
To run the command, runDebhelperIO would loop over all the packages and run the action, in the Debhelper IO monad. But, this also allows making an examineDebhelper that takes an action like installDocs, and runs it in a Debhelper Writer monad. That would accumulate a list of all the inputs used by the action, and return it, without performing any side effecting IO actions. It's been 15 years since I last changed the language debhelper was written in. I did that for less gains than this, really. (The issue back then was that shell getopt sucked.) IIRC it was not very hard, and only took a few days. Still, I don't really anticipate reimplementing debhelper in Haskell any time soon. For one thing, individual Haskell binaries are quite large, statically linking all Haskell libraries they use, and so the installed size of debhelper would go up quite a bit. I hope that forthcoming changes will move things toward dynamically linked haskell libraries, and make it more appealing for projects that involve a lot of small commands. So, just a thought experiment for now..

7 May 2013

Timo Jyrinki: Qt 5 in Debian and Ubuntu, patches upstreaming

PackagesI quite like the current status of Qt 5 in Debian and Ubuntu (the links are to the qtbase packages, there are ca. 15 other modules as well). Despite Qt 5 being bleeding edge and Ubuntu having had the need to use it before even the first stable release came out in December, the co-operation with Debian has gone well. Debian is now having the first Qt 5 uploads done to experimental and later on to unstable. My work contributed to pkg-kde git on the modules has been welcomed, and even though more work has been done there by others, there haven't been drastic changes that would cause too big transition problems on the Ubuntu side. It has of course helped to ask others what they want, like the whole usage of qtchooser. Now with Qt 5.0.2 I've been able to mostly re-sync all newer changes / fixes to my packaging from Debian to Ubuntu and vice versa.

There will remain some delta, as pkg-kde plans to ask for a complete transition to qtchooser so that all Qt using packages would declare the Qt version either by QT_SELECT environment variable (preferable) or a package dependency (qt5-default or qt4-default). As a temporary change related to that, Debian will have a debhelper modification that defaults QT_SELECT to qt4 for the duration of the transition. Meanwhile, Ubuntu already shipped the 13.04 release with Qt 5, and a shortcut was taken there instead to prevent any Qt 4 package breakage. However, after the transition period in Debian is over, that small delta can again be removed.

I will also need to continue pushing any useful packaging I do to Debian. I pushed qtimageformats and qtdoc last week, but I know I'm still behind with some "possibly interesting" git snapshot modules like qtsensors and qtpim.

PatchesMore delta exists in the form of multiple patches related to the recent Ubuntu Touch efforts. I do not think they are of immediate interest to Debian let's start packaging Qt 5 apps to Debian first. However, about all of those patches have already been upstreamed to be part of Qt 5.1 or Qt 5.2, or will be later on. Some already were for 5.0.2.

A couple of months ago Ubuntu did have some patches hanging around with no clear author information. This was a result of the heated preparation for the Ubuntu Touch launches, and the fact that patches flew (too) quickly in place into various PPA:s. I started hunting down the authors, and the situation turned out to be better than I thought. About half of the patches were already upstreamed, and work on properly upstreaming the other ones was swiftly started after my initial contact. Proper DEP3 fields do help understanding the overall situation. There are now 10 Canonical individuals in the upstream group of contributors, and in the last week's sprint it turned out more people will be joining them to upstream their future patches.

Nowadays about all the requests I get for including patches from developers are stuff that was already upstreamed, like the XEmbed support in qtbase. This is how it should be.

One big patch still being Ubuntu only is the Unity appmenu support. There was a temporary solution for 13.04 that forward-ported the Qt 4 way of doing it. This will be however removed from the first 13.10 ('saucy') upload, as it's not upstreamable (the old way of supporting Unity appmenus was deliberately dropped from Qt 5). A re-implementation via QPA plugin support is on its way, but it may be that the development version users will be without appmenu support for some duration. Another big patch is related to qtwebkit's device pixel ratio, which will need to be fixed. Apart from these two areas of work that need to be followed through, patches situation is quite nice as mentioned.
ConclusionFree software will do world domination, and I'm happy to be part of it.

6 May 2013

Sylvestre Ledru: Follow up on Debian & Clang

A week after the Euro LLVM conference, I am glad to share the results I presented and discussed during the event. 1) Parallel building infrastructure using clang instead of gcc Based on the work that Alexander Pashaliyski and I did during the GSoC 2012, we are now able to automatically rebuild each package uploaded in Unstable transparently. Results are published here:
http://buildd-clang.debian.net/
This provides a quick and easy way to any Debian contributors or upstream packaged in Debian to check how the package behaves with clang.
For Debian Developer and maintainer, a per maintainer view is also provided. This will list all packages maintained by a Debian packager. My page as example Here is an example of the list of package:
And an example of package failing:
At time of writing, most of the packages in unstable have been rebuilt (16 % are remaining). However, due to resource constraints, only amd64 packages are built. Contact me if you have resources to share for other architectures. Thanks to Paul Wise, the PTS (Debian package tracking system) proposes a link to the clang build logs (failing or not). A repository will be automatically updated with the results of these builds (once I find the time to get into buildd). 2) clang-built repository
Thanks to the Amazon Cloud, with our new DPL (Debian Project Leader), Lucas Nussbaum, we rebuilt unstable last April and published it at the following address:
deb http://clang.debian.net/repository-2013-04-07/ unstable-clang main
This allows some cool stuff like:
$ echo "deb http://clang.debian.net/repository-2013-04-07/ unstable-clang main">>/etc/apt/sources.list
$ apt-get update
$ apt-get install coreutils/unstable-clang
$ ls
$ awk
Note that this repository was only one-shoot and does not get updated. Quality of the binary produced (performance and size) can be tested thanks to this repository. Feedbacks are welcome!

Sylvestre Ledru: Follow up on Debian & Clang

A week after the Euro LLVM conference, I am glad to share the results I presented and discussed during the event. 1) Parallel building infrastructure using clang instead of gcc Based on the work that Alexander Pashaliyski and I did during the GSoC 2012, we are now able to automatically rebuild each package uploaded in Unstable transparently. Results are published here:
http://buildd-clang.debian.net/
This provides a quick and easy way to any Debian contributors or upstream packaged in Debian to check how the package behaves with clang.
For Debian Developer and maintainer, a per maintainer view is also provided. This will list all packages maintained by a Debian packager. My page as example Here is an example of the list of package:
And an example of package failing:
At time of writing, most of the packages in unstable have been rebuilt (16 % are remaining). However, due to resource constraints, only amd64 packages are built. Contact me if you have resources to share for other architectures. Thanks to Paul Wise, the PTS (Debian package tracking system) proposes a link to the clang build logs (failing or not). A repository will be automatically updated with the results of these builds (once I find the time to get into buildd). 2) clang-built repository
Thanks to the Amazon Cloud, with our new DPL (Debian Project Leader), Lucas Nussbaum, we rebuilt unstable last April and published it at the following address:
deb http://clang.debian.net/repository-2013-04-07/ unstable-clang main
This allows some cool stuff like:
$ echo "deb http://clang.debian.net/repository-2013-04-07/ unstable-clang main">>/etc/apt/sources.list
$ apt-get update
$ apt-get install coreutils/unstable-clang
$ ls
$ awk
Note that this repository was only one-shoot and does not get updated. Quality of the binary produced (performance and size) can be tested thanks to this repository. Feedbacks are welcome!

Sylvestre Ledru: Follow up on Debian & Clang

A week after the Euro LLVM conference, I am glad to share the results I presented and discussed during the event. 1) Parallel building infrastructure using clang instead of gcc Based on the work that Alexander Pashaliyski and I did during the GSoC 2012, we are now able to automatically rebuild each package uploaded in Unstable transparently. Results are published here:
http://buildd-clang.debian.net/
This provides a quick and easy way to any Debian contributors or upstream packaged in Debian to check how the package behaves with clang.
For Debian Developer and maintainer, a per maintainer view is also provided. This will list all packages maintained by a Debian packager. My page as example Here is an example of the list of package:
And an example of package failing:
At time of writing, most of the packages in unstable have been rebuilt (16 % are remaining). However, due to resource constraints, only amd64 packages are built. Contact me if you have resources to share for other architectures. Thanks to Paul Wise, the PTS (Debian package tracking system) proposes a link to the clang build logs (failing or not). A repository will be automatically updated with the results of these builds (once I find the time to get into buildd). 2) clang-built repository
Thanks to the Amazon Cloud, with our new DPL (Debian Project Leader), Lucas Nussbaum, we rebuilt unstable last April and published it at the following address:
deb http://clang.debian.net/repository-2013-04-07/ unstable-clang main
This allows some cool stuff like:
$ echo "deb http://clang.debian.net/repository-2013-04-07/ unstable-clang main">>/etc/apt/sources.list
$ apt-get update
$ apt-get install coreutils/unstable-clang
$ ls
$ awk
Note that this repository was only one-shoot and does not get updated. Quality of the binary produced (performance and size) can be tested thanks to this repository. Feedbacks are welcome!

3 May 2013

Richard Hartmann: Release Critical Bug report for Week 18

Releezy Tomorrow morning at around 0900 UTC, a lot of very dedicated people will start working on the actual release of Debian 7.0 (wheezy). A huge thank you to everyone who worked on getting Debian to this point. I for one welcome our new penguin overlord! Housekeeping These will obviously be the last rc bug stats in this release cycle. I am still unsure what to do with this particular series until the next freeze and feedback would be appreciated. Possible approaches are continuing as is, putting this into its own specific feed, decreasing the frequency to once a month, or to simply stop until jessie is frozen. An ikiwiki poll may be overkill. Or not. Remaining bugs As of right now, there are two bugs open: Both relate to weird behaviour by apt when upgrading from squeeze to wheezy. These bugs will not be truly closed, but merely ignored, for wheezy's release. If you happen to encounter the message above, it's best to consult the release notes for possible workarounds. Stats The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the Squeeze release cycle?
Week Squeeze Wheezy Diff
43 284 (213+71) 468 (332+136) +184 (+119/+65)
44 261 (201+60) 408 (265+143) +147 (+64/+83)
45 261 (205+56) 425 (291+134) +164 (+86/+78)
46 271 (200+71) 401 (258+143) +130 (+58/+72)
47 283 (209+74) 366 (221+145) +83 (+12/+71)
48 256 (177+79) 378 (230+148) +122 (+53/+69)
49 256 (180+76) 360 (216+155) +104 (+36/+79)
50 204 (148+56) 339 (195+144) +135 (+47/+90)
51 178 (124+54) 323 (190+133) +145 (+66/+79)
52 115 (78+37) 289 (190+99) +174 (+112/+62)
1 93 (60+33) 287 (171+116) +194 (+111/+83)
2 82 (46+36) 271 (162+109) +189 (+116/+73)
3 25 (15+10) 249 (165+84) +224 (+150/+74)
4 14 (8+6) 244 (176+68) +230 (+168/+62)
5 2 (0+2) 224 (132+92) +222 (+132/+90)
6 release! 212 (129+83) +212 (+129/+83)
7 release+1 194 (128+66) +194 (+128/+66)
8 release+2 206 (144+62) +206 (+144/+62)
9 release+3 174 (105+69) +174 (+105/+69)
10 release+4 120 (72+48) +120 (+72/+48)
11 release+5 115 (74+41) +115 (+74/+41)
12 release+6 93 (47+46) +93 (+47/+46)
13 release+7 50 (24+26) +50 (+24/+26)
14 release+8 51 (32+19) +51 (+32/+19)
15 release+9 39 (32+7) +39 (+32/+7)
16 release+10 20 (12+8) +20 (+12/+8)
17 release+11 24 (19+5) +24 (+19/+5)
18 release+12 2 (2+0) +2 (+2/+0)
Graphical overview of bug stats thanks to azhag:

2 May 2013

Russ Allbery: Review: The System of the World

Review: The System of the World, by Neal Stephenson
Series: The Baroque Cycle #3
Publisher: William Morrow
Copyright: 2004
ISBN: 0-06-052387-5
Format: Hardcover
Pages: 892
This is the third book of the three-volume Baroque Cycle. I think you could, if you really wanted, read it without reading the previous volumes; Stephenson is certainly long-winded enough that you can pick up most of what's going on while you read. It's been a year since I read the second volume, and I only resorted to Wikipedia a couple of times to remember plot elements (and mostly from the first book). However, I wouldn't recommended starting here. Many of the character relationships, and most of the underpinning of the plot, is established in the previous volumes and given more significance by them. You would also miss The Confusion, which is the best book of the series, although none of this series rises to the level at which I'd recommend it except under specific circumstances. Quicksilver establishes the characters of Daniel Waterhouse, a fictional Puritan whose family was close to Cromwell and who became a friend to Isaac Newton in the days following the Restoration; Jack Shaftoe, a vagabond who wanders Europe in a sequence of improbable adventures; and Eliza, who becomes a friend to Leibniz and a spy for William of Orange. The Waterhouse sections are prominent in Quicksilver: full of the early history of the Royal Society, alchemy, and a small amount of politics. Of those three characters, Eliza is by far the most interesting, which meant that I was delighted when The Confusion dropped Waterhouse almost entirely and mixed Eliza's further story with more improbable but entertaining sea adventures of Jack Shaftoe. You will immediately sense my root problem with The System of the World when you hear that it is almost entirely about Daniel Waterhouse. While Eliza and Jack both appear, they play supporting roles at best, and Eliza's wonderful sharp intelligence and pragmatic survival skills are left out almost entirely. Instead, this is a novel about Waterhouse's return to England after spending quite a bit of time in the American colonies working on calculating machines. He is almost immediately entangled in dangerous politics from multiple directions: the precarious national politics in England near the end of the reign of Queen Anne, Isaac Newton's attempts to maintain the currency of England as Master of the Mint, and a bombing attempt that may have been aimed at him, may have been aimed at Newton, and may have been aimed at someone else entirely. Much of the book consists of an extended investigation of this bombing plot, skullduggery involving counterfeiters, and attempts to use the currency and the Mint as part of the political conflict between Whigs and Tories, mixed in with attempts to construct a very early computer (this is Stephenson, after all). Leibniz and Eliza come into this only as confidants of the Hanoverians. All this may sound exciting, and there are parts of it that hold the attention. But this book sprawls as badly as Quicksilver did. There's just too much detail without either enough plot or enough clarity. Stephenson tries to make you feel, smell, and hear the streets of London and the concerns of an idiosyncratic group of semi-nobles during one of the more interesting junctures of British history, but he does that by nearly drowning you in it, and without providing enough high-level guidance. For most of the book, I felt like I was being given a tour of a house on my hands and knees with a magnifying glass. It's a bad sign when the reader of a historical novel is regularly resorting to Wikipedia, not to follow interesting tangents of supporting material, but to try to get a basic sense of the players and the politics involved because the author never explains them clearly. If you're more familiar with the details of British history than I am, and can more easily follow the casual intermixing of two or three forms of address for the same historical figure, you may not have that problem. But I think other structural issues remain, and one of the largest is Waterhouse himself. Jack Shaftoe, and particularly Eliza, are more interesting characters because they're characters. They're not always particularly believable, but they attack the world with panache and are constantly squirming into the center of things. Stephenson's portrayals of Newton, Leibniz, the Duke of Marlborough, Sophia of Hanover, Peter the Great, and the other historical figures who show up here are interesting for different reasons: Stephenson has history to draw on and elaborate, and it's fascinating to meet those people from a different angle than dry lists of accomplishments. History has a way of providing random details that are too bizarre to make up; Isaac Newton, for example, actually did disguise himself to infiltrate London criminal society in pursuit of counterfeiters while he was Master of the Mint! Waterhouse, for me, has none of these advantages. He is an invented character in whom I have no pre-existing interest. He drifts through events largely through personal connections, all of which seem to be almost accidental. He's welcome in the councils of the Royal Society because he's apparently a scientist, but the amount of actual science we see him doing is quite limited. His nonconformist background allies him squarely with the Whigs, but his actual position on religious matters seems much less set than the others around him. What he seems to want, more than anything else, is to help Leibniz in the development of a computer and to reconcile Newton and Leibniz. And he's not particularly effective at either. In short, he has little in the way of memorable character or dynamism, despite being the primary viewpoint character, and seems to exist mostly to know everyone and be everywhere that's important to the story. He feels like an authorial insertion more than a character. It's quite easy to believe that Stephenson himself would have loved to be in exactly the role and situation that Waterhouse finds himself in throughout the book, in the middle of the councils of the wise and powerful, in just the right position to watch the events of history. I can sympathize, but it doesn't make for engrossing reading. Novels live and die by the strength of their characters, particularly their protagonists; I want more than just a neutral viewpoint. The third major structural problem that I had with this book is that I think Stephenson buries his lede. After finishing it, I think this is a book with a point, a central premise around which all the events of the story turn, and which is the philosophical culmination of The Baroque Cycle as a whole. But Stephenson seems oddly unwilling to state that premise outright until the very end of the book. For the first half, one could be forgiven in thinking this is a story about alchemy and the oddly heavy gold that's been a part of the story since The Confusion, or perhaps about foundational but forgotten work on computation that preceded Babbage by a century. But those all turn out to be side stories, sometimes even without a proper conclusion. I appreciate honoring the intelligence of the reader, and I presume that Stephenson would like to guide the reader through the same process of realization that the characters go through, but I think he takes this much too far and fails to make the realization clear. I'll therefore state what I believe is the premise outright, since I think it's a stronger book with this idea in mind: The System of the World is a continuation of the transformational economics shown in The Confusion into the realm of politics. Specifically, it's about the replacement of people with systems, about the journey towards Parliamentary supremacy, central banking, and the persistent state, and about the application of scientific principles of consistency and reproducibility to politics and economics (however fitfully and arbitrarily). Quicksilver was about the rise of science; The Confusion was, in retrospect, about the rise of economics; and The System of the World tries to be about the rise of technocratic modern politics, barely perceptible among the squabbles between Tories and Whigs. I think that's a fascinating premise, and I would have loved to read a book that tackles it head-on. That's a concept that is much more familiar from the late 19th and early 20th centuries in the context of Marxism, early socialism, technological utopianism, and similar attempted applications of scientific analysis to political and human behavior for the betterment of human civilization. Shifting that 200 years earlier and looking at a similar question from the perspective of the giants of the Enlightenment feels full of of potential. There are moments when I think Stephenson captures the sense of a seismic shift in how economies are run, knowledge is established, and civilizations are knit together. But, most of the time, it just isn't clear. There's so much other stuff in this book, and in the whole series: so many false starts, digressions, abandoned plots, discarded characters, and awkward attempts at romance (as much as I like the characters, Stephenson's portrayal of the relationship between Eliza and Jack is simply ridiculous and not particularly funny) that the whole weight of the edifice crushes what I think is the core concept. Stephenson is never going to be sparse. When you start a Stephenson novel, you know it's going to be full of chunks of partly digested encyclopedia and random research findings that may have nothing to do with the plot. But his best books (Snow Crash, The Diamond Age, even Cryptonomicon) have an underlying structure off of which all of those digressions are hung. You can see the bones beneath the flesh, and the creature they create is one you want to get to know. I'm not sure there are any bones here, and that may be the peril, for Stephenson, of writing historical fiction. I wonder if he felt that the structure of history would provide enough structure by itself that he could wrap a few plots around the outside of it and call it good. If so, it didn't work, at least for me. A lot of things happen. Some of them are even exciting and tense. A lot of people meet, interact, and show off their views of the world. A great deal of history, research, and sense of place is described in painstaking detail. But at the end of the book, I felt like I had to reach for some sort of point and try to retrofit it to the story. Lots happened, but there wasn't a novel. And that makes it quite hard to get enthused by the book. If you adored Quicksilver, I suspect you will also like this. I think they're the most similar. If, like I did, you thought The Confusion was a significant step up in enjoyment in the series and were hoping the trend will continue, I'm sad to report that it didn't. If you were considering whether to read the whole series and were waiting to see what I thought of the end, my advice is to give The Baroque Cycle a pass unless you absolutely love Stephenson's digressions, don't care if they're about history instead of current technology, and cannot live without 3,000 pages of them. It's not that they're bad books, but they're very long books, they take a significant investment of time and attention, and I think that, for most readers, there are other books that would repay a similar investment with more enjoyment. Rating: 5 out of 10

1 May 2013

Antoine Beaupr : New Debian release: my contributions to the #newinwheezy game

So while I haven't work exceptionally hard to help with the Debian Wheezy release (I merely help with a few RC bugs and organised a bug squashing party), I still have a few significant contributions to the release. So here's my claim to fame to the #newinwheezy game:

Next.

Previous.