Search Results: "beh"

6 August 2015

Eduard Sanou: Reproducible builds on Debian for GSoC 2015, 1st update

This is the second blog post I m writing about my experiences contributing to Debian for Google Summer of Code 2015 (check my first post)

Status update

First month It s been two months and a few days since the GSoC started. During the first month I worked on fixing specific packages, mainly concerning issues with timestamps, which is a very common source of unreproducibility. In many cases, during the build process files are compressed into gzip or zip archives, which store the creation time of files in the metadata. This can lead to unreproducible results when there is timezone variation between builds (easily fixed setting the timezone to UTC before the compression happens). In some cases the compressed files are generated during the build, and thus add build times in the metadata of compressed files (in this case the creation date of the files needs to be normalized somehow). As explained in my application, I finished exams on the end of June, that s why I chose to work on small fixes first, so that I could make the most out of my time between studying and finishing university projects and reports. I m happy with my first month, as I have worked as originally planned. Actually, my estimation of the number of bugs I could submit every week was surpassed in reality!

Second month Once the university was over, I started dedicating myself fully to the project. This allowed me to start working on toolchain fixes, following my original plan on working with timestamp related issues. In particular I have been working a lot in implementing a proposal for deterministic timestamps that appeared in the reproducible builds project. The idea is to define an environment variable called SOURCE_DATE_EPOCH that contains a known timestamp in Unix epoch format. With this variable exported, tools that would embed the localtime in their generated or processed files, can use this externally supplied date. This would happen only if SOURCE_DATE_EPOCH is exported, so the behaviour of the tool wouldn t change if the variable is not set. The first package I patched to implement this behaviour was gcc. The reason behind this is that there are about 420 unreproducible packages due to using the __DATE__, __TIME__ and __TIMESTAMP__ C macros. My patch changes the behavior of the macros __DATE__ and __TIME__ if SOURCE_DATE_EPOCH is exported. I submitted this patch to the gcc-patches list. Even though there was some interesting discussions in the list, the patch has not been accepted yet. Seeing how the reproducible builds idea is gaining momentum and becoming widespread, I m positive that at some point the gcc people will be more receptive for such patch. The second work with SOURCE_DATE_EPOCH was in debhelper; I patched this building tool to export the variable with the latest debian/changelog entry timestamp. With this patch, all the tools that run under dh will be able to use it to embed deterministic timestamps. Unfortunately some parts of the build process of some packages don t happen under debhelper, so the variable needs to be exported in a different way. Having submitted the debhelper patch allowed many packages to become reproducible after the tools that embedded timestamps were patched to honour SOURCE_DATE_EPOCH. As of today, the toolchain packages I have patched to do that are: gcc, libxslt, gettext, ghostscript and qt4-x11 (qhelpgenerator). I have also continued working on fixing individual packages affected by timestamps, random orderings (such as the ones from listing hash keys) and locale depending orderings; I have tagged packages in our infrastructure to note what kind of issue makes them unreproducible; I have updated some parts of the Reproducible Builds Wiki.

Impressions about reproducible builds The work I did during the first month felt a bit tedious sometimes: it didn t require much creativity or thinking as most of the fixes where quite mechanical, following a recipe. After I became free from university duties, I started looking into less trivial issues, which require more deep investigation and feel more rewarding once they are fixed. I also worked on toolchain fixes, which need more work and need more care. Fixing toolchain packages feels particularly rewarding because they can cause many packages to become reproducible at once. There is a very active community in the reproducible builds project! It s great to see so many people contributing to this project spending so many hours and putting so much effort. I ve felt very welcome from the beginning and I have gotten kind replies and helpful answers to all the questions and doubts I ve had, both from my mentor and from the other people in the project. I want to add that I m still amazed by the awesome infrastructure set up for the reproducible builds project. The team is using a Jenkins machine to continuously build packages to check for reproducibility, with irc notifications for changes, and also with a really useful web interface to list all the packages organized by issues that allows exploring them individually with all the available information. Also not only the infrastructure is used to build Debian amd64 packages, but also FreeBSD, NetBSD, OpenWRT, coreboot and lately Debian armhf with the help of a few new arm nodes.

Impressions about working on a free software project This was my first time working on a community driven free software project and I ve learned so many things. Something I learned during the first days, which is even more present in such wide project like the reproducible builds, is that contributing is not just writing patches for the features you want; you also need to convince the maintainer of the package to accept the patch! I was a bit surprised at the beginning because even if this is a Debian project, that aims to make changes to the whole distribution, decisions are not absolute for the whole Debian project. After taking decisions within the reproducible builds teams on how to approach things, we need to convince the rest of the Debian developers (mainly maintainers) to follow them, as they are allowed to disagree. So it is required to work together for solutions that makes everyone comfortable, usually with discussions on mailing lists or irc. There is another fact that I wasn t expecting before getting involved in this project. The kind of teamwork I have done previously involves having a leader who decides how stuff is done, who takes decisions when needed and oversees the whole project. There seems to be a different philosophy in Debian. Instead of having a leader, everyone tries their best, trying to convince the others that their solution is good, often by showing an implementation of the solution and providing proof that it works, rather than trying to get the solution accepted before coding it. Also, solutions and ideas are valued for their quality rather than by the position of the person submitting them, and there is no hierarchy within the group: all comments and advices are taken equally, valuing their usefulness regardless of who gives them. Usually there are no votes when deciding things. Members try their best on their approaches, trying to convince others as best as they can. And even if someone disagrees they may end up accepting the solution if they don t manage to convince the original proposer of doing things differently. The idea is to spend more time working and coding than arguing and deciding on the way to do things. So far I ve seen this approach to be very efficient :) I ve been told by my mentor that for difficult cases there exist a Debian committee that helps mediate on disagreements, but that is only used as a last option, probably when the discussion gets heated up.

Personal experience Overall I m very happy to finally having set my foot in the free software community, where I m able to contribute to the kind of software I have been using for years. The sense of community in Debian is really big, and everyone is invited to participate. I think that Google is doing an awesome job with the Google Summer of Code, not only because it gives a lot to free software but because it helps students to join the free software world as contributors (which is something that may be difficult to get into when you don t know how to begin, as it happened to myself for some time). I plan to continue contributing to the free software world, and I d encourage anyone to find projects to get involved and contribute as well! Happy hacking to everyone!

5 August 2015

Don Armstrong: Introducing dqsub

I've been using qsub for a while now on the cluster here at the IGB at UofI. qsub is a command line program which is used to submit jobs to a scheduler to eventually be run on one (or more) nodes of a cluster. Unfortunately, qsub's interface is horrible. It requires that you write a shell script for every single little thing you run, and doesn't do simple things like providing defaults or running multiple jobs at once with slightly different arguments. I've dealt with this for a while using some rudimentary shell scripting, but I finally had enough. So instead, I wrote a wrapper around qsub called dqsub. What used to require a complicated invocation like:
echo -e '#!/bin/bash\nmake foo'  \
 qsub -q default -S /bin/bash -d $(pwd) \
  -l mem=8G,nodes=1:ppn=4 -;
can now be run with
dqsub --mem 8G --ppn 4 make foo;
Want to run some command in every single directory which starts with SRX? That's easy:
ls -1 SRX* dqsub --mem 8G --ppn 4 --array chdir make bar;
Want instead to behave like xargs but do the same thing?
ls -1 SRX* dqsub --mem 8G --ppn 4 --array xargs make bar -C;
Now, this wrapper isn't complete yet, but it's already more than enough to do what I require, and has saved me quite a bit of time already. You can steal dqsub for yourself Feel free to request specific features, too.

4 August 2015

Matthew Garrett: Reverse this

The TODO group is an industry body that appears to be trying to define community best practices or something. I don't really know what their backstory is and whether they're trying to do meaningful work or just provide a fig leaf of respectability to organisations that dislike being criticised for doing nothing to improve the state of online communities but don't want to have to actually do anything, and their initial work on codes of conduct was, perhaps, suboptimal. But they do appear to be trying to improve things - this commit added a set of inappropriate behaviours, and also clarified that reverseisms were not actionable behaviour.

At which point Reddit lost its shit, because Reddit is garbage. And now the repository is a mess of white men attempting to explain how any policy that could allow them to be criticised is the real racism.

Fuck that shit.

Being a cis white man who's a native English speaker from a fairly well-off background, I'm pretty familiar with privilege. Spending my teenage years as an atheist of Irish Catholic upbringing in a Protestant school in a region of Northern Ireland that made parts of the bible belt look socially progressive, I'm also pretty familiar with the idea that that said privilege doesn't shield me from everything bad in life. Having privilege isn't a guarantee that my life will be better, in the same way that avoiding smoking doesn't mean I won't die of lung cancer. But there's an association in both cases, one that's strong enough to alter the statistical likelihood in meaningful ways.

And that inherently affects discussions about race or gender or sexuality. The probability that I've been subject to systematic discrimination because of these traits is vanishingly small. In the communities this policy is intended to cover, I'm the default. It's very difficult for any minority to exercise power over me. "You're white, you wouldn't understand" isn't fundamentally about my colour, it's about the fact that my colour means I haven't been subject to society trying to make my life more difficult at every opportunity. A community that considers saying that to be racist is a community that will never change the default, a community that will never be able to empower people who didn't grow up with that privilege. A code of conduct that makes it clear that "reverse racism" isn't grounds for complaint makes it clear that certain conversations are legitimate and helps ensure we have the framework we need to gradually change that default, and as such is better than one that doesn't.

(comments disabled because I don't trust any of you)

comment count unavailable comments

2 August 2015

Benjamin Mako Hill: Understanding Hydroplane Races for the New Seattleite

It s Seafair weekend in Seattle. As always, the centerpiece is the H1 Unlimited hydroplane races on Lake Washington. EllstromManufacturingHydroplaneIn my social circle, I m nearly the only person I know who grew up in area. None of the newcomers I know had heard of hydroplane racing before moving to Seattle. Even after I explain it to them i.e., boats with 3,000+ horse power airplane engines that fly just above the water at more than 320kph (200mph) leaving 10m+ (30ft) wakes behind them! most people seem more puzzled than interested. I grew up near the shore of Lake Washington and could see (and hear!) the races from my house. I don t follow hydroplane racing throughout the year but I do enjoy watching the races at Seafair. Here s my attempt to explain and make the case for the races to new Seattleites. Before Microsoft, Amazon, Starbucks, etc., there were basically three major Seattle industries: (1) logging and lumber based industries like paper manufacturing; (2) maritime industries like fishing, shipbuilding, shipping, and the navy; (3) aerospace (i.e., Boeing). Vintage hydroplane racing represented the Seattle trifecta: Wooden boats with airplane engines! The wooden U-60 Miss Thriftway circa 1955 (Thriftway is a Washinton-based supermarket that nobody outside has heard of) below is a picture of old-Seattle awesomeness. Modern hydroplanes are now made of fiberglass but two out of three isn t bad. miss_thriftwayAlthough the boats are racing this year in events in Indiana, San Diego, and Detroit in addition to the two races in Washington, hydroplane racing retains deep ties to the region. Most of the drivers are from the Seattle area. Many or most of the teams and boats are based in Washington throughout the year. Many of the sponsors are unknown outside of the state. This parochialness itself cultivates a certain kind of appeal among locals. In addition to old-Seattle/new-Seattle cultural divide, there s a class divide that I think is also worth challenging. Although the demographics of hydro-racing fans is surprisingly broad, it can seem like Formula One or NASCAR on the water. It seems safe to suggest that many of the demographic groups moving to Seattle for jobs in the tech industry are not big into motorsports. Although I m no follower of motorsports in general, I ve written before cultivated disinterest in professional sports, and it remains something that I believe is worth taking on. It s not all great. In particular, the close relationship between Seafair and the military makes me very uneasy. That said, even with the military-heavy airshow, I enjoy the way that Seafair weekend provides a little pocket of old-Seattle that remains effectively unchanged from when I was a kid. I d encourage others to enjoy it as well!

Benjamin Mako Hill: Understanding Hydroplane Races for the New Seattleite

It s Seafair weekend in Seattle. As always, the centerpiece is the H1 Unlimited hydroplane races on Lake Washington. EllstromManufacturingHydroplaneIn my social circle, I m nearly the only person I know who grew up in area. None of the newcomers I know had heard of hydroplane racing before moving to Seattle. Even after I explain it to them i.e., boats with 3,000+ horse power airplane engines that fly just above the water at more than 320kph (200mph) leaving 10m+ (30ft) wakes behind them! most people seem more puzzled than interested. I grew up near the shore of Lake Washington and could see (and hear!) the races from my house. I don t follow hydroplane racing throughout the year but I do enjoy watching the races at Seafair. Here s my attempt to explain and make the case for the races to new Seattleites. Before Microsoft, Amazon, Starbucks, etc., there were basically three major Seattle industries: (1) logging and lumber based industries like paper manufacturing; (2) maritime industries like fishing, shipbuilding, shipping, and the navy; (3) aerospace (i.e., Boeing). Vintage hydroplane racing represented the Seattle trifecta: Wooden boats with airplane engines! The wooden U-60 Miss Thriftway circa 1955 (Thriftway is a Washinton-based supermarket that nobody outside has heard of) below is a picture of old-Seattle awesomeness. Modern hydroplanes are now made of fiberglass but two out of three isn t bad. miss_thriftwayAlthough the boats are racing this year in events in Indiana, San Diego, and Detroit in addition to the two races in Washington, hydroplane racing retains deep ties to the region. Most of the drivers are from the Seattle area. Many or most of the teams and boats are based in Washington throughout the year. Many of the sponsors are unknown outside of the state. This parochialness itself cultivates a certain kind of appeal among locals. In addition to old-Seattle/new-Seattle cultural divide, there s a class divide that I think is also worth challenging. Although the demographics of hydro-racing fans is surprisingly broad, it can seem like Formula One or NASCAR on the water. It seems safe to suggest that many of the demographic groups moving to Seattle for jobs in the tech industry are not big into motorsports. Although I m no follower of motorsports in general, I ve written before cultivated disinterest in professional sports, and it remains something that I believe is worth taking on. It s not all great. In particular, the close relationship between Seafair and the military makes me very uneasy. That said, even with the military-heavy airshow, I enjoy the way that Seafair weekend provides a little pocket of old-Seattle that remains effectively unchanged from when I was a kid. I d encourage others to enjoy it as well!

28 July 2015

Lisandro Damián Nicanor Pérez Meyer: Plasma/KF5 : Testing situation

Dear Debian/KDE users,

We are aware that the current situation in testing is very unfortunate, with two main issues:

  1. systemsettings transitioned to testing before the corresponding KDE Control Modules. The result is that systemsettings displays an empty screen. This is tracked in the following bug https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=790703.
  2. plasmoids such as plasma-nm transitioned to testing before plasma-desktop 5. The result is that the plasmoid are no longer displayed in the system tray.

We are working on getting plasma-desktop to transition to testing as soon as possible (hopefully in 2 days time), which will resolve both those issues. We appreciate that the transition to KF5 is much rougher than we would have liked, and apologize to all those impacted.

On behalf of the Qt/KDE team,
Lisandro.

27 July 2015

Dirk Eddelbuettel: Evading the "Hadley tax": Faster Travis tests for R

Hadley is a popular figure, and rightly so as he successfully introduced many newcomers to the wonders offered by R. His approach strikes some of us old greybeards as wrong---I particularly take exception with some of his writing which frequently portrays a particular approach as both the best and only one. Real programming, I think, is often a little more nuanced and aware of tradeoffs which need to be balanced. As a book on another language once popularized: "There is more than one way to do things." But let us leave this discussion for another time. As the reach of the Hadleyverse keeps spreading, we sometimes find ourselves at the receiving end of a cost/benefit tradeoff. That is what this post is about, and it uses a very concrete case I encountered yesterday. As blogged earlier, the RcppZiggurat package was updated. I had not touched it in a year, but Brian Ripley had sent a brief and detailed note concerning something flagged by the Solaris compiler (correctly suggesting I replace fabs() with abs() on integer types). (Allow me to stray from the main story line here for a second to stress just how insane a work load he is carrying, essentially for all of us. R and the R community are so just so indebted to him for all his work---which makes the usual social media banter about him so unfortunate. But that too shall be left for another time.) Upon making the simple fix, and submitting to GitHub the usual Travis CI was triggered. And here is what I saw: first travis build in a year
All happy, all green. Previous build a year ago, most recent build yesterday, both passed. But hold on: test time went from 2:54 minutes to 7:47 minutes for an increase of almost five minutes! And I knew that I had not added any new dependencies, or altered any build options. What did happen was that among the dependencies of my package, one had decided to now also depend on ggplot2. Which leads to a chain of sixteen additional packages being loaded besides the four I depend upon---when it used to be just one. And that took five minutes as all those packages are installed from source, and some are big and take a long time to compile. There is however and easy alternative, and for that we have to praise Michael Rutter who looks after a number of things for R on Ubuntu. Among these are the R builds for Ubuntu but also the rrutter PPA as well as the c2d4u PPA. If you have not heard this alphabet soup before, a PPA is a package repository for Ubuntu where anyone (who wants to sign up) can upload (properly setup) source files which are then turned into Ubuntu binaries. With full dependency resolution and all other goodies we have come to expect from the Debian / Ubuntu universe. And Michael uses this facility with great skill and calm to provide us all with Ubuntu binaries for R itself (rebuilding what yours truly uploads into Debian), as well as a number of key packages available via the CRAN mirrors. Less know however is this "c2d4u" which stands for CRAN to Debian for Ubuntu. And this builds on something Charles Blundell once built under my mentorship in a Google Summer of Code. And Michael does a tremdous job covering well over a thousand CRAN source packages---and providing binaries for all. Which we can use for Travis! What all that means is that I could now replace the line
 - ./travis-tool.sh install_r RcppGSL rbenchmark microbenchmark highlight
which implies source builds of the four listed packages and all their dependencies with the following line implying binary installations of already built packages:
 - ./travis-tool.sh install_aptget libgsl0-dev r-cran-rcppgsl r-cran-rbenchmark r-cran-microbenchmark r-cran-highlight
In this particular case I also needed to build a binary package of my RcppGSL package as this one is not (yet) handled by Michael. I happen to have (re-)discovered the beauty of PPAs for Travis earlier this year and revitalized an older and largely dormant launchpad account I had for this PPA of mine. How to build a simple .deb package will also have to left for a future post to keep this more concise. This can be used with the existing r-travis setup---but one needs to use the older, initial variant in order to have the ability to install .deb packages. So in the .travis.yml of RcppZiggurat I just use
before_install:
## PPA for Rcpp and some other packages
- sudo add-apt-repository -y ppa:edd/misc
## r-travis by Craig Citro et al
- curl -OL http://raw.github.com/craigcitro/r-travis/master/scripts/travis-tool.sh
- chmod 755 ./travis-tool.sh
- ./travis-tool.sh bootstrap
to add my own PPA and all is good. If you do not have a PPA, or do not want to create your own packages you can still benefit from the PPAs by Michael and "mix and match" by installing from binary what is available, and from source what is not. Here we were able to use an all-binary approach, so let's see the resulting performance: latest travis build
Now we are at 1:03 to 1:15 minutes---much better. So to conclude, while the every expanding universe of R packages is fantastic for us as users, it can be seen to be placing a burden on us as developers when installing and testing. Fortunately, the packaging infrastructure built on top of Debian / Ubuntu packages can help and dramatically reduce build (and hence test) times. Learning about PPAs can be a helpful complement to learning about Travis and continued integration. So maybe now I need a new reason to blame Hadley? Well, there is always snake case ...

Follow-up: The post got some pretty immediate feedback shortly after I posted it. Craig Citro pointed out (quite correctly) that I could use r_binary_install which would also install the Ubuntu binaries based on their R packages names. Having built R/CRAN packages for Debian for so long, I am simply more used to the r-cran-* notations, and I think I was also the one contributing install_aptget to r-travis ... Yihui Xie spoke up for the "new" Travis approach deploying containers, caching of packages and explicit whitelists. It was in that very (GH-based) discussion that I started to really lose faith in the new Travis approach as they want use to whitelist each and every package. With 6900 and counting at CRAN I fear this simply does not scale. But different approaches are certainly welcome. I posted my 1:03 to 1:15 minutes result. If the "New School" can do it faster, I'd be all ears.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

21 July 2015

Dmitry Shachnev: GNOME Flashback 3.16 available in archive, needs your help

Some time ago GNOME Flashback 3.16/3.17 packages landed in Debian testing and Ubuntu wily. GNOME Flashback is the project which continues the development of components of classic GNOME session, including the GNOME Panel, the Metacity window manager, and so on. Screenshot The full changelog can be found in official announcement mail by Alberts and in changelog.gz files in each package, but I want to list the most imporant improvements in this release (compared to 3.14): Sounds interesting? Contribute! If you are interested in helping us, please write to our mailing list: gnome-flashback-list@gnome.org. The current TODO list is:
  1. Notification Daemon needs GTK notification support.
  2. GNOME Flashback needs screenshot, screencast, keyboard layout switching and bluetooth status icon.
  3. Fix/replace deprecated function usage in all modules.
  4. libstatus-notifier get it in usable state, create a new applet for gnome-panel.

19 July 2015

Laura Arjona: Family games: Robots

I play Robots with my kid. I ve tested the game with other kids and it seems that for ages 5 to 7 they like it. I ve talked about the game to several adults and it seems they like too, so I thought maybe writing about it here may be useful for somebody to enjoy some summer days. Players One player is the Robot. The other one is the programmer. If there are more players, it can be several robots and several programmers. If players are older, you can make the game more complicated making robots cooperate or programmers cooperate. If not, you make pairs 1-1 or 1 programmer 2 robots if the number is odd. The game The programmer must turn on the robot, pressing the ON/OFF button (robot chooses where s the button: nose, ear, belly, whatever).
Then, the robot say hello , and the programmer asks for the list of commands available (like Hello, robot, give me the list of commands ). The robot says the list of commands available, for example Run, stop, jump, sing a song, somersault, say something in a different language . Then, the programmer thinks a program, and loads it to the robot (speaks the list of orders, loudly, to the robot). Then the programmer presses the START button (Robot choses where it is) and then the robot has to perform the program without errors. If the robot performs correctly, wins one point. If it fails, looses one point. The programmer can design another program (maybe longer, maybe with some conditional expression) and tries the limits of the memory of robot. If the robot is tired, needs to charge batteries, or whatever, the roles programmer/robot are interchanged, and the one with more points in a certain amount of time or rounds, wins. Variants, tips If the programmer does not like the list, of commands, she can ask for updates, and maybe some new commands will be installed (and/or other uninstalled, who knows). Please be creative with the list of commands, or the game will be very boring. Depending on the operating system which runs the robot, it will give more or less options to the programmer, and the behaviour will be more evil or good. Robots shouldn t behave too much evil, though, otherwise the programmer will erase their disk and install Debian on them to make them obedient ;) You can play with a third person being the Robot manufacturer, who controls the robot, even sometimes overriding the programmer instructions (if the robot has an OS which is not free software). Robot will win one point obeying the manufacturer, but if there are more robots, will loose one round of playing because the programmer got angry and turned it off or reinstalled the software. The manufacturer and the programmer cooperate if the robot runs free software, though. Together they can expand robot memory (for example, lend a piece of paper where to store the program), or create new commands, fix bugs, or whatever. Comments? You can comment about this post in Pump.io thread about this post.
Filed under: My experiences and opinion Tagged: Debian, Education, English, Free culture, Free Software, Games, kids

14 July 2015

Matthew Palmer: Why DANE isn't going to win

In a comment to my previous post, Daniele asked the entirely reasonable question,
Would you like to comment on why you think that DNSSEC+DANE are not a possible and much better alternative?
Where DANE fails to be a feasible alternative to the current system is that it is not widely acknowledged to be superior in every possible way . A weak demonstration of this is that no browser has implemented DANE support, and very few other TLS-using applications have, either. The only thing I use which has DANE support that I m aware of is Postfix and SMTP is an application in which the limitations of DANE have far less impact. My understanding of the limitations of DANE, for large-scale deployment, are enumerated below.

DNS Is Awful Quoting Google security engineer Adam Langley:
But many (~4% in past tests) of users can t resolve a TXT record when they can resolve an A record for the same name. In practice, consumer DNS is hijacked by many devices that do a poor job of implementing DNS.
Consider that TXT records are far, far older than TLSA records. It seems likely that TLSA records would fail to be retrieved greater than 4% of the time. Extrapolate to the likely failure rate for lookup of TLSA records would be, and imagine what that would do to the reliability of DANE verification. It would either be completely unworkable, or else would cause a whole new round of just click through the security error training. Ugh. This also impacts DNSSEC itself. Lots of recursive resolvers don t validate DNSSEC, and some providers mangle DNS responses in some way, which breaks DNSSEC. Since OSes don t support DNSSEC validation by default (for example, by having the name resolution APIs indicate DNSSEC validation status), browsers would essentially have to ship their own validating resolver code. Some people have concerns around the single point of control for DNS records, too. While the weakest link nature of the CA model is terribad, there is a significant body of opinion that replacing it with a single, minimally-accountable organisation like ICANN isn t a great trade. Finally, performance is also a concern. Having to go out-of-band to retrieve TLSA records delays page generation, and nobody likes slow page loads.

DNSSEC Is Awful Lots of people don t like DNSSEC, for all sorts of reasons. While I don t think it is quite as bad as people make out (I ve deployed it for most zones I manage, there are some legitimate issues that mean browser vendors aren t willing to rely on DNSSEC. 1024 bit RSA keys are quite common throughout the DNSSEC system. Getting rid of 1024 bit keys in the PKI has been a long-running effort; doing the same for DNSSEC is likely to take quite a while. Yes, rapid rotation is possible, by splitting key-signing and zone-signing (a good design choice), but since it can t be enforced, it s entirely likely that long-lived 1024 bit keys for signing DNSSEC zones is the rule, rather than exception.

DNS Providers are Awful While we all poke fun at CAs who get compromised, consider how often someone s DNS control panel gets compromised. Now ponder the fact that, if DANE is supported, TLSA records can be manipulated in that DNS control panel. Those records would then automatically be DNSSEC signed by the DNS provider and served up to anyone who comes along. Ouch. In theory, of course, you should choose a suitably secure DNS provider, to prevent this problem. Given that there are regular hijackings of high-profile domains (which, presumably, the owners of those domains would also want to prevent), there is something in the DNS service provider market which prevents optimal consumer behaviour. Market for lemons, perchance?

Conclusion None of these problems are unsolvable, although none are trivial. I like DANE as a concept, and I d really, really like to see it succeed. However, the problems I ve listed above are all reasonable objections, made by people who have their hands in browser codebases, and so unless they re fixed, I don t see that anyone s going to be able to rely on DANE on the Internet for a long, long time to come.

6 July 2015

Ben Hutchings: Debian LTS work, June 2015

This was my seventh month working on Debian LTS. I was assigned 14.75 hours of work by Freexian's Debian LTS initiative. p7zip I did not receive any feedback from upstream for my proposed fix for CVE-2015-1038 mentioned last month, so I went ahead and uploaded it based on my own testing. (I also uploaded the fix to wheezy-security, jessie-security and sid.) Afterwards, I received a request from upstream for a patch against their latest release (even the version in sid is quite a long way behind that), so I ported the fix forward to that. linux-2.6 I backported further security fixes, but had to give up on one (CVE-2014-8172, AIO soft lockup) as the fix depends on wide-ranging changes. For CVE-2015-1805 (pipe iovec overrun leading to memory corruption), the upstream fix was also not applicable, but this looked so serious that we needed to fix it anyway. Red Hat had already fixed this in their 2.6.32-based kernel and they didn't have overlapping changes to the pipe implementation, so I was able to extract this fix from their source tarball. I uploaded and issued DLA-246-1. Unfortunately, I failed to notice that Linux 2.6.32.66 had introduced two regressions that were fixed in 2.6.32.67. While these didn't appear in my testing, one of them did affect several users that were quick to upgrade. I applied the upstream fixes, made a second upload and issued DLA-246-2. I also triaged the issues that are still unfixed, and I spent some time working on a fix for CVE-2015-1350 (unprivileged chown removes setcap attribute), but I haven't yet completed the backport to 2.6.32 or tested it. openssl I looked at OpenSSL, which is still marked as affected by CVE-2015-4000 (encryption downgrade aka Logjam). After discussion with the LTS team I made a note of the current situation, which is that a full fix (rejecting Diffie-Hellman keys shorter than 1024 bits) must wait until more servers have been upgraded.

2 July 2015

Petter Reinholdtsen: MakerCon Nordic videos now available on Frikanalen

Last oktober I was involved on behalf of NUUG with recording the talks at MakerCon Nordic, a conference for the Maker movement. Since then it has been the plan to publish the recordings on Frikanalen, which finally happened the last few days. A few talks are missing because the speakers asked the organizers to not publish them, but most of the talks are available. The talks are being broadcasted on RiksTV channel 50 and using multicast on Uninett, as well as being available from the Frikanalen web site. The unedited recordings are available on Youtube too. This is the list of talks available at the moment. Visit the Frikanalen video pages to view them. Part of the reason this took so long was that the scripts NUUG had to prepare a recording for publication were five years old and no longer worked with the current video processing tools (command line argument changes). In addition, we needed better audio normalization, which sent me on a detour to package bs1770gain for Debian. Now this is in place and it became a lot easier to publish NUUG videos on Frikanalen.

Michael Prokop: HAProxy with Debian/squeeze clients causing random Hash Sum mismatch

Update on 2015-07-02 22:15 UTC: as Petter Reinholdtsen noted in the comments:
Try adding /etc/apt/apt.conf.d/90squid with content like this: Acquire::http::Pipeline-Depth 0; It turn off the feature in apt confusing proxies.
this indeed avoids those Hash Sum mismatch failures with HAProxy as well. Thanks, Petter! Many of you might know apt s Hash Sum mismatch issue and there are plenty of bug reports about it (like #517874, #624122, #743298 + #762079). Recently I saw the Hash Sum mismatch usually only when using random mirrors with e.g. httpredir.debian.org in apt s sources.list, but with a static mirror such issues usually don t exist anymore. A customer of mine has a Debian mirror and this issue wasn t a problem there neither, until recently: Since the mirror also includes packages provided to customers and the mirror needs to be available 24/7 we decided to provide another instance of the mirror and put those systems behind HAProxy (version 1.5.8-3 as present in Debian/jessie). The HAProxy setup worked fine and we didn t notice any issues in our tests, until the daily Q/A builds randomly started to report failures:
Failed to fetch http://example.org/foobar_amd64.deb Hash Sum mismatch
When repeating the download there was no problem though. This problem only appeared about once every 15-20 minutes with random package files and it affected only Debian/squeeze clients (wheezy and jessie aren t affected at all). The problem also didn t appear when directly accessing the mirrors behind HAproxy. We tried plenty of different options for apt (Acquire::http::No-Cache=true, Acquire::http::No-Partial=true, ) and also played with some HAProxy configurations, nothing really helped. With apt s Debug::Acquire::http=True we saw that there really was a checksum failure and HTTP status code 102 ( Processing , or in terms of apt: Waiting for headers ) seems to be involved. The actual problem between apt on Debian/squeeze and HAProxy is still unknown to us though. While digging deeper into this issue is on my todo list yet, I found a way to avoid those Hash Sum mismatch failures: switch from http to https in sources.list. As soon as https is used the problem doesn t appear anymore. I m documenting it here just in case anyone else should run into it.

1 July 2015

Steve McIntyre: Quick trip to Sweden

Jo and I spent a few days in Sweden and had an awesome time! The main reason for being there was Leif and Maria's wedding way up north in Skellefte . They cunningly organised their ceremony for the Midsummer weekend, which was an excellent plan - we had a full weekend of partying while we were there. :-) the happy couple We had some time to ourselves while we were there, so we wandered about a little and got to see some of the beautiful coastal countryside. Trund n Then on the way home we stopped off in Ume to visit Mattias Wadenstein (maswan) and his wife Melanie, and he showed me around some of the machines that he's been admining on behalf of Debian. Maybe I'm a sad geek, but I feel quite a bond with one of the machines there, pettersson.debian.org. It's the official CD build machine for Debian, and I've been responsible for thrashing it really hard for the last 5 years or so... :-) Pettersson and friends Massive thanks to the University of Ume and their Academic Computer Club for hosting Debian machines and serving all the CD images for us! maswan and a lot of disks The only downsides from the trip were the massive tiredness (midnight sun is pretty, but notconducive to sleep!) the mosquito bites and the nasty plague^Wcold that we picked up while we were there... Ah well. :-)

18 June 2015

B lint R czey: Debian is preparing the transition to FFmpeg!

Ending an era of shipping Libav as default the Debian Multimedia Team is working out the last details of switching to FFmpeg. If you would like to read more about the reasons please read the rationale behind the change on the dedicated wiki page. If you feel so be welcome to test the new ffmpeg packages or join the discussion starting here. (Warning, the thread is loooong!)

11 June 2015

Petter Reinholdtsen: Measuring and adjusting the loudness of a TV channel using bs1770gain

Television loudness is the source of frustration for viewers everywhere. Some channels are very load, others are less loud, and ads tend to shout very high to get the attention of the viewers, and the viewers do not like this. This fact is well known to the TV channels. See for example the BBC white paper "Terminology for loudness and level dBTP, LU, and all that" from 2011 for a summary of the problem domain. To better address the need for even loadness, the TV channels got together several years ago to agree on a new way to measure loudness in digital files as one step in standardizing loudness. From this came the ITU-R standard BS.1770, "Algorithms to measure audio programme loudness and true-peak audio level". The ITU-R BS.1770 specification describe an algorithm to measure loadness in LUFS (Loudness Units, referenced to Full Scale). But having a way to measure is not enough. To get the same loudness across TV channels, one also need to decide which value to standardize on. For European TV channels, this was done in the EBU Recommondaton R128, "Loudness normalisation and permitted maximum level of audio signals", which specifies a recommended level of -23 LUFS. In Norway, I have been told that NRK, TV2, MTG and SBS have decided among themselves to follow the R128 recommondation for playout from 2016-03-01. There are free software available to measure and adjust the loudness level using the LUFS. In Debian, I am aware of a library named libebur128 able to measure the loudness and since yesterday morning a new binary named bs1770gain capable of both measuring and adjusting was uploaded and is waiting for NEW processing. I plan to maintain the latter in Debian under the Debian multimedia umbrella. The free software based TV channel I am involved in, Frikanalen, plan to follow the R128 recommondation ourself as soon as we can adjust the software to do so, and the bs1770gain tool seem like a good fit for that part of the puzzle to measure loudness on new video uploaded to Frikanalen. Personally, I plan to use bs1770gain to adjust the loudness of videos I upload to Frikanalen on behalf of the NUUG member organisation. The program seem to be able to measure the LUFS value of any media file handled by ffmpeg, but I've only successfully adjusted the LUFS value of WAV files. I suspect it should be able to adjust it for all the formats handled by ffmpeg.

10 June 2015

Norbert Preining: Tribute to Hermann Zapf

Last weekend Hermann Zapf, one of the greatest font designers and typographers of our times, has passed away at the age of 96. Much has been written about his life, his immense sphere of influence, his excellent typefaces, and some of the links to other obituaries can be found here at the end. There is nothing of value I can contribute, but let Hermann Zapf speak for himself with a few images from two books by or dedicated to him. zapf-heart-blood-ink The books I have taken these photos from are: zapf-books In his long life, Hermann Zapf has created a wide range of typefaces, some if which have become the main staple in the printing business, most notably Palatino, Janson, Optima, and Zapf Dingbats. But there are many more: zapf-alphabets While probably not as popular anymore as some years (decades?) ago, Palatino still is one of my favorite typefaces. I often wished to have the freedom to choose my own fonts for publications, but normally that doesn t happen. Palatino has found many offspring (or imitations?), and there are excellent renderings for use with (La)TeX, too. zapf-palatino Hermann Zapf also had connections with Don Knuth and the AMS (American Mathematical Society), and he designed for the AMS the Euler typeface, using Knuth s MetaFont program. zapf-euler Last weekend, when I heard of Hermann Zapf s death, I was at a conference at Tsukuba University. What a coincidence that all the signs in the building (floor numbers, indications, etc) were set optimain another of his great typefaces, the Optima. I had a strange feeling, and reset my presentation in Optima/Euler in honor of Hermann Zapf s great life. Although I never met Hermann Zapf personally I missed the TeX conferences where he participated it was to a great extent his influence that years ago I got started in typography. Living a life like he did, dedicating himself to the beauty and expressiveness of letters, design, calligraphy, seems like a dream for me. But beauty can be found even in the strangest equation in mathematics, the queerest theorem in logic. While not all of us are as gifted as Hermann Zapf in creating beauty, we should strive to increase our ability to appreciate the beauty. The beauty of letters is subtle, evasive, and needs training, that is practice, to be appreciated. Hermann Zapf taught us how even self-training can lead to excellence. His letters, which I meet every day, are a permanent reminder to strive for beauty and clarity, and never relent in our exercise. Thanks Hermann Zapf. zapf-letters Links to tributes around the world English Linotype (Type foundry where Zapf started working)
Bigelow and Holmes (Type designers)
New York Times (Newspaper)
Type Drawers
Quarz (News magazine)
Kaveh Bazargan (next president of TUG)
TeX Overflow (Q&A website concerning TeX) German FontShop (Type foundry and internet shop)
Deutschland Funk (News station)
S ddeutsche (Newspaper)
Echo (Newspaper)
Heise (Techology News page)

8 June 2015

Lunar: Reproducible builds: week 6 in Stretch cycle

What happened about the reproducible builds effort for this week: Presentations On May 26th,Holger Levsen presented reproducible builds in Debian at CCC Berlin for the Datengarten 52. The presentation was in German and the slides in English. Audio and video recordings are available. Toolchain fixes Niels Thykier fixed the experimental support for the automatic creation of debug packages in debhelper that being tested as part of the reproducible toolchain. Lunar added to the reproducible build version of dpkg the normalization of permissions for files in control.tar. The patch has also been submitted based on the main branch. Daniel Kahn Gillmor proposed a patch to add support for externally-supplying build date to help2man. This sparkled a discussion about agreeing on a common name for an environment variable to hold the date that should be used. It seems opinions are converging on using SOURCE_DATE_UTC which would hold a ISO-8601 formatted date in UTC) (e.g. 2015-06-05T01:08:20Z). Kudos to Daniel, Brendan O'Dea, Ximin Luo for pushing this forward. Lunar proposed a patch to Tar upstream adding a --clamp-mtime option as a generic solution for timestamp variations in tarballs which might also be useful for dpkg. The option changes the behavior of --mtime to only use the time specified if the file mtime is newer than the given time. So far, upstream is not convinced that it would make a worthwhile addition to Tar, though. Daniel Kahn Gillmor reached out to the libburnia project to ask for help on how to make ISO created with xorriso reproducible. We should reward Thomas Schmitt with a model upstream trophy as he went through a thorough analysis of possible sources of variations and ways to improve the situation. Most of what is missing with the current version in Debian is available in the latest upstream version, but libisoburn in Debian needs help. Daniel backported the missing option for version 1.3.2-1.1. akira submitted a new issue to Doxygen upstream regarding the timestamps added to the generated manpages. Packages fixed The following 49 packages became reproducible due to changes in their build dependencies: activemq-protobuf, bnfc, bridge-method-injector, commons-exec, console-data, djinn, github-backup, haskell-authenticate-oauth, haskell-authenticate, haskell-blaze-builder, haskell-blaze-textual, haskell-bloomfilter, haskell-brainfuck, haskell-hspec-discover, haskell-pretty-show, haskell-unlambda, haskell-x509-util, haskelldb-hdbc-odbc, haskelldb-hdbc-postgresql, haskelldb-hdbc-sqlite3, hasktags, hedgewars, hscolour, https-everywhere, java-comment-preprocessor, jffi, jgit, jnr-ffi, jnr-netdb, jsoup, lhs2tex, libcolor-calc-perl, libfile-changenotify-perl, libpdl-io-hdf5-perl, libsvn-notify-mirror-perl, localizer, maven-enforcer, pyotherside, python-xlrd, python-xstatic-angular-bootstrap, rt-extension-calendar, ruby-builder, ruby-em-hiredis, ruby-redcloth, shellcheck, sisu-plexus, tomcat-maven-plugin, v4l2loopback, vim-latexsuite. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which did not make their way to the archive yet: Daniel Kahn Gilmor also started discussions for emacs24 and the unsorted lists in generated .el files, the recording of a PID number in lush, and the reproducibility of ISO images in grub2. reproducible.debian.net Notifications are now sent when the build environment for a package has changed between two builds. This is a first step before automatically building the package once more. (Holger Levsen) jenkins.debian.net was upgraded to Debian Jessie. (Holger Levsen) A new variation is now being tested: $PATH. The second build will be done with a /i/capture/the/path added. (Holger Levsen) Holger Levsen with the help of Alexander Couzens wrote extra job to test the reproducibility of coreboot. Thanks James McCoy for helping with certificate issues. Mattia Rizollo made some more internal improvements. strip-nondeterminism development Andrew Ayer released strip-nondeterminism/0.008-1. This new version fixes the gzip handler so that it now skip adding a predetermined timestamp when there was none. Holger Levsen sponsored the upload. Documentation update The pages about timestamps in manpages generated by Doxygen, GHC .hi files, and Jar files have been updated to reflect their status in upstream. Markus Koschany documented an easy way to prevent Doxygen to write timestamps in HTML output. Package reviews 83 obsolete reviews have been removed, 71 added and 48 updated this week. Meetings A meeting was held on 2015-06-03. Minutes and full logs are available. It was agreed to hold such a meeting every two weeks for the time being. The time of the next meeting should be announced soon.

Craig Small: Checking Cloudflare SSL

My website for a while has used CloudFlare as its front-end. It s a rather nice setup and means my real server gets less of a hammering, which is a good thing. A few months ago they enabled a feature called Universal SSL which I have also added to my site. Around the same time, my SSL check scripts started failing for the website, the certificate had expired apparently many many days ago. Something wasn t right. The Problem The problem was simply at first I d get emails saying The SSL certificate for enc.com.au (CN: ) has expired! . I use a program called ssl-cert-check that would check all (web, smtp, imap) of my certificates. It s very easy to forget to renew and this program runs daily and does a simple check. Running the program on the command line gave some more information, but nothing (for me) that really helped:
$ /usr/bin/ssl-cert-check -s enc.com.au -p 443
Host Status Expires Days
----------------------------------------------- ------------ ------------ ----
unable to load certificate
140364897941136:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:701:Expecting: TRUSTED CERTIFICATE
unable to load certificate
139905089558160:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:701:Expecting: TRUSTED CERTIFICATE
unable to load certificate
140017829234320:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:701:Expecting: TRUSTED CERTIFICATE
unable to load certificate
140567473276560:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:701:Expecting: TRUSTED CERTIFICATE
enc.com.au:443 Expired -2457182
So, apparently, there was something wrong with the certificate. The problem was this was CloudFlare who seem to have a good idea on how to handle certificates and all my browsers were happy. ssl-cert-check is a shell script that uses openssl to make the connection, so the next stop was to see what openssl had to say.
$ echo ""   /usr/bin/openssl s_client -connect enc.com.au:443 CONNECTED(00000003)
140115756086928:error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error:s23_clnt.c:769:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 345 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
---
No peer certificate available. That was the clue I was looking for. Where s my Certificate? CloudFlare Universal SSL uses certificates that have multiple domains in the one certificate. The do this by having one canonical name which is something like sni(numbers).cloudflaressl.com and then multiple Subject Alternative Names (a bit like ServerAlias in apache configurations). This way a single server with a single certificate can serve multiple domains. The way that the client tells the server which website it is looking for is Server Name Indication (SNI). As part of the TLS handshaking the client tells the server I want website www.enc.com.au . The thing is, by default, both openssl s_client and the check script do not use this feature. That was fail the SSL certificate checks were failing. The server was waiting for the client to ask what website it wanted. Modern browsers do this automatically so it just works for them. The Fix For openssl on the command line, there is a flag -servername which does the trick nicely:
$ echo ""   /usr/bin/openssl s_client -connect enc.com.au:443 -servername enc.com.au 
CONNECTED(00000003)
depth=2 C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO ECC Certification Authority
verify error:num=20:unable to get local issuer certificate
---
(lots of good SSL type messages)
That was openssl happy now. We asked the server what website we were interested in with the -servername and got the certificate. The fix for ssl-cert-check is even simpler. Like a lot of things once you know the problem, the solution is not only easy to work out but someone has done it for you already. There is a Debian bug report on this problem with a simple fix from Francois Marier. Just edit the check script and change the line that has:
 TLSSERVERNAME="FALSE"
and change it to true. Then the script is happy too:
$ ssl-cert-check -s enc.com.au -p https
Host Status Expires Days
----------------------------------------------- ------------ ------------ ----
enc.com.au:https Valid Sep 30 2015 114
All working and as expected! This isn t really a CloudFlare problem as such, it is just that s the first place I had seen these sort of SNI certificates being used in something I administer (or more correctly something behind the something).

5 June 2015

Simon McVittie: Why polkit (or, how to mount a disk on modern Linux)

I've recently found myself explaining polkit (formerly PolicyKit) to one of Collabora's clients, and thought that blogging about the same topic might be useful for other people who are confused by it; so, here is why udisks2 and polkit are the way they are. As always, opinions in this blog are my own, not Collabora's. Privileged actions Broadly, there are two ways a process can do something: it can do it directly (i.e. ask the kernel directly), or it can use inter-process communication to ask a service to do that operation on its behalf. If it does it directly, the components that say whether it can succeed are the Linux kernel's normal permissions checks (DAC), and if configured, AppArmor, SELinux or a similar MAC layer. All very simple so far. Unfortunately, the kernel's relatively coarse-grained checks are not sufficient to express the sorts of policies that exist on a desktop/laptop/mobile system. My favourite example for this sort of thing is mounting filesystems. If I plug in a USB stick with a FAT filesystem, it's reasonable to expect my chosen user interface to either mount it automatically, or let me press a button to mount it. Similarly, to avoid data loss, I should be able to unmount it when I'm finished with it. However, mounting and unmounting a USB stick is fundamentally the same system call as mounting and unmounting any other filesystem - and if ordinary users can do arbitrary mount system calls, they can cause all sorts of chaos, for instance by mounting a filesystem that contains setuid executables (privilege escalation), or umounting a critical OS filesystem like /usr (denial of service). Something needs to arbitrate: you can mount filesystems, but only under certain conditions . The kernel developer motto for this sort of thing is mechanism, not policy : they are very keen to avoid encoding particular environments' policies (the sort of thing you could think of as business rules ) in the kernel, because that makes it non-generic and hard to maintain. As a result, direct mount/unmount actions are only allowed by privileged processes, and it's up to user-space processes to arrange for a privileged process to make the desired mount syscall. Here are some other privileged actions which laptop/desktop users can reasonably expect to just work , with or without requiring a sysadmin-like (root-equivalent) user: In environments that use a MAC framework like AppArmor, actions that would normally be allowed can become privileged: for instance, in a framework for sandboxed applications, most apps shouldn't be allowed to record audio. This prevents carrying out these actions directly, again resulting in the only way to achieve them being to ask a service to carry out the action. Ask a system service to do it On to the next design, then: I can submit a request to a privileged process, which does some checks to make sure I'm not trying to break the system (or alternatively, that I have enough sysadmin rights that I'm allowed to break the system if I want to), and then does the privileged action for me. You might think I'm about to start discussing D-Bus and daemons, but actually, a prominent earlier implementation of this was mount(8), which is normally setuid root:
% ls -l /bin/mount
-rwsr-xr-x 1 root root 40000 May 22 11:37 /bin/mount
If you look at it from an odd angle, this is inter-process communication across a privilege boundary: I run the setuid executable, creating a process. Because the executable has the setuid bit set, the kernel makes the process highly privileged: its effective uid is root, and it has all the necessary capabilities to mount filesystems. I submit the request by passing it in the command-line arguments. mount does some checks - specifically, it looks in /etc/fstab to see whether the filesystem I'm trying to mount has the user or users flag - then carries out the mount system call. There are a few obvious problems with this: Ask a system service to do it, via D-Bus or other IPC To avoid the issues of setuid, we could use inter-process communication in the traditional sense: run a privileged daemon (on boot or on-demand), make it listen for requests, and use the IPC channel as our privilege boundary. udisks2 is one such privileged daemon, which uses D-Bus as its IPC channel. D-Bus is a commonly-used inter-process system; one of its intended/designed uses is to let user processes and system services communicate, especially this sort of communication between a privileged daemon and its less-privileged clients. People sometimes criticize D-Bus as not doing anything you couldn't do yourself with some AF_UNIX sockets. Well, no, of course it doesn't - the important bit of the reference implementation and the various interoperable reimplementations consists of a daemon and some AF_UNIX sockets, and the rest is a simple matter of programming. However, it's sufficient for most uses in its problem space, and is usually better than inventing your own. The advantage of D-Bus over doing your own thing is precisely that you are not doing your own thing: good IPC design is hard, and D-Bus makes some structural decisions so that fewer application authors have to think about them. For instance, it has a central hub daemon (the dbus-daemon, or message bus ) so that n communicating applications don't need O(n ) sockets; it uses the dbus-daemon to provide a total message ordering so you don't have to think about message reordering; it has a distributed naming model (which can also be used as a distributed mutex) so you don't have to design that; it has a serialization format and a type system so you don't have to design one of those; it has a framework for activating" run-on-demand daemons so they don't have to use resources initially, implemented using a setuid helper and/or systemd; and so on. If you have religious objections to D-Bus, you can mentally replace D-Bus with AF_UNIX or something and most of this article will still be true. Is this OK? In either case - exec'ing a privileged helper, or submitting a request to a privileged daemon via IPC - the privileged process has two questions that it needs to answer before it does its work: It needs to make some sort of decision on the latter based on the information available to it. However, before we even get there, there is another layer: In the setuid model, there is a simple security check that you can apply: you can make /bin/mount only executable by a particular group, or only executable by certain AppArmor profiles, or similar. That works up to a point, but cannot distinguish between physically-present and not-physically-present users, or other facts that might be interesting to your local security policy. Similarly, in the IPC model, you can make certain communication channels impossible, for instance by using dbus-daemon's ability to decide which messages to deliver, or AF_UNIX sockets' filesystem permissions, or a MAC framework like AppArmor. Both of these are quite coarse-grained checks which don't really understand the finer details of what is going on. If the answer to "is this safe? is something of the form maybe, it depends on... , then they can't do the right thing: they must either let it through and let the domain-specific privileged process do the check, or deny it and lose potentially useful functionality. For instance, in an AppArmor environment, some applications have absolutely no legitimate reason to talk to udisks2, so the AppArmor policy can just block it altogether. However, once again, this is a coarse-grained check: the kernel has mechanism, not policy, and it doesn't know what the service does or why. If the application does need to be able to talk to the service at all, then finer-grained access control (obeying some, but not all, requests) has to be the service's job. dbus-daemon does have the ability to match messages in a relatively fine-grained way, based on the object path, interface and member in the message, as well as the routing information that it uses itself (i.e. the source and destination). However, it is not clear that this makes a great deal of sense conceptually: these are facts about the mechanics of the IPC, not facts about the domain-specific request (because the mechanics of the IPC are all that dbus-daemon understands). For instance, taking the udisks2 example again, dbus-daemon can't distinguish between an attempt to adjust mount options for a USB stick (probably fine) and an attempt to adjust mount options for /usr (not good). To have a domain-specific security policy, we need a domain-specific component, for instance udisks2, to get involved. Unlike dbus-daemon, udisks2 knows that not all disks are equal, knows which categories make sense to distinguish, and can identify which categories a particular disk is in. So udisks2 can make a more informed decision. So, a naive approach might be to write a function in udisks2 that looks something like this pseudocode:
may_i_mount_this_disk (user, disk, mount options)   boolean
 
    if (user is root   user is root-equivalent)
        return true;
    if (disk is not removable)
       return false;
    if (mount options are scary)
       return false;
    if (user is in  manipulate non-local disks  group)
        return true;
    if (user is not logged-in locally)
       return false;
    # https://en.wikipedia.org/wiki/Multiseat_configuration
    if (user is not logged-in on the same seat where the disk is
            plugged in)
       return false;
    return true;
 
Delegating the security policy to something central The pseudocode security policy outlined above is reasonably complicated already, and doesn't necessarily cover everything that you might want to consider. Meanwhile, not every system is the same. A general-purpose Linux distribution like Debian might run on server/mainframe systems with only remote users, personal laptops/desktops with one root-equivalent user, locked-down corporate laptops/desktops, mobile devices and so on; these systems should not necessarily all have the same security policy. Another interesting factor is that for some privileged operations, you might want to carry out interactive authorization: ask the requesting user to confirm that the action (which might have come from a background process) should take place (like Windows' UAC), or to prove that the person currently at the keyboard is the same as the person who logged in by giving their password (like sudo). We could in principle write code for all of this in udisks2, and in NetworkManager, and in systemd, ... - but that clearly doesn't scale, particularly if you want the security policy to be configurable. Enter polkit (formerly PolicyKit), a system service for applying security policies to actions. The way polkit works is that the application does its domain-specific analysis of the request - in the case of udisks2, whether the device to be mounted is removable, whether the mount options are reasonable, etc. - and converts it into an action. The action gives polkit a way to distinguish between things that are conceptually different, without needing to know the specifics. For instance, udisks2 currently divides up filesystem-mounting into org.freedesktop.udisks2.filesystem-mount, org.freedesktop.udisks2.filesystem-mount-fstab, org.freedesktop.udisks2.filesystem-mount-system and org.freedesktop.udisks2.filesystem-mount-other-seat. The application also finds the identity of the user making the request. Next, the application sends the action, the identity of the requesting user, and any other interesting facts to polkit. As currently implemented, polkit is a D-Bus service, so this is an IPC request via D-Bus. polkit consults its database of policies in order to choose one of several results: So how does polkit decide this? The first thing is that it reads the machine-readable description of the actions, in /usr/share/polkit-1/actions, which specifies a default policy. Next, it evaluates a local security policy to see what that says. In the current version of polkit, the local security policy is configured by writing JavaScript in /etc/polkit-1/rules.d (local policy) and /usr/share/polkit-1/rules.d (OS-vendor defaults). In older versions such as the one currently shipped in Debian unstable, there was a plugin architecture; but in practice nobody wrote plugins for it, and instead everyone used the example local authority shipped with polkit, which was configured via files in /etc/polkit-1/localauthority and /etc/polkit-1/localauthority.d. These policies can take into account useful facts like: For instance, gnome-control-center on Debian installs this snippet:
polkit.addRule(function(action, subject)  
        if ((action.id == "org.freedesktop.locale1.set-locale"  
             action.id == "org.freedesktop.locale1.set-keyboard"  
             action.id == "org.freedesktop.hostname1.set-static-hostname"  
             action.id == "org.freedesktop.hostname1.set-hostname"  
             action.id == "org.gnome.controlcenter.datetime.configure") &&
            subject.local &&
            subject.active &&
            subject.isInGroup ("sudo"))  
                    return polkit.Result.YES;
             
 );
which is reasonably close to being pseudocode for active local users in the sudo group may set the system locale, keyboard layout, hostname and time, without needing to authenticate . A system administrator could of course override that by dropping a higher-priority policy for some or all of these actions into /etc/polkit-1/rules.d. Summary

Next.

Previous.