Search Results: "will"

5 September 2015

Niels Thykier: The gcc-5 transition is coming to testing tonight

Thanks to hard work of Adam, Julien, Jonathan, Matthias, Scott, Simon and many others, the GCC-5/libstdc++ transition has progressed to a state, where we are ready to migrate the bulk of it to testing. It should be a mostly smooth ride. However, there will a few packages that are going to be uninstallable in testing for a few days and some packages will be temporarily removed from testing. If APT is unable to provide you with an upgrade for all of your packages, please try again in a few days. We apologise for the inconvenience. Currently, we expect about 36 binary packages to become temporarily uninstallable on amd64 and 34 on i386. This involves Britney accepting at least 4800 change items on testing (some of these are removals). Many thanks to Julien for providing a proposed set of hints and Adam extending them. Update: We now got a list of the packages being removed and a list of packages becoming uninstallable. It will be available on debian-devel within 20 minutes from now.
Filed under: Debian

Russell Coker: A Long Term Review of Android Devices

Xperia X10 My first Android device was The Sony Ericsson Xperia X10i [1]. One of the reasons I chose it was for the large 4 screen, nowadays the desirable phones (the ones that are marketed as premium products) are all bigger than that (the Galaxy S6 is 5.1 ) and even the slightly less expensive phones are bigger. At the moment Aldi is advertising an Android phone with a 4.5 screen for $129. But at the time there was nothing better in the price range that I was willing to pay. I devoted a lot of my first review to the default apps for SMS and Email. Shortly after that I realised that the default email app is never going to be adequate (I now use K9 mail) and the SMS app is barely adequate (but I mostly use instant messaging). I ve got used to the fact that most apps that ship with an Android device are worthless, the camera app and the app to make calls are the only built in apps I regularly use nowadays. In the bug list from my first review the major issue was lack of Wifi tethering which was fixed by an update to Android 2.3. Unfortunately Android 2.3 ran significantly more slowly which decreased the utility of the phone. The construction of the phone is very good. Over the last 2 years the 2 Xperia X10 phones I own have been on loan to various relatives, many of whom aren t really into technology and can t be expected to take good care of things. But they have not failed in any way. Apart from buying new batteries there has been no hardware failure in either phone. While 2 is a small sample size I haven t see any other Android device last nearly as long without problems. Unfortunately I have no reason to believe that Sony has continued to design devices as well. The Xperia X10 phones crash more often than most Android phones with spontaneous reboots being a daily occurrence. While that is worse than any other Android device I ve used it s not much worse. My second review of the Xperia X10 had a section about ways of reducing battery use [2]. Wow, I d forgotten how much that sucked! When I was last using the Xperia X10 the Life360 app that my wife and I use to track each other was taking 15% of the battery, on more recent phones the same app takes about 2%. The design of modern phones seems to be significantly more energy efficient for background tasks and the larger brighter displays use more energy instead. My father is using one of the Xperia phones now, when I give him a better phone to replace it I will have both as emergency Wifi access points. They aren t useful for much else nowadays. Samsung Galaxy S In my first review of the Galaxy S I criticised it for being thin, oddly shaped, and slippery [3]. After using it for a while I found the shape convenient as I could easily determine the bottom of the phone in my pocket and hold it the right way up before looking at it. This is a good feature for a phone that s small enough to rotate in my pocket the Samsung Galaxy Note series of phones is large enough to not rotate in a pocket. In retrospect I think that being slippery isn t a big deal as almost everyone buys a phone case anyway. But it would still be better for use on a desk if the bulge was at the top. I wrote about my Galaxy S failing [4]. Two of my relatives had problems with those phones too. Including a warranty replacement I ve seen 4 of those phones in use and only one worked reliably. The one that worked reliably is now being used by my mother, it s considerably faster than the Xperia X10 because it has more RAM and will probably remain in regular use until it breaks. CyanogenMod I tried using CyanogenMod [5]. The phone became defective 9 months later so even though CyanogenMod is great I don t think I got good value for the amount of time spent installing it. I haven t tried replacing the OS of an Android phone since then. I really wish that they would start manufacturing phones that can have the OS replaced as easily as a PC. Samsung Galaxy S3 and Wireless Charging The Galaxy S3 was the first phone I owned which competes with phones that are currently on sale [6]. A relative bought one at the same time as me and her phone is running well with no problems. But my S3 had some damage to it s USB port which means that the vast majority of USB cables don t charge it (only Samsung cables can be expected to work). After I bought the S3 I bought a Qi wireless phone charging device [7]. One of the reasons for buying that is so if a phone gets a broken USB port then I can still use it. It s ironic that the one phone that had a damaged USB port also failed to work correctly with the Qi card installed. The Qi charger is gathering dust. One significant benefit of the S3 (and most Samsung phones) is that it has a SD socket. I installed a 32G SD card in the S3 and now one of my relatives is happily using it as a media player. Nexus 4 I bought a Nexus 4 [8] for my wife as she needed a better phone but didn t feel like paying for a Galaxy S3. The Nexus 4 is a nice phone in many ways but the lack of storage is a serious problem. At the moment I m only keeping it to use with Google Cardboard, I will lend it to my parents soon. In retrospect I made a mistake buying the Nexus 4. If I had spent a little more money on another Galaxy S3 then I would have had a phone with a longer usage life as well as being able to swap accessories with my wife. The Nexus 4 seems reasonably solid, the back of the case (which is glass) broke on mine after a significant impact but the phone continues to work well. That s a tribute to the construction of the phone and also the Ringke Fusion case [9]. Generally the Nexus 4 is a good phone so I don t regret buying it. I just think that the Galaxy S3 was a better choice. Galaxy Note 2 I got a Samsung Galaxy Note 2 in mid 2013 [10]. In retrospect it was a mistake to buy the Galaxy S3, the Note series is better suited to my use. If I had known how good it is to have a larger phone I d have bought the original Galaxy Note when it was first released. Generally everything is good about the Note 2. While it only has 16G of storage (which isn t much by today s standards) it has an SD socket to allow expansion. It s currently being used by a relative as a small tablet. With a 32G SD card it can fit a lot of movies. Bluetooth Speakers I received Bluetooth speakers in late 2013 [11]. I was very impressed by them but ended up not using them for a while. After they gathered dust for about a year I started using them again recently. While nothing has changed regarding my review of the Hive speakers (which I still like a lot) it seems that my need for such things isn t as great as I thought. One thing that made me start using the Bluetooth speakers again is that my phone case blocks the sound from my latest phone and makes it worse than phone sound usually is. I bought Bluetooth speakers for some relatives as presents, the relatives seemed to appreciate them but I wonder how much they actually use them. Nexus 5 The Nexus 5 [12] is a nice phone. When I first reviewed it there were serious problems with overheating when playing Ingress. I haven t noticed such problems recently so I think that an update to Android might have made it more energy efficient. In that review I was very impressed by the FullHD screen and it made me want a Note 3, at the time I planned to get a Note 3 in the second half of 2014 (which I did). Galaxy Note 3 Almost a year ago I bought the Samsung Galaxy Note 3 [13]. I m quite happy with it at the moment but I don t have enough data for a long term review of it. The only thing to note so far is that in my first review I was unhappy with the USB 3 socket as that made it more difficult to connect a USB cable in the dark. I ve got used to the socket and I can now reliably plug it in at night with ease. I wrote about Rivers jeans being the only brand that can fit a Samsung Galaxy Note series phone in the pocket [14]. The pockets of my jeans have just started wearing out and I think that it s partly due to the fact that I bought a Armourdillo Hybrid case [15] for my Note 3. I ve had the jeans for over 3 years with no noticable wear apart from the pockets starting to wear out after 10 months of using the Armourdillo case. I don t think that the Armourdillo case is bad, but the fact that it has deep grooves and hard plastic causes it to rub more on material when I take the phone out of my pocket. As I check my phone very frequently this causes some serious wear. This isn t necessarily a problem given that a phone costs 20* more than a pair of jeans, if the case was actually needed to save the phone then it would be worth having some jeans wear out. But I don t think I need more protection than a gel case offers. Another problem is that the Armourdillo case is very difficult to remove. This isn t a problem if you don t need access to your phone, IE if you use a phone like the Nexus 5 that doesn t permit changing batteries or SD cards. But if you need to change batteries, SD cards, etc then it s really annoying. My wife seems quite happy with her Armoudillo case but I don t think it was a good choice for me. I m considering abandoning it and getting one of the cheap gel cases. The sound on the Note 3 is awful. I don t know how much of that is due to a limitation in the speaker and how much is due to the case. It s quite OK for phone calls but not much good for music. Tablets I m currently on my third tablet. One was too cheap and nasty so I returned it. Another was still cheap and I hardly ever used it. The third is a Galaxy Note 10 which works really well. I guess the lesson is to buy something worthwhile so you can use it. A tablet that s slower and has less storage than a phone probably isn t going to get used much. Phone Longevity I owned the Xperia X10 for 22 months before getting the Galaxy S3. As that included 9 months of using a Galaxy S I only had 13 months of use out of that phone before lending it to other people. The Galaxy S3 turned out to be a mistake as I replaced it in only 7 months. I had the Note 2 for 15 months before getting the Note 3. I have now had the Note 3 for 11 months and have no plans for a replacement any time soon this is the longest I ve owned an Android phone and been totally satisfied with it. Also I only need to use it for another 4 months to set a record for using an Android phone. The Xperia was free as part of a telco contract. The other phones were somewhere between $500 and $600 each when counting the accessories (case, battery, etc) that I bought with them. So in 4 years and 7 months I ve spent somewhere between $1500 and $1800 on phones plus the cost of the Xperia that was built in to the contract. The Xperia probably cost about the same so I ll assume that I spent $2000 on phones and accessories. This seems like a lot. However that averages out to about $1.20 per day (and hopefully a lot less if my Note 3 lasts another couple of years). I could justify $1.20 per day for either the amount of paid work I do on Android phones or the amount of recreational activities that I perform (the Galaxy S3 was largely purchased for Ingress). Conclusion I think that phone companies will be struggling to maintain sales of high end phones in the future. When I chose the Xperia X10 I knew I was making a compromise, the screen resolution was an obvious limitation on the use of the device (even though it was one of the best devices available). The storage in the Xperia was also a limitation. Now FullHD is the minimum resolution for any sort of high-end device and 32G of storage is small. I think that most people would struggle to observe any improvement over a Nexus 5 or Note 3 at this time. I think that this explains the massive advertising campaign for the Galaxy S6 that is going on at the moment. Samsung can t sell the S6 based on it being better than previous phones because there s not much that they can do to make it obviously better. So they try and sell it for the image.
Related posts:
  1. Long Term Adverts I ve just seen a mailing list post from someone who...
  2. Samsung Galaxy S3 First Review with Power Case My new Samsung Galaxy S3 arrived a couple of days...
  3. Back to the Xperia X10 10 months ago I was given a Samsung Galaxy S...

4 September 2015

Julien Danjou: Data validation in Python with voluptuous

Continuing my post series on the tools I use these days in Python, this time I would like to talk about a library I really like, named voluptuous. It's no secret that most of the time, when a program receives data from the outside, it's a big deal to handle it. Indeed, most of the time your program has no guarantee that the stream is valid and that it contains what is expected. The robustness principle says you should be liberal in what you accept, though that is not always a good idea neither. Whatever policy you chose, you need to process those data and implement a policy that will work lax or not. That means that the program need to look into the data received, check that it finds everything it needs, complete what might be missing (e.g. set some default), transform some data, and maybe reject those data in the end.
Data validation The first step is to validate the data, which means checking all the fields are there and all the types are right or understandable (parseable). Voluptuous provides a single interface for all that called a Schema. 
>>> from voluptuous import Schema
>>> s = Schema( 
...   'q': str,
...   'per_page': int,
...   'page': int,
...  )
>>> s( "q": "hello" )
 'q': 'hello' 
>>> s( "q": "hello", "page": "world" )
voluptuous.MultipleInvalid: expected int for dictionary value @ data['page']
>>> s( "q": "hello", "unknown": "key" )
voluptuous.MultipleInvalid: extra keys not allowed @ data['unknown']

The argument to voluptuous.Schema should be the data structure that you expect. Voluptuous accepts any kind of data structure, so it could also be a simple string or an array of dict of array of integer. You get it. Here it's a dict with a few keys that if present should be validated as certain types. By default, Voluptuous does not raise an error if some keys are missing. However, it is invalid to have extra keys in a dict by default. If you want to allow extra keys, it is possible to specify it. 
>>> from voluptuous import Schema
>>> s = Schema( "foo": str , extra=True)
>>> s( "bar": 2 )
 "bar": 2

It is also possible to make some keys mandatory. 
>>> from voluptuous import Schema, Required
>>> s = Schema( Required("foo"): str )
>>> s( )
voluptuous.MultipleInvalid: required key not provided @ data['foo']

You can create custom data type very easily. Voluptuous data types are actually just functions that are called with one argument, the value, and that should either return the value or raise an Invalid or ValueError exception. 
>>> from voluptuous import Schema, Invalid
>>> def StringWithLength5(value):
...     if isinstance(value, str) and len(value) == 5:
...             return value
...     raise Invalid("Not a string with 5 chars")
...
>>> s = Schema(StringWithLength5)
>>> s("hello")
'hello'
>>> s("hello world")
voluptuous.MultipleInvalid: Not a string with 5 chars

Most of the time though, there is no need to create your own data types. Voluptuous provides logical operators that can, combined with a few others provided primitives such as voluptuous.Length or voluptuous.Range, create a large range of validation scheme. 
>>> from voluptuous import Schema, Length, All
>>> s = Schema(All(str, Length(min=3, max=5)))
>>> s("hello")
"hello"
>>> s("hello world")
voluptuous.MultipleInvalid: length of value must be at most 5

The voluptuous documentation has a good set of examples that you can check to have a good overview of what you can do. Data transformation
What's important to remember, is that each data type that you use is a function that is called and returns a value, if the value is considered valid. That value returned is what is actually used and returned after the schema validation: 
>>> import uuid
>>> from voluptuous import Schema
>>> def UUID(value):
...     return uuid.UUID(value)
...
>>> s = Schema( "foo": UUID )
>>> data_converted = s( "foo": "uuid?" )
voluptuous.MultipleInvalid: not a valid value for dictionary value @ data['foo']
>>> data_converted = s( "foo": "8B7BA51C-DFF5-45DD-B28C-6911A2317D1D" )
>>> data_converted
 'foo': UUID('8b7ba51c-dff5-45dd-b28c-6911a2317d1d')

By defining a custom UUID function that converts a value to a UUID, the schema converts the string passed in the data to a Python UUID object validating the format at the same time. Note a little trick here: it's not possible to use directly uuid.UUID in the schema, otherwise Voluptuous would check that the data is actually an instance of uuid.UUID: 
>>> from voluptuous import Schema
>>> s = Schema( "foo": uuid.UUID )
>>> s( "foo": "8B7BA51C-DFF5-45DD-B28C-6911A2317D1D" )
voluptuous.MultipleInvalid: expected UUID for dictionary value @ data['foo']
>>> s( "foo": uuid.uuid4() )
 'foo': UUID('60b6d6c4-e719-47a7-8e2e-b4a4a30631ed')

And that's not what is wanted here. That mechanism is really neat to transform, for example, strings to timestamps. 
>>> import datetime
>>> from voluptuous import Schema
>>> def Timestamp(value):
...     return datetime.datetime.strptime(value, "%Y-%m-%dT%H:%M:%S")
...
>>> s = Schema( "foo": Timestamp )
>>> s( "foo": '2015-03-03T12:12:12' )
 'foo': datetime.datetime(2015, 3, 3, 12, 12, 12) 
>>> s( "foo": '2015-03-03T12:12' )
voluptuous.MultipleInvalid: not a valid value for dictionary value @ data['foo']

Recursive schemas So far, Voluptuous has one limitation so far: the ability to have recursive schemas. The simplest way to circumvent it is by using another function as an indirection. 
>>> from voluptuous import Schema, Any
>>> def _MySchema(value):
...     return MySchema(value)
...
>>> from voluptuous import Any
>>> MySchema = Schema( "foo": Any("bar", _MySchema) )
>>> MySchema( "foo":  "foo": "bar" )
 'foo':  'foo': 'bar' 
>>> MySchema( "foo":  "foo": "baz" )
voluptuous.MultipleInvalid: not a valid value for dictionary value @ data['foo']['foo']

Usage in REST API
I started to use Voluptuous to validate data in a the REST API provided by Gnocchi. So far it has been a really good tool, and we've been able to create a complete REST API that is very easy to validate on the server side. I would definitely recommend it for that. It blends with any Web framework easily. One of the upside compared to solution like JSON Schema, is the ability to create or re-use your own custom data types while converting values at validation time. It is also very Pythonic, and extensible it's pretty great to use for all of that. It's also not tied to any serialization format. On the other hand, JSON Schema is language agnostic and is serializable itself as JSON. That makes it easy to be exported and provided to a consumer so it can understand the API and validate the data potentially on its side.

Dirk Eddelbuettel: RcppArmadillo 0.5.500.2.0

armadillo image Once again time for the monthly upstream Armadillo update -- version 5.500.2 was released earlier today by Conrad. And a new and matching RcppArmadillo release 0.5.500.2.0 in already on CRAN and will go to Debian shortly. Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. This release contains mostly bug fixes and some internal code refactoring:
Changes in RcppArmadillo version 0.5.500.2.0 (2015-09-03)
  • Upgraded to Armadillo 5.500.2 ("Molotov Cocktail")
    • expanded object constructors and generators to handle size() based specification of dimensions
    • faster handling of submatrix rows
    • faster clamp()
    • fixes for handling sparse matrices
Courtesy of CRANberries, there is also a diffstat report for the most recent CRAN release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

3 September 2015

John Goerzen: There s still a chance to save WiFi

You may not know it, but wifi is under assault in the USA due to proposed FCC regulations about modifications to devices with modular radios. In short, it would make it illegal for vendors to sell devices with firmware that users can replace. This is of concern to everyone, because Wifi routers are notoriously buggy and insecure. It is also of special concern to amateur radio hobbyists, due to the use of these devices in the Amateur Radio Service (FCC Part 97). I submitted a comment to the FCC about this, which I am pasting in here. This provides a background and summary of the issues for those that are interested. Here it is: My comment has two parts: one, the impact on the Amateur Radio service; and two, the impact on security. Both pertain primarily to the 802.11 ( Wifi ) services typically operating under Part 15. The Amateur Radio Service (FCC part 97) has long been recognized by the FCC and Congress as important to the nation. Through it, amateurs contribute to scientific knowledge, learn skills that bolster the technological competitiveness of the United States, and save lives through their extensive involvement in disaster response. Certain segments of the 2.4GHz and 5GHz Wifi bands authorized under FCC Part 15 also fall under the frequencies available to licensed amateurs under FCC Part 97 [1]. By scrupulously following the Part 97 regulations, many amateur radio operators are taking devices originally designed for Part 15 use and modifying them for legal use under the Part 97 Amateur Radio Service. Although the uses are varied, much effort is being devoted to fault-tolerant mesh networks that provide high-speed multimedia communications in response to a disaster, even without the presence of any traditional infrastructure or Internet backbone. One such effort [2] causes users to replace the firmware on off-the-shelf Part 15 Wifi devices, reconfiguring them for proper authorized use under Part 97. This project has many vital properties, particularly the self-discovery of routes between nodes and self-healing nature of the mesh network. These features are not typically available in the firmware of normal Part 15 devices. It should also be noted that there is presently no vendor of Wifi devices that operate under Part 97 out of the box. The only route available to amateurs is to take Part 15 devices and modify them for Part 97 use. Amateur radio users of these services have been working for years to make sure they do not cause interference to Part 15 users [3]. One such effort takes advantage of the modular radio features of consumer Wifi gear to enable communication on frequencies that are within the Part 97 allocation, but outside (though adjacent) to the Part 15 allocation. For instance, the chart at [1] identifies frequencies such as 2.397GHz or 5.660GHz that will never cause interference to Part 15 users because they lie entirely outside the Part 15 Wifi allocation. If the FCC prevents the ability of consumers to modify the firmware of these devices, the following negative consequences will necessarily follow: 1) The use of high-speed multimedia or mesh networks in the Amateur Radio service will be sharply curtailed, relegated to only outdated hardware. 2) Interference between the Amateur Radio service which may use higher power or antennas with higher gain and Part 15 users will be expanded, because Amateur Radio service users will no longer be able to intentionally select frequencies that avoid Part 15. 3) The culture of inventiveness surrounding wireless communication will be curtailed in an important way. Besides the impact on the Part 97 Amateur Radio Service, I also wish to comment on the impact to end-user security. There have been a terrible slew of high-profile situations where very popular consumer Wifi devices have had incredible security holes. Vendors have often been unwilling to address these issues [4]. Michael Horowitz maintains a website tracking security bugs in consumer wifi routers [5]. Sadly these bugs are both severe and commonplace. Within just the last month, various popular routers have been found vulnerable to remote hacking [6] and platforms for launching Distributed Denial-of-Service (DDoS) attacks [7]. These impacted multiple models from multiple vendors. To make matters worse, most of these issues should have never happened in the first place, and were largely the result of carelessness or cluelessness on the part of manufacturers. Consumers should not be at the mercy of vendors to fix their own networks, nor should they be required to trust unauditable systems. There are many well-regarded efforts to provide better firmware for Wifi devices, which still keep them operating under Part 15 restrictions. One is OpenWRT [8], which supports a wide variety of devices with a system built upon a solid Linux base. Please keep control of our devices in the hands of consumers and amateurs, for the benefit of all. References: [1] https://en.wikipedia.org/wiki/High-speed_multimedia_radio [2] http://www.broadband-hamnet.org/just-starting-read-this.html [3] http://www.qsl.net/kb9mwr/projects/wireless/modify.html [4] http://www.tomsitpro.com/articles/netgear-routers-security-hole,1-2461.html [5] http://routersecurity.org/bugs.php [6] https://www.kb.cert.org/vuls/id/950576 [7] https://www.stateoftheinternet.com/resources-cloud-security-2015-q2-web-security-report.html [8] https://openwrt.org/

Petter Reinholdtsen: Book cover for the Free Culture book finally done

Creating a good looking book cover proved harder than I expected. I wanted to create a cover looking similar to the original cover of the Free Culture book we are translating to Norwegian, and I wanted it in vector format for high resolution printing. But my inkscape knowledge were not nearly good enough to pull that off. But thanks to the great inkscape community, I was able to wrap up the cover yesterday evening. I asked on the #inkscape IRC channel on Freenode for help and clues, and Marc Jeanmougin (Mc-) volunteered to try to recreate it based on the PDF of the cover from the HTML version. Not only did he create a SVG document with the original and his vector version side by side, he even provided an instruction video explaining how he did it. But the instruction video is not easy to follow for an untrained inkscape user. The video is a recording on how he did it, and he is obviously very experienced as the menu selections are very quick and he mentioned on IRC that he did use some keyboard shortcuts that can't be seen on the video, but it give a good idea about the inkscape operations to use to create the stripes with the embossed copyright sign in the center. I took his SVG file, copied the vector image and re-sized it to fit on the cover I was drawing. I am happy with the end result, and the current english version look like this: I am not quite sure about the text on the back, but guess it will do. I picked three quotes from the official site for the book, and hope it will work to trigger the interest of potential readers. The Norwegian cover will look the same, but with the texts and bar code replaced with the Norwegian version. The book is very close to being ready for publication, and I expect to upload the final draft to Lulu in the next few days and order a final proof reading copy to verify that everything look like it should before allowing everyone to order their own copy of Free Culture, in English or Norwegian Bokm l. I'm waiting to give the the productive proof readers a chance to complete their work.

2 September 2015

Luca Falavigna: Resource control with systemd

I m receiving more requests for upload accounts to the Deb-o-Matic servers lately (yay!), but that means the resources need to be monitored and shared between the build daemons to prevent server lockups. My servers are running systemd, so I decided to give systemd.resource-control a try. My goal was to assign lower CPU shares to the build processes (debomatic itself, sbuild, and all the related tools), in order to avoid blocking other important system services from being spawned when necessary. I created a new slice, and set a lower CPU share weight:
$ cat /etc/systemd/system/debomatic.slice
[Slice]
CPUAccounting=true
CPUShares=512
$
 Then, I assigned the slice to the service unit file controlling debomatic daemons by adding the Slice=debomatic.slice option under the Service directive. That was not enough, though, as some processes were assigned to the user slice instead, which groups all the processes spawned by users:
systemd-cgls This is probably because schroot spawns a login shell, and systemd considers it belonging to a different process group. So, I had to launch the command systemctl set-property user.slice CPUShares=512, so all processes belonging to the user.slice will receive the same share of the debomatic ones. I consider this a workaround, I m open to suggestions how to properly solve this issue :) I ll try to explore more options in the coming days, so I can improve my knowledge of systemd a little bit more :)

1 September 2015

Enrico Zini: if-you-know-a-browser-developer

If you happen to know a browser developer... Do you happen to know a developer of Firefox or Chrome or some other mainstream browser? If so, can you please talk to them about our experiments with Client Certificate authentication in Debian? Client Certificate authentication rocks; with just a couple of little tweaks in the interface, it would be pretty close to perfect. Visiting sites without using a certificate If I want to browse a site unauthenticated instead of using a certificate, at the moment I can hit "Cancel" on the certificate popup menu, and it works nicely. I feel quite confused when I do that, though, because it's not clear to me if I am canceling use of certificates, or canceling the visit to the site. Can you please change the wording on the Cancel button to something more descriptive? See/change current certificate selection My top wish is, once I choise to use (or not use) a certificate for a site, to be able to see which certificate I'm using and possibly change it. At the moment I did not find a way to see what certificate I'm using, and the browser will remember the choice until it gets closed and reopened. At the moment I can use a Private or Incognito window to switch identities or to stop authenticated access and continue anonymously, and that helps me immensely. I think however that the ultimate solution could be to have the https lockpad popup show an indication of what certificate is currently being used, and offer a way to re-trigger certificate selection. That would be so cool. Also, once the certificate choice can be seen and changed at any time, it could just get remembered so that sites can be visited again without any prompts, even after the browser has been closed and reopened. That would be, to me, the ultimate convenience. Thanks! <3 Thank you very much for all the work you have already put into this: I have been told that a few years ago using client certificate was unthinkable, and now it seems to be down to just a couple of papercuts. And SPKAC/keygen seriously rocks! I have been constantly impressed by how well this all works right now.

Lunar: Reproducible builds: week 18 in Stretch cycle

What happened in the reproducible builds effort this week: Toolchain fixes Aur lien Jarno uploaded glibc/2.21-0experimental1 which will fix the issue were locales-all did not behave exactly like locales despite having it in the Provides field. Lunar rebased the pu/reproducible_builds branch for dpkg on top of the released 1.18.2. This made visible an issue with udebs and automatically generated debug packages. The summary from the meeting at DebConf15 between ftpmasters, dpkg mainatainers and reproducible builds folks has been posted to the revelant mailing lists. Packages fixed The following 70 packages became reproducible due to changes in their build dependencies: activemq-activeio, async-http-client, classworlds, clirr, compress-lzf, dbus-c++, felix-bundlerepository, felix-framework, felix-gogo-command, felix-gogo-runtime, felix-gogo-shell, felix-main, felix-shell-tui, felix-shell, findbugs-bcel, gco, gdebi, gecode, geronimo-ejb-3.2-spec, git-repair, gmetric4j, gs-collections, hawtbuf, hawtdispatch, jack-tools, jackson-dataformat-cbor, jackson-dataformat-yaml, jackson-module-jaxb-annotations, jmxetric, json-simple, kryo-serializers, lhapdf, libccrtp, libclaw, libcommoncpp2, libftdi1, libjboss-marshalling-java, libmimic, libphysfs, libxstream-java, limereg, maven-debian-helper, maven-filtering, maven-invoker, mochiweb, mongo-java-driver, mqtt-client, netty-3.9, openhft-chronicle-queue, openhft-compiler, openhft-lang, pavucontrol, plexus-ant-factory, plexus-archiver, plexus-bsh-factory, plexus-cdc, plexus-classworlds2, plexus-component-metadata, plexus-container-default, plexus-io, pytone, scolasync, sisu-ioc, snappy-java, spatial4j-0.4, tika, treeline, wss4j, xtalk, zshdb. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: Chris Lamb also noticed that binaries shipped with libsilo-bin did not work. Documentation update Chris Lamb and Ximin Luo assembled a proper specification for SOURCE_DATE_EPOCH in the hope to convince more upstreams to adopt it. Thanks to Holger it is published under a non-Debian domain name. Lunar documented easiest way to solve issues with file ordering and timestamps in tarballs that came with tar/1.28-1. Some examples on how to use SOURCE_DATE_EPOCH have been improved to support systems without GNU date. reproducible.debian.net armhf is finally being tested, which also means the remote building of Debian packages finally works! This paves the way to perform the tests on even more architectures and doing variations on CPU and date. Some packages even produce the same binary Arch:all packages on different architectures (1, 2). (h01ger) Tests for FreeBSD are finally running. (h01ger) As it seems the gcc5 transition has cooled off, we schedule sid more often than testing again on amd64. (h01ger) disorderfs has been built and installed on all build nodes (amd64 and armhf). One issue related to permissions for root and unpriviliged users needs to be solved before disorderfs can be used on reproducible.debian.net. (h01ger) strip-nondeterminism Version 0.011-1 has been released on August 29th. The new version updates dh_strip_nondeterminism to match recent changes in debhelper. (Andrew Ayer) disorderfs disorderfs, the new FUSE filesystem to ease testing of filesystem-related variations, is now almost ready to be used. Version 0.2.0 adds support for extended attributes. Since then Andrew Ayer also added support to reverse directory entries instead of shuffling them, and arbitrary padding to the number of blocks used by files. Package reviews 142 reviews have been removed, 48 added and 259 updated this week. Santiago Vila renamed the not_using_dh_builddeb issue into varying_mtimes_in_data_tar_gz_or_control_tar_gz to align better with other tag names. New issue identified this week: random_order_in_python_doit_completion. 37 FTBFS issues have been reported by Chris West (Faux) and Chris Lamb. Misc. h01ger gave a talk at FrOSCon on August 23rd. Recordings are already online. These reports are being reviewed and enhanced every week by many people hanging out on #debian-reproducible. Huge thanks!

Rapha&#235;l Hertzog: My Free Software Activities in August 2015

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I have been paid to work 6.5 hours on Debian LTS. In that time I did the following: Apart from that, I also gave a talk about Debian LTS at DebConf 15 in Heidelberg and also coordinated a work session to discuss our plans for Wheezy. Have a look at the video recordings: DebConf 15 I attended DebConf 15 with great pleasure after having missed DebConf 14 last year. While I did not do lots of work there, I participated in many discussions and I certainly came back with a renewed motivation to work on Debian. That s always good. :-) For the concrete work I did during DebConf, I can only claim two schroot uploads to fix the lack of support of the new overlay filesystem that replaces aufs in the official Debian kernel, and some Distro Tracker work (fixing an issue that some people had when they were logged in via Debian s SSO). While the numerous discussions I had during DebConf can t be qualified as work , they certainly contribute to build up work plans for the future: As a Kali developer, I attended multiple sessions related to derivatives (notably the Debian Derivatives Panel). I was also interested by the Debian in the corporate IT BoF led by Michael Meskes (Credativ s CEO). He pointed out a number of problems that corporate users might have when they first consider using Debian and we will try to do something about this. Expect further news and discussions on the topic. Martin Kraff, Luca Filipozzi, and me had a discussion with the Debian Project Leader (Neil) about how to revive/transform the Debian s Partner program. Nothing is fleshed out yet, but at least the process initiated by the former DPL (Lucas) is again moving forward. Other Debian work Sponsorship. I sponsored an NMU of pep8 by Daniel Stender as it was a requirement for prospector which I also sponsored since all the required dependencies are now available in Debian. \o/ Packaging. I NMUed libxml2 2.9.2+really2.9.1+dfsg1-0.1 fixing 3 security issues and a RC bug that was breaking publican. Since there s no upstream fix for more than 8 months, I went back to the former version 2.9.1. It s in line with the new requirement of release managers a package in unstable should migrate to testing reasonably quickly, it s not acceptable to keep it unfixed for months. With this annoying bug fixed, I could again upload a new upstream release of publican so I prepared and uploaded 4.3.2-1. It was my first source only upload. This release was more work than I expected and I filed no less than 3 bug to upstream (new bash-completion install path, request to provide sources of a minified javascript file, drop a .po file for an invalid language code). GPG issues with smartcard. Back from DebConf, when I wanted to sign some key, I stumbled again upon the problem which makes it impossible for me to use my two smartcards one after the other without first deleting the stubs for the private key. It s not a new issue but I decided that it was time to report it upstream, so I did it: #2079 on bugs.gnupg.org. Some research helped me to find a way to work-around the problem. Later in the month, after a dist-upgrade and a reboot, I was no longer able to use my smartcard as a SSH authentication key again it was already reported but there was no clear analysis, so I tried to do my own one and added the results of my investigation in #795368. It looks like the culprit is pinentry-gnome3 not working when started by the gpg-agent which is started before the DBUS session. Simple fix is to restart the gpg-agent in the session but I have no idea yet of what the proper fix should be (letting systemd manage the graphical user session and start gpg-agent would be my first answer, but that doesn t solve the issue for users of other init systems so it s not satisfying). Distro Tracker. I merged two patches from Orestis Ioannou fixing some bugs tagged newcomer. There are more such bugs (I even filed two: #797096 and #797223), go grab them and do a first contribution to Distro Tracker like Orestis just did! I also merged a change from Christophe Siraut who presented Distro Tracker at DebConf. I implemented in Distro Tracker the new authentication based on SSL client certificates that was recently announced by Enrico Zini. It s working nice, and this authentication scheme is far easier to support. Good job, Enrico! tracker.debian.org broke during DebConf, it stopped being updated with new data. I tracked this down to a problem in the archive (see #796892). Apparently Ansgar Burchardt changed the set of compression tools used on some jessie repositorie, replacing bz2 by xz. He dropped the old Packages.bz2 but missed some Sources.bz2 which were thus stale and APT reported Hashsum mismatch on the uncompressed content. Misc. I pushed some small improvement to my Salt formulas: schroot-formula and sbuild-formula. They will now auto-detect which overlay filesystem is available with the current kernel (previously aufs was hardcoded). Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Russ Allbery: Review: Kanban

Review: Kanban, by David J. Anderson
Publisher: Blue Hole
Copyright: 2010
ISBN: 0-9845214-0-2
Format: Trade paperback
Pages: 240
Another belated review, this time of a borrowed book. Which I can now finally return! A delay in the review of this book might be a feature if I had actually used it for, as the subtitle puts it, successful evolutionary change in my technology business. Sadly, I haven't, so it's just late. So, my background: I've done a lot of variations of traditional project management for IT projects (both development and more operational ones), both as a participant and as a project manager. (Although I've never done the latter as a full-time job, and have no desire to do so.) A while back at Stanford, my team adopted Agile, specifically Scrum, so I did a fair bit of research about Scrum including a couple of training courses. Since then, at Dropbox, I've used a few different variations on a vaguely Agile-inspired planning process, although it's not particularly consistent with any one system. I've been hearing about Kanban for a while and have friends who swear by it, but I only had a vague idea of how it worked. That seemed like a good excuse to read a book. And Anderson's book is a good one if, like me, you're looking for a basic introduction. It opens with a basic description and definition, talks about motivation and the expected benefits, and then provides a detailed description of Kanban as a system. The tone is on the theory side, written using the terminology of (I presume) management theory and operations management, areas about which I know almost nothing, but the terminology wasn't so heavy as to make the book hard to read. Anderson goes into lots of detail, and I thought he did a good job of distinguishing between basic principles, optional techniques, and variations that may be appropriate for particular environments. If you're also not familiar, the basic concept of Kanban is to organize work using an in-progress queue. It eschews time-bounded planning entirely in favor of staging work at the start of a sequence of queues and letting the people working on each queue pull from the previous queue when they're ready to take on a new task. As you might guess from that layout, Kanban was originally invented for assembly-line manufacturing (at Toyota). That was one of the problems that I had with it, or at least the presentation in this book: most of my software development doesn't involve finishing one part of something and handing it off to someone else, which made it hard to identify with the pipeline model. Anderson has clearly spent a lot of time working with large-scale programming shops, including outsourced development firms, with dedicated QA and operations roles. This is not at all how Silicon Valley agile development works, so parts of this book felt like missives from a foreign country. That said, the key point of Kanban is not the assembly line but the work limit. One of the defining characteristics of Kanban, at least as Anderson presents it, is that one does not try to estimate work and tile it based on estimates, not even to the extent that Scrum and other Agile methodologies do within the sprint. Instead, each person takes as long as they take on the things they're working on, and pulls a new task when they have free capacity. The system instead puts a limit on how many things they can have in progress at a time. The problem of pipeline stalls is dealt with both via continuous improvement and "swarming" of a problem to unblock the line (since other teams may not be able to work until the block is fixed), and with being careful about the sizing of work items (I'm used to calling them stories) that go in the front end. Predictability, which Scrum uses story sizing and team velocity analysis to try to achieve, is basically statistical. One uses a small number of buckets of sizes of stories, and the whole pipeline will finish some number of work items per unit time, with a variance. The promise made to clients and other teams is that some percentage of the work items will be finished within some time frame from when they enter the system. Most importantly, these are all descriptive properties, determined statistically after the fact, rather than planned properties worked out through story sizing and extensive team discussion. If you, like me, are pretty thoroughly sick of two-hour sprint planning meetings and endless sizing exercises, this is rather appealing. My problem with most work planning systems is that I think they overplan and put too much weight our ability to estimate how long work will take. Kanban is very appealing when viewed through that lens: it gives up on things we're bad at in favor of simple measurement and building a system with enough slack that it can handle work of various different sizes. As mentioned at the top, I haven't had a chance to try it (and I'm not sure it's a good fit with the inter-group planning methods in use at my current workplace), but I came away from this book wanting to do so. If, like me, your experience is mostly with small combined teams or individual programming work, Anderson's examples may seem rather large, rather formal, and bureaucratic. But this is a solid introduction and well worth reading if your only experience with Agile is sprint planning, story writing and sizing, and fast iterations. Rating: 7 out of 10

31 August 2015

Matthew Garrett: Working with the kernel keyring

The Linux kernel keyring is effectively a mechanism to allow shoving blobs of data into the kernel and then setting access controls on them. It's convenient for a couple of reasons: the first is that these blobs are available to the kernel itself (so it can use them for things like NFSv4 authentication or module signing keys), and the second is that once they're locked down there's no way for even root to modify them.

But there's a corner case that can be somewhat confusing here, and it's one that I managed to crash into multiple times when I was implementing some code that works with this. Keys can be "possessed" by a process, and have permissions that are granted to the possessor orthogonally to any permissions granted to the user or group that owns the key. This is important because it allows for the creation of keyrings that are only visible to specific processes - if my userspace keyring manager is using the kernel keyring as a backing store for decrypted material, I don't want any arbitrary process running as me to be able to obtain those keys[1]. As described in keyrings(7), keyrings exist at the session, process and thread levels of granularity.

This is absolutely fine in the normal case, but gets confusing when you start using sudo. sudo by default doesn't create a new login session - when you're working with sudo, you're still working with key posession that's tied to the original user. This makes sense when you consider that you often want applications you run with sudo to have access to the keys that you own, but it becomes a pain when you're trying to work with keys that need to be accessible to a user no matter whether that user owns the login session or not.

I spent a while talking to David Howells about this and he explained the easiest way to handle this. If you do something like the following:
$ sudo keyctl add user testkey testdata @u
a new key will be created and added to UID 0's user keyring (indicated by @u). This is possible because the keyring defaults to 0x3f3f0000 permissions, giving both the possessor and the user read/write access to the keyring. But if you then try to do something like:
$ sudo keyctl setperm 678913344 0x3f3f0000
where 678913344 is the ID of the key we created in the previous command, you'll get permission denied. This is because the default permissions on a key are 0x3f010000, meaning that the possessor has permission to do anything to the key but the user only has permission to view its attributes. The cause of this confusion is that although we have permission to write to UID 0's keyring (because the permissions are 0x3f3f0000), we don't possess it - the only permissions we have for this key are the user ones, and the default state for user permissions on new keys only gives us permission to view the attributes, not change them.

But! There's a way around this. If we instead do:
$ sudo keyctl add user testkey testdata @s
then the key is added to the current session keyring (@s). Because the session keyring belongs to us, we possess any keys within it and so we have permission to modify the permissions further. We can then do:
$ sudo keyctl setperm 678913344 0x3f3f0000
and it works. Hurrah! Except that if we log in as root, we'll be part of another session and won't be able to see that key. Boo. So, after setting the permissions, we should:
$ sudo keyctl link 678913344 @u
which ties it to UID 0's user keyring. Someone who logs in as root will then be able to see the key, as will any processes running as root via sudo. But we probably also want to remove it from the unprivileged user's session keyring, because that's readable/writable by the unprivileged user - they'd be able to revoke the key from underneath us!
$ sudo keyctl unlink 678913344 @s
will achieve this, and now the key is configured appropriately - UID 0 can read, modify and delete the key, other users can't.

This is part of our ongoing work at CoreOS to make rkt more secure. Moving the signing keys into the kernel is the first step towards rkt no longer having to trust the local writable filesystem[2]. Once keys have been enrolled the keyring can be locked down - rkt will then refuse to run any images unless they're signed with one of these keys, and even root will be unable to alter them.

[1] (obviously it should also be impossible to ptrace() my userspace keyring manager)
[2] Part of our Secure Boot work has been the integration of dm-verity into CoreOS. Once deployed this will mean that the /usr partition is cryptographically verified by the kernel at runtime, making it impossible for anybody to modify it underneath the kernel. / remains writable in order to permit local configuration and to act as a data store, and right now rkt stores its trusted keys there.

comment count unavailable comments

Russ Allbery: Review: Through Struggle, The Stars

Review: Through Struggle, The Stars, by John J. Lumpkin
Series: Human Reach #1
Publisher: John J. Lumpkin
Copyright: July 2011
ISBN: 1-4611-9544-6
Format: Kindle
Pages: 429
Never let it be said that I don't read military SF. However, it can be said that I read books and then get hellishly busy and don't review them for months. So we'll see if I can remember this well enough to review it properly. In Lumpkin's future world, mankind has spread to the stars using gate technology, colonizing multiple worlds. However, unlike most science fiction futures of this type, it's not all about the United States, or even the United States and Russia. The great human powers are China and Japan, with the United States relegated to a distant third. The US mostly maintains its independence from either, and joins the other lesser nations and UN coalitions to try to pick up a few colonies of its own. That's the context in which Neil and Rand join the armed services: the former as a pilot in training, and the latter as an army grunt. This is military SF, so of course a war breaks out. But it's a war between Japan and China: improved starship technology and the most sophisticated manufacturing in the world against a much larger economy with more resources and a larger starting military. For reasons that are not immediately clear, and become a plot point later on, the United States president immediately takes an aggressive tone towards China and pushes the country towards joining the war on the side of Japan. Most of this book is told from Neil's perspective, following his military career during the outbreak of war. His plans to become a pilot get derailed as he gets entangled with US intelligence agents (and a bad supervisor). The US forces are not in a good place against China, struggling when they get into ship-to-ship combat, and Neil's ship goes on a covert mission to attempt to complicate the war with political factors. Meanwhile, Rand tries to help fight off a Chinese invasion of one of the rare US colony worlds. Through Struggle, The Stars does not move quickly. It's over 400 pages, and it's a bit surprising how little happens. What it does instead is focus on how the military world and the war feels to Neil: the psychological experience of wanting to serve your country but questioning its decisions, the struggle of working with people who aren't always competent but who you can't just make go away, the complexities of choosing a career path when all the choices are fraught with politics that you didn't expect to be involved in, and the sheer amount of luck and random events involved in the progression of one's career. I found this surprisingly engrossing despite the slow pace, mostly because of how true to life it feels. War is not a never-ending set of battles. Life in a military ship has moments when everything is happening far too quickly, but other moments when not much happens for a long time. Lumpkin does a great job of reflecting that. Unfortunately, I thought there were two significant flaws, one of which means I probably won't seek out further books in this series. First, one middle portion of the book switches away from Neil to follow Rand instead. The first part of that involves the details of fighting orbiting ships with ground-based lasers, which was moderately interesting. (All the technological details of space combat are interesting and thoughtfully handled, although I'm not the sort of reader who will notice more subtle flaws. But David Weber this isn't, thankfully.) But then it turns into a fairly generic armed resistance story, which I found rather boring. It also ties into the second and more serious flaw: the villain. The overall story is constructed such that it doesn't need a personal villain. It's about the intersection of the military and politics, and a war that may be ill-conceived but that is being fought anyway. That's plenty of conflict for the story, at least in my opinion. But Lumpkin chose to introduce a specific, named Chinese character in the villain role, and the characterization is... well. After he's humiliated early in the story by the protagonists, Li Xiao develops an obsession with killing them, for his honor, and then pursues them throughout the book in ways that are sometimes destructive to the Chinese war efforts. It's badly unrealistic compared to the tone of realism taken by the rest of the story. Even if someone did become this deranged, it's bizarre that a professional military (and China's forces are otherwise portrayed as fairly professional) would allow this. Li reads like pure caricature, and despite being moderately competent apart from his inexplicable (but constantly repeated) obsession, is cast in a mustache-twirling role of personal villainy. This is weirdly out of place in the novel, and started irritating me enough that it took me out of the story. Through Struggle, The Stars is the first book of a series, and does not resolve much by the end of the novel. That plus its length makes the story somewhat unsatisfying. I liked Neil's development, and liked him as a character, and those who like the details of combat mixed with front-lines speculation about politics may enjoy this. But a badly-simplified mustache-twirling victim and some extended, uninteresting bits mar the book enough that I probably won't seek out the sequels. Followed by The Desert of Stars. Rating: 6 out of 10

30 August 2015

Andrew Cater: Rescuing a Windows 10 failed install using GParted Live on CD

Windows 10 is here, for better or worse. As the family sysadmin, I've been tasked to update the Windows machines: ultimately, failure modes are not well documented and I needed Free software and several hours to recover a vital machine.

The "free upgrade for users of prior Windows versions" is a limited time offer for a year from launch. Microsoft do not offer licence keys for the upgrade: once a machine has updated to Windows 10 and authenticated on the 'Net, then a machine can be re-installed and will be regarded by Microsoft as pre-authorised. Users don't get the key at any point.

Although Microsoft have pushed the fact that this can be done through Windows Update, there is the option to use Microsoft's Media Creation tool to do the upgrade directly on the Windows machine concerned. This would be necessary to get the machine to upgrade and register before a full clean install of Windows 10 from media.

This Media Creation Tool failed for me on three machines with "Unable to access System reserved partition'

All the machines have SSDs from various makers: a colleague suggested that resizing the partition might enable the upgrade to continue. Of the machines that failed, all were running Windows 7 - two were running using BIOS, one using UEFI boot on a machine with no Legacy/CSM mode.

Using GParted live .iso - itself based on Debian Live - allowed me to resize the System partition from 100MiB to 200MiB by moving the Windows partition but Windows became unbootable.

In two cases, I was able to boot from DVD Windows installation media and make Windows bootable again at which point the Microsoft Media Creation Tool could be used to install Windows 10

The UEFI boot machine proved more problematic: I had to create a Windows 7 System Repair disk and repair Windows booting before Windows 10 could proceed.

My Windows-using colleaague had used only Windows-based recovery disks: using Linux tools allowed me to repair Windows installations I couldn't boot

DebConf team: DebConf15: Farewell, and thanks for all the Fisch (Posted by DebConf Team)

A week ago, we concluded our biggest DebConf ever! It was a huge success. Handwritten feedback note We are overwhelmed by the positive feedback, for which we re very grateful. We want to thank you all for participating in the talks; speakers and audience alike, in person or live over the global Internet it wouldn t be the fantastic DebConf experience without you! Many of our events were recorded and streamed live, and are now available for viewing, as are the slides and photos. To share a sense of the scale of what all of us accomplished together, we ve compiled a few statistics: Our very own designer Valessio Brito made a lovely video of impressions and images of the conference.
We re collecting impressions from attendees as well as links to press articles, including Linux Weekly News coverage of specific sessions of DebConf. If you find something not yet included, please help us by adding links to the wiki.
DebConf15 group photo (by Aigars Mahinovs)
We tried a few new ideas this year, including a larger number of invited and featured speakers than ever before. On the Open Weekend, some of our sponsors presented their career opportunities at our job fair, which was very well attended. And a diverse selection of entertainment options provided the necessary breaks and ample opportunity for socialising. On the last Friday, the Oscar-winning documentary Citizenfour was screened, with some introductory remarks by Jacob Appelbaum and a remote address by its director, Laura Poitras, and followed by a long Q&A session by Jacob. DebConf15 was also the first DebConf with organised childcare (including a Teckids workshop for kids of age 8-16), which our DPL Neil McGovern standardised for the future: it s a thing now, he said. The participants used the week before the conference for intensive work, sprints and workshops, and throughout the main conference, significant progress was made on Debian and Free Software. Possibly the most visible was the endeavour to provide reproducible builds, but the planning of the next stable release stretch received no less attention. Groups like the Perl team, the diversity outreach programme and even DebConf organisation spent much time together discussing next steps and goals, and hundreds of commits were made to the archive, as well as bugs closed. DebConf15 was an amazing conference, it brought together hundreds of people, some oldtimers as well as plenty of new contributors, and we all had a great time, learning and collaborating with each other, says Margarita Manterola of the organiser team, and continues: The whole team worked really hard, and we are all very satisfied with the outcome. Another organiser, Martin Krafft adds: We mainly provided the infrastructure and space. A lot of what happened during the two weeks was thanks to our attendees. And that s what makes DebConf be DebConf. Photo of hostel staff wearing DebConf15 staff t-shirts (by Martin Krafft) Our organisation was greatly supported by the staff of the conference venue, the Jugendherberge Heidelberg International, who didn t take very long to identify with our diverse group, and who left no wishes untried. The venue itself was wonderfully spacious and never seemed too full as people spread naturally across the various conference rooms, the many open areas, the beergarden, the outside hacklabs and the lawn. The network installed specifically for our conference in collaboration with the nearby university, the neighbouring zoo, and the youth hostel provided us with a 1 Gbps upstream link, which we managed to almost saturate. The connection will stay in place, leaving the youth hostel as one with possibly the fastest Internet connection in the state. And the kitchen catered high-quality food to all attendees and their special requirements. Regional beer and wine, as well as local specialities, were provided at the bistro. DebConf exists to bring people together, which includes paying for travel, food and accomodation for people who could not otherwise attend. We would never have been able to achieve what we did without the support of our generous sponsors, especially our Platinum Sponsor Hewlett-Packard. Thank you very much. See you next year in Cape Town, South Africa!
The DebConf16 logo with white background

Philipp Kern: Automating the 3270 part of a Debian System z install

If you try to install Debian on System z within z/VM you might be annoyed at the various prompts it shows before it lets you access the network console via SSH. We can do better. From within CMS copy the default EXEC and default PARMFILE:

COPYFILE DEBIAN EXEC A DEBAUTO EXEC A
COPYFILE PARMFILE DEBIAN A PARMFILE DEBAUTO A

Now edit DEBAUTO EXEC A and replace the DEBIAN in 'PUNCH PARMFILE DEBIAN * (NOHEADER' with DEBAUTO. This will load the alternate kernel parameters file into the card reader, while still loading the original kernel and initrd files.

Replace PARMFILE DEBAUTO A's content with this (note the 80 character column limit):

ro locale=C
s390-netdevice/choose_networktype=qeth s390-netdevice/qeth/layer2=true
s390-netdevice/qeth/choose=0.0.fffc-0.0.fffd-0.0.fffe
netcfg/get_ipaddress=<IPADDR> netcfg/get_netmask=255.255.255.0
netcfg/get_gateway=<GW> netcfg/get_nameservers=<FIRST-DNS>
netcfg/confirm_static=true netcfg/get_hostname=debian
netcfg/get_domain=
network-console/authorized_keys_url=http://www.kern.lc/id_rsa.pub
preseed/url=http://www.kern.lc/preseed-s390x.cfg

Replace <IPADDR>, <GW>, and <FIRST-DNS> to suit your local network config. You might also need to change the netmask, which I left in for clarity about the format. Adjust the device address of your OSA network card. If it's in layer 3 mode (very likely) you should set layer2=false. Note that mixed case matters, hence you will want to SET CASE MIXED in xedit.

Then there are the two URLs that need to be changed. The authorized_keys_url file contains your SSH public key and is fetched unencrypted and unauthenticated, so be careful what networks you traverse with your request (HTTPS is not supported by debian-installer in Debian).

 preseed/url is needed for installation parameters that do not fit the parameters file - there is an upper character limit that's about two lines longer than my example. This is why this example only contains the bare minimum for the network part, everything else goes into this preseeding file. It file can optionally be protected with a MD5 checksum in preseed/url/checksum.

Both URLs need to be very short. I thought that there was a way to specify a line continuation, but in my tests I was unable to produce one. Hence it needs to fit on one line, including the key. You might want to use an IPv4 as the hostname.

To skip the initial boilerplate prompts and to skip straight to the user and disk setup you can use this as preseed.cfg:

d-i debian-installer/locale string en_US
d-i debian-installer/country string US
d-i debian-installer/language string en
d-i time/zone US/Eastern
d-i mirror/country manual
d-i mirror/http/mirror string httpredir.debian.org
d-i mirror/http/directory string /debian
d-i mirror/http/proxy string

I'm relatively certain that the DASD disk setup part cannot be automated yet. But the other bits of the installation should be preseedable just like on non-mainframe hardware.

Dirk Eddelbuettel: RcppGSL 0.3.0

A new version of RcppGSL just arrived on CRAN. The RcppGSL package provides an interface from R to the GNU GSL using our Rcpp package. Following on the heels of an update last month we updated the package (and its vignette) further. One of the key additions concern memory management: Given that our proxy classes around the GSL vector and matrix types are real C++ object, we can monitor their scope and automagically call free() on them rather then insisting on the user doing it. This renders code much simpler as illustrated below. Dan Dillon added const correctness over a series on pull request which allows us to write more standard (and simply nicer) function interfaces. Lastly, a few new typedef declaration further simply the use of the (most common) double and int vectors and matrices. Maybe a code example will help. RcppGSL contains a full and complete example package illustrating how to write a package using the RcppGSL facilities. It contains an example of computing a column norm -- which we blogged about before when announcing an much earlier version. In its full glory, it looks like this: 
#include <RcppGSL.h>
#include <gsl/gsl_matrix.h>
#include <gsl/gsl_blas.h>
extern "C" SEXP colNorm(SEXP sM)  
  try  
        RcppGSL::matrix<double> M = sM;     // create gsl data structures from SEXP
        int k = M.ncol();
        Rcpp::NumericVector n(k);           // to store results
        for (int j = 0; j < k; j++)  
            RcppGSL::vector_view<double> colview = gsl_matrix_column (M, j);
            n[j] = gsl_blas_dnrm2(colview);
         
        M.free() ;
        return n;                           // return vector
    catch( std::exception &ex )  
        forward_exception_to_r( ex );
    catch(...)  
        ::Rf_error( "c++ exception (unknown reason)" );
   
  return R_NilValue; // -Wall
We manually translate the SEXP coming from R, manually cover the try and catch exception handling, manually free the memory etc pp. Well in the current version, the example is written as follows: 
#include <RcppGSL.h>
#include <gsl/gsl_matrix.h>
#include <gsl/gsl_blas.h>
// [[Rcpp::export]]
Rcpp::NumericVector colNorm(const RcppGSL::Matrix & G)  
    int k = G.ncol();
    Rcpp::NumericVector n(k);           // to store results
    for (int j = 0; j < k; j++)  
        RcppGSL::VectorView colview = gsl_matrix_const_column (G, j);
        n[j] = gsl_blas_dnrm2(colview);
     
    return n;                           // return vector
This takes full advantage of Rcpp Attributes automagically creating the interface and exception handler (as per the previous release), adds a const & interface, does away with the tedious and error-pronce free() and uses the shorter-typedef forms for RcppGSL::Matrix and RcppGSL::VectorViews using double variables. Now the function is short and concise and hence easier to read and maintain. The package vignette has more details on using RcppGSL. The NEWS file entries follows below:
Changes in version 0.3.0 (2015-08-30)
  • The RcppGSL matrix and vector class now keep track of object allocation and can therefore automatically free allocated object in the destructor. Explicit x.free() use is still supported.
  • The matrix and vector classes now support const reference semantics in the interfaces (thanks to PR #7 by Dan Dillon)
  • The matrix_view and vector_view classes are reorganized to better support const arguments (thanks to PR #8 and #9 by Dan Dillon)
  • Shorthand forms such as Rcpp::Matrix have been added for double and int vectors and matrices including views.
  • Examples such as fastLm can now be written in a much cleaner and shorter way as GSL objects can appear in the function signature and without requiring explicit .free() calls at the end.
  • The included examples, as well as the introductory vignette, have been updated accordingly.
Courtesy of CRANberries, a summary of changes to the most recent release is available. More information is on the RcppGSL page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Sven Hoexter: 1960 SubjectAlternativeNames on one certificate

tl;dr; You can add 1960+ SubjectAlternativeNames on one certificate and at least Firefox and Chrome are working fine with that. Internet Explorer failed but I did not investigate why. So why would you want to have close to 2K SANs on one certificate? While we're working on adopting a more dynamic development workflow at my workplace we're currently bound to a central development system. From there we serve a classic virtual hosting setup with "projectname.username.devel.ourdomain.example" mapped on "/web/username/projectname/". That is 100% dynamic with wildcard DNS entries and you can just add a new project to your folder and use it directly. All of that is served from just a single VirtualHost. Now our developers started to go through all our active projects to make them fit for serving via HTTPS. While we can verify the proper usage of https on our staging system where we've validating certificates, that's not the way you'd like to work. So someone approached me to look into a solution for our development system. Obvious choices like wildcard certificates do not work here because we've two dynamic components in the FQDN. So we would've to buy a wildcard certificate for every developer and we would've to create a VirtualHost entry for every new developer. That's expensive and we don't want all that additional work. So I started to search for documented limits on the number of SANs you can have on a certificate. The good news: there are none. The RFC does not define a limit. So much about the theory. ;) Following Ivans excellent documentation I setup an internal CA and an ugly "find ... sed ... tr ..." one-liner later I had a properly formated openssl config file to generate a CSR with all 1960 "projectname.username..." SAN combinations found on the development system. Two openssl invocations (CSR generation and signing) later I had a signed certificate with 1960 SANs on it. I imported the internal CA I created in Firefox and Chrome, and to my surprise it worked. Noteworthy: To sign with "openssl ca" without interactive prompts you've to use the "-batch" option. I'm thinking about regenerating the certificate every morning so our developers just have to create a new project directory and within 24h serving via HTTPS would be enabled. The only thing I'm currently pondering about is how to properly run the CA in a corporate Windows world. We could of course ask the Windows guys to include it for everyone but then we would've to really invest time in properly running the CA. I'd like to avoid that hassle. So I'd guess we just stick to providing the CA for those developers who need it. This all or nothing model is a constant PITA, and you really do not want to get owned via your own badly managed CA. :( Regarding Internet Explorer it jumped in my face with a strange error message that recommended to enable TLS 1.0, 1.1 and 1.2 in the options menu. Of course that's already enable. I'll try to take a look at the handshake next week, but I bet we've to accept for the moment that IE will not work with so many SANs. Would be interesting to try out Windows 10 with Spartan, but well I'm not that interested in Windows to invest more time on that front. Other TLS implementations, like Java, would be also interesting to test.

Ben Hutchings: Securing www.decadent.org.uk

Sven Hoexter replied to my previous entry to say that WoSign also provides free DV TLS certificates. What's more, they allow up to 10 alternate names, unlike StartSSL. So I've gone ahead with a new certificate for www.decadent.org.uk and other virtual servers including git.decadent.org.uk. WoSign sensibly mandates a key length of 2048 bits, and together with the default TLS configuration for Apache in Debian 'jessie' this resulted in a A- rating from Qualys SSL Server Test. I then disabled non-PFS and otherwise weak cipher suites in /etc/apache2/mods-enabled/ssl.conf: 
SSLCipherSuite HIGH:!aNULL:!kRSA:!3DES
This resulted in an A rating. Finally, I added redirection of all plaintext HTTP connections to HTTP-S (which is easier than working out how to make the virtual server work with and without TLS, anyway). I enabled HSTS for each VirtualHost: 
Header always set Strict-Transport-Security "max-age=31536000; includeSubDomains"
This resulted in an A+ rating. These web sites will now be inaccessible to Java 6 and IE on Windows XP, but that's no great loss (1 in 1500 hits over the past few weeks).

Russ Allbery: Review: Pound Foolish

Review: Pound Foolish, by Helaine Olen
Publisher: Penguin
Copyright: 2012, 2013
Printing: 2013
ISBN: 1-59184-679-X
Format: Trade paperback
Pages: 241
For at least the last six years, it's not been surprising news that the relationship between the average person and the US financial system is tense at best, and downright exploitative at worst. The collapse of the housing bubble revealed a morass of predatory lending practices, spectacularly poor (or spectacularly cynical) investment decisions, and out-of-control personal debt coupled with erosion of bankruptcy law. Despite this, there's always a second story in all discussions of the finances of the US population: the problem isn't with financial structures and products, but with us. We're too stupid, or naive, or emotional, or uninformed, or greedy, or unprincipled, or impatient. Finances are complicated, yes, but that just means we have to be more thoughtful. All of these complex financial products could have been used properly. Helaine Olen's Pound Foolish is a systemtic, biting, well-researched, and pointed counter to that second story. The short summary of this book is that it's not us. We're being set up for failure, and there is a large and lucrative industry profiting off of that failure. And many (although not all) people in that industry know exactly what they're doing. Pound Foolish is one of my favorite forms of non-fiction: long-form journalism. This is an investigative essay into the personal finance and investment industry, developed to book length. Readers of Michael Lewis will feel right at home. Olen doesn't have Lewis's skill with characterization and biography, but she makes up for it in big-picture clarity. She takes a wealth of individual facts about who is involved in personal finance, how they make money, what they recommend, and who profits, and develops it into a clear and coherent overview. If you have paid any attention to US financial issues, you'll know some of this already. Frontline has done a great job of covering administrative fees in mutual funds. Lots of people have warned about annuities. The spectacular collapse of the home mortgage is old news now. But Olen does a great job of finding the connections between these elements and adding some less familiar ones, including an insightful and damning analysis of financial literacy campaigns and the widespread belief that these problems are caused by lack of consumer understanding. I've read and watched a lot of related material, including several full-book treatments of the mortgage crisis, so I think it's telling that I never got bored in the middle of Olen's treatment. I find the deep US belief in the power of personal improvement fascinating. It feels like one of the defining characteristics of US culture, for both good and for ill. We're very good at writing compelling narratives of personal improvement, and sometimes act on them. We believe that everyone can and should improve themselves. But that comes coupled to a dislike and distrust of expertise, even when it is legitimate and earned (Hofstadter's Anti-Intellectualism in American Life develops this idea at length). And I believe we significantly overestimate the ability of individuals to act despite systems that are stacked against us, and significantly overestimate our responsibility for the inevitable results. This was the main message I took from Pound Foolish: we desperately want to believe in the myth of personal control. We want to believe that our financial troubles are something we can fix through personal education, more will power, better decisions, or just the right investment. And so, we turn to gurus like Suze Orman and buy their mix of muddled financial advice and "tough love" that completely ignores broader social factors. We're easy marks for psychologically-trained investment sellers who mix fear, pressure, and a fantasy of inside knowledge and personal control. We're fooled by a narrative of empowerment and stand by while a working retirement system (guaranteed benefit pensions) is undermined and destroyed in favor of much riskier investment schemes... riskier for us, at least, but loaded with guaranteed profits for the people who "advise" us. And we cling to financial literacy campaigns that are funded by exactly the same credit card companies who fight tooth and nail against regulations that would require they provide simple, comprehensible descriptions of loan terms. One wonders if they support them precisely because they know they don't work. Olen mentions, in passing, the Stanford marshmallow experiment, which is often used as a foundation for arguments about personal responsibility for financial outcomes, but she doesn't do a detailed critique. I wish she had, since I think it's such a good example of the theme of this book. The Stanford marshmallow experiment was a psychological experiment from the late 1960s and early 1970s in delayed gratification. Children were put in a room in front of some treat (marshmallows, cookies, pretzels) and told that they could eat it if they wished. But if they didn't eat the treat before the monitor came back, they would get two of the treat instead. Long-term follow-up studies found that the children who refrained from eating the treat and got the reward had better life outcomes along multiple metrics: SAT scores, educational attainment, and others. On the surface, this seems to support everything US culture loves to believe about the power of self-control, self-improvement, and the Protestant work ethic. People who can delay gratification and save for a future reward do better in the world. (The darker interpretation, also common since the experiment was performed on children, is that the ability to delay gratification has a genetic component, and some people are just doomed to make poor decisions due to their inability to exercise self-control.) However, I can call the traditional interpretation into question with one simple question that the experimenters appeared not to ask: under what circumstances would taking the treat immediately be the rational and best choice? One answer, of course, is when one does not trust the adult to produce the promised reward. If the adult might come back, take the treat away, and not give any treat, it's to the child's advantage to take the treat immediately. Even if the adult left the treat but wouldn't actually double it, it's to the child's advantage to take the treat immediately. The traditional interpretation assumes the child trusts the adults performing the experiment a logical assumption for those of us whose childhood experience was that adults could generally be trusted and that promised rewards would materialize. If the child instead came from a chaotic family where adults weren't reliable, or just one where frequent unexpected money problems meant that promised treats often didn't materialize, the most logical assumption may be much different. One has to ask if such a background may have more to do with the measured long-term life outcomes than the child's unwillingness to trust in a future reward. And this is one of the major themes of this book. Problems the personal finance industry attributes to our personal shortcomings (which they're happy to take our money to remedy) are often systematic, or at least largely outside of our control. We may already be making the most logical choices given our personal situations. We're in worse financial shape because we're making less money. Our retirements are in danger because our retirement systems were dismantled and replaced with risky and expensive alternatives. And where problems are attributed to our poor choices, one can find entire industries that focus on undermining our ability to make good choices: scaring us, confusing us, hiding vital information, and exploiting known weaknesses of human psychology to route our money to them. These are not problems that can be solved by watching Suze Orman yell at us to stop buying things. These are systematic social problems that demand a sweeping discussion about regulation, automatic savings systems, and social insurance programs to spread risk and minimize the weaknesses of human psychology. Exactly the kind of discussion that the personal finance industry doesn't want us to have. Those who are well-read in these topics probably won't find a lot new here. Those who aren't in the US will shake their heads at some of the ways that the US fails its citizens, although many of Olen's points apply even to countries with stronger social safety nets. But if you're interested in solid long-form journalism on this topic, backed by lots of data on just how badly a focus on personal accountability is working for us, I recommend Pound Foolish. Rating: 8 out of 10

Next.

Previous.