Search Results: "nate"

8 September 2015

Dirk Eddelbuettel: It was twenty years ago today...

sgt pepper Hm, wasn't there a catchy tune that started that way? Just kidding. But about twenty years ago today I sent this email about a new Debian package upload -- and that makes it (as best as I can tell) the earliest trace of me doing Debian work. It so happened that I did upload two packages in July of 1995 as well, but it was so early in the project that we did not have a mailing list archive for such things yet (!!). And I have a vague recollection that the two in July were an adoption---whereas the post referenced above announced efax as my first new package added to the distribution. And there were more to come ... Anyway, thanks for all the fish! Twenty years with Debian has been a great, great ride. I learned a lot from a lot of fantastic people, and I hope I helped a few people along the way with a package or two I still maintain. Debian remains a truly fabulous project which I hope will go strongly for another 20 (or even 22).

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

7 September 2015

Steve Kemp: The Jessie 8.2 point-release broke for me

I have about 18 personal hosts, all running the Jessie release of Debian GNU/Linux. To keep up with security updates I use unattended-upgrades. The intention is that every day, via cron, the system will look for updates and apply them. Although I mostly expect it to handle security updates I also have it configured such that point-releases will be applied by magic too. Unfortunately this weekend, with the 8.2 release, things broke in a significant way - The cron deamon was left in a broken state, such that all cronjobs failed to execute. I was amazed that nobody had reported a bug, as several people on twitter had the same experience as me, but today I read through a lot of bug-reports and discovered that #783683 is to blame: The solution:
# dpkg --configure -a
# apt-get upgrade
I guess the good news is I spotted it promptly, with the benefit of hindsight the bug report does warn of this as being a concern, but I guess there wasn't a great solution. Anyway I hope others see this, or otherwise spot the problem themselves. In unrelated news the seaweedfs file-store I previously introduced is looking more and more attractive to me. I reported a documentation-related bug which was promptly handled, even though it turned out I was wrong, and I contributed CIDR support to whitelisting hosts which was merged in well. I've got a two-node "cluster" setup at the moment, and will be expanding that shortly. I've been doing a lot of little toy-projects in Go recently. This weekend I was mostly playing with the nats.io message-bus, and tying it together with sinatra.

6 September 2015

Dirk Eddelbuettel: RcppDE 0.1.4

A second maintenance release 0.1.4 of our RcppDE package, following on the heels of the recent 0.1.3. release arrived on CRAN today. RcppDE is a "port" of DEoptim, a popular package for derivative-free optimisation using differential optimization, to C++. By using RcppArmadillo, the code becomes a lot shorter and more legible. This version changes the package-internal code to use Rcpp Attributes which makes things a little simpler (and possibly more robust as well). Courtesy of CRANberries, there is also a diffstat report for the most recent release.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

5 September 2015

Russell Coker: A Long Term Review of Android Devices

Xperia X10 My first Android device was The Sony Ericsson Xperia X10i [1]. One of the reasons I chose it was for the large 4 screen, nowadays the desirable phones (the ones that are marketed as premium products) are all bigger than that (the Galaxy S6 is 5.1 ) and even the slightly less expensive phones are bigger. At the moment Aldi is advertising an Android phone with a 4.5 screen for $129. But at the time there was nothing better in the price range that I was willing to pay. I devoted a lot of my first review to the default apps for SMS and Email. Shortly after that I realised that the default email app is never going to be adequate (I now use K9 mail) and the SMS app is barely adequate (but I mostly use instant messaging). I ve got used to the fact that most apps that ship with an Android device are worthless, the camera app and the app to make calls are the only built in apps I regularly use nowadays. In the bug list from my first review the major issue was lack of Wifi tethering which was fixed by an update to Android 2.3. Unfortunately Android 2.3 ran significantly more slowly which decreased the utility of the phone. The construction of the phone is very good. Over the last 2 years the 2 Xperia X10 phones I own have been on loan to various relatives, many of whom aren t really into technology and can t be expected to take good care of things. But they have not failed in any way. Apart from buying new batteries there has been no hardware failure in either phone. While 2 is a small sample size I haven t see any other Android device last nearly as long without problems. Unfortunately I have no reason to believe that Sony has continued to design devices as well. The Xperia X10 phones crash more often than most Android phones with spontaneous reboots being a daily occurrence. While that is worse than any other Android device I ve used it s not much worse. My second review of the Xperia X10 had a section about ways of reducing battery use [2]. Wow, I d forgotten how much that sucked! When I was last using the Xperia X10 the Life360 app that my wife and I use to track each other was taking 15% of the battery, on more recent phones the same app takes about 2%. The design of modern phones seems to be significantly more energy efficient for background tasks and the larger brighter displays use more energy instead. My father is using one of the Xperia phones now, when I give him a better phone to replace it I will have both as emergency Wifi access points. They aren t useful for much else nowadays. Samsung Galaxy S In my first review of the Galaxy S I criticised it for being thin, oddly shaped, and slippery [3]. After using it for a while I found the shape convenient as I could easily determine the bottom of the phone in my pocket and hold it the right way up before looking at it. This is a good feature for a phone that s small enough to rotate in my pocket the Samsung Galaxy Note series of phones is large enough to not rotate in a pocket. In retrospect I think that being slippery isn t a big deal as almost everyone buys a phone case anyway. But it would still be better for use on a desk if the bulge was at the top. I wrote about my Galaxy S failing [4]. Two of my relatives had problems with those phones too. Including a warranty replacement I ve seen 4 of those phones in use and only one worked reliably. The one that worked reliably is now being used by my mother, it s considerably faster than the Xperia X10 because it has more RAM and will probably remain in regular use until it breaks. CyanogenMod I tried using CyanogenMod [5]. The phone became defective 9 months later so even though CyanogenMod is great I don t think I got good value for the amount of time spent installing it. I haven t tried replacing the OS of an Android phone since then. I really wish that they would start manufacturing phones that can have the OS replaced as easily as a PC. Samsung Galaxy S3 and Wireless Charging The Galaxy S3 was the first phone I owned which competes with phones that are currently on sale [6]. A relative bought one at the same time as me and her phone is running well with no problems. But my S3 had some damage to it s USB port which means that the vast majority of USB cables don t charge it (only Samsung cables can be expected to work). After I bought the S3 I bought a Qi wireless phone charging device [7]. One of the reasons for buying that is so if a phone gets a broken USB port then I can still use it. It s ironic that the one phone that had a damaged USB port also failed to work correctly with the Qi card installed. The Qi charger is gathering dust. One significant benefit of the S3 (and most Samsung phones) is that it has a SD socket. I installed a 32G SD card in the S3 and now one of my relatives is happily using it as a media player. Nexus 4 I bought a Nexus 4 [8] for my wife as she needed a better phone but didn t feel like paying for a Galaxy S3. The Nexus 4 is a nice phone in many ways but the lack of storage is a serious problem. At the moment I m only keeping it to use with Google Cardboard, I will lend it to my parents soon. In retrospect I made a mistake buying the Nexus 4. If I had spent a little more money on another Galaxy S3 then I would have had a phone with a longer usage life as well as being able to swap accessories with my wife. The Nexus 4 seems reasonably solid, the back of the case (which is glass) broke on mine after a significant impact but the phone continues to work well. That s a tribute to the construction of the phone and also the Ringke Fusion case [9]. Generally the Nexus 4 is a good phone so I don t regret buying it. I just think that the Galaxy S3 was a better choice. Galaxy Note 2 I got a Samsung Galaxy Note 2 in mid 2013 [10]. In retrospect it was a mistake to buy the Galaxy S3, the Note series is better suited to my use. If I had known how good it is to have a larger phone I d have bought the original Galaxy Note when it was first released. Generally everything is good about the Note 2. While it only has 16G of storage (which isn t much by today s standards) it has an SD socket to allow expansion. It s currently being used by a relative as a small tablet. With a 32G SD card it can fit a lot of movies. Bluetooth Speakers I received Bluetooth speakers in late 2013 [11]. I was very impressed by them but ended up not using them for a while. After they gathered dust for about a year I started using them again recently. While nothing has changed regarding my review of the Hive speakers (which I still like a lot) it seems that my need for such things isn t as great as I thought. One thing that made me start using the Bluetooth speakers again is that my phone case blocks the sound from my latest phone and makes it worse than phone sound usually is. I bought Bluetooth speakers for some relatives as presents, the relatives seemed to appreciate them but I wonder how much they actually use them. Nexus 5 The Nexus 5 [12] is a nice phone. When I first reviewed it there were serious problems with overheating when playing Ingress. I haven t noticed such problems recently so I think that an update to Android might have made it more energy efficient. In that review I was very impressed by the FullHD screen and it made me want a Note 3, at the time I planned to get a Note 3 in the second half of 2014 (which I did). Galaxy Note 3 Almost a year ago I bought the Samsung Galaxy Note 3 [13]. I m quite happy with it at the moment but I don t have enough data for a long term review of it. The only thing to note so far is that in my first review I was unhappy with the USB 3 socket as that made it more difficult to connect a USB cable in the dark. I ve got used to the socket and I can now reliably plug it in at night with ease. I wrote about Rivers jeans being the only brand that can fit a Samsung Galaxy Note series phone in the pocket [14]. The pockets of my jeans have just started wearing out and I think that it s partly due to the fact that I bought a Armourdillo Hybrid case [15] for my Note 3. I ve had the jeans for over 3 years with no noticable wear apart from the pockets starting to wear out after 10 months of using the Armourdillo case. I don t think that the Armourdillo case is bad, but the fact that it has deep grooves and hard plastic causes it to rub more on material when I take the phone out of my pocket. As I check my phone very frequently this causes some serious wear. This isn t necessarily a problem given that a phone costs 20* more than a pair of jeans, if the case was actually needed to save the phone then it would be worth having some jeans wear out. But I don t think I need more protection than a gel case offers. Another problem is that the Armourdillo case is very difficult to remove. This isn t a problem if you don t need access to your phone, IE if you use a phone like the Nexus 5 that doesn t permit changing batteries or SD cards. But if you need to change batteries, SD cards, etc then it s really annoying. My wife seems quite happy with her Armoudillo case but I don t think it was a good choice for me. I m considering abandoning it and getting one of the cheap gel cases. The sound on the Note 3 is awful. I don t know how much of that is due to a limitation in the speaker and how much is due to the case. It s quite OK for phone calls but not much good for music. Tablets I m currently on my third tablet. One was too cheap and nasty so I returned it. Another was still cheap and I hardly ever used it. The third is a Galaxy Note 10 which works really well. I guess the lesson is to buy something worthwhile so you can use it. A tablet that s slower and has less storage than a phone probably isn t going to get used much. Phone Longevity I owned the Xperia X10 for 22 months before getting the Galaxy S3. As that included 9 months of using a Galaxy S I only had 13 months of use out of that phone before lending it to other people. The Galaxy S3 turned out to be a mistake as I replaced it in only 7 months. I had the Note 2 for 15 months before getting the Note 3. I have now had the Note 3 for 11 months and have no plans for a replacement any time soon this is the longest I ve owned an Android phone and been totally satisfied with it. Also I only need to use it for another 4 months to set a record for using an Android phone. The Xperia was free as part of a telco contract. The other phones were somewhere between $500 and $600 each when counting the accessories (case, battery, etc) that I bought with them. So in 4 years and 7 months I ve spent somewhere between $1500 and $1800 on phones plus the cost of the Xperia that was built in to the contract. The Xperia probably cost about the same so I ll assume that I spent $2000 on phones and accessories. This seems like a lot. However that averages out to about $1.20 per day (and hopefully a lot less if my Note 3 lasts another couple of years). I could justify $1.20 per day for either the amount of paid work I do on Android phones or the amount of recreational activities that I perform (the Galaxy S3 was largely purchased for Ingress). Conclusion I think that phone companies will be struggling to maintain sales of high end phones in the future. When I chose the Xperia X10 I knew I was making a compromise, the screen resolution was an obvious limitation on the use of the device (even though it was one of the best devices available). The storage in the Xperia was also a limitation. Now FullHD is the minimum resolution for any sort of high-end device and 32G of storage is small. I think that most people would struggle to observe any improvement over a Nexus 5 or Note 3 at this time. I think that this explains the massive advertising campaign for the Galaxy S6 that is going on at the moment. Samsung can t sell the S6 based on it being better than previous phones because there s not much that they can do to make it obviously better. So they try and sell it for the image.

4 September 2015

Norbert Preining: How to rename a local OfflineIMAP managed folder

Since quite some time I am using OfflineIMAP to keep my mailbox status from various servers locally available on my machine. I did more or less follow the excellent guide A unix style mail setup, but didn t use folder name translation right from the beginning. So I got stuck with four folders named INBOX: acc1/INBOX, acc2/INBOX, etc. This wouldn t be a big problem, but when using Mutt with the folder sidebar patch (as available in the mutt-patched package in Debian), there is the problem that only the INBOX part is shown. offlineimap So I decided to rename all the local folders to the scheme described in the above guide. At the end I want to have a folder layout as follows:
  Maildir/acc1/acc1
  Maildir/acc1/acc1.folder1
  Maildir/acc1/acc1.folder2
where the acc1 related to the INBOX on the remote account acc1, and the others are folders on the remote end. Unfortunately, renaming is not supported in offlineimap, it would simply download everything again at best! So I checked what needs to be done, and it turned out it is not as complicated as I thought. The necessary steps are: To make this more specific, let us assume I want to reorganize the account acc1 in this way, and there are two folders synced. That is, before we start the Maildir folder contains:
Maildir/acc1/INBOX
Maildir/acc1/folder1
Maildir/acc1/folder2
and we want to achieve that the layout is
Maildir/acc1/acc1
Maildir/acc1/acc1.folder1
Maildir/acc1/acc1.folder2
Due to the fact that both the LocalStatus files and the FolderValidity files are naned in the same way as the folders, the following trivial script does all the necessary renaming for this simple case:
for d in ~/Maildir/acc1 ~/.offlineimap/Account-acc1/LocalStatus ~/.offlineimap/Repository-acc1local/FolderValidity ; do
  mv $d/INBOX $d/acc1
  mv $d/folder1 $d/acc1.folder1
  mv $d/folder2 $d/acc1.folder2
done
Finally, the correct renaming has to be done by adding nametrans rules to the helper script. The functions oimaptransfolder_acc1 and localtransfolder_acc1 as defined in the above tutorial can be used as is, and need only be added to the respective Repository sections in the configuration file. After having done the necessary renaming, what remains is to give it a good test. Best way to do it is with
offlineimap --dry-run -o -a acc1
which runs the syncronization in dry-run mode, so nothing is actually copied, and only once and only for acc1. If during this test run nothing strange appears, like many many emails are deleted on either end, then you are fine. Enjoy.

Dirk Eddelbuettel: RcppArmadillo 0.5.500.2.0

armadillo image Once again time for the monthly upstream Armadillo update -- version 5.500.2 was released earlier today by Conrad. And a new and matching RcppArmadillo release 0.5.500.2.0 in already on CRAN and will go to Debian shortly. Armadillo is a powerful and expressive C++ template library for linear algebra aiming towards a good balance between speed and ease of use with a syntax deliberately close to a Matlab. This release contains mostly bug fixes and some internal code refactoring:
Changes in RcppArmadillo version 0.5.500.2.0 (2015-09-03)
  • Upgraded to Armadillo 5.500.2 ("Molotov Cocktail")
    • expanded object constructors and generators to handle size() based specification of dimensions
    • faster handling of submatrix rows
    • faster clamp()
    • fixes for handling sparse matrices
Courtesy of CRANberries, there is also a diffstat report for the most recent CRAN release. As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

1 September 2015

Raphaël Hertzog: My Free Software Activities in August 2015

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I have been paid to work 6.5 hours on Debian LTS. In that time I did the following: Apart from that, I also gave a talk about Debian LTS at DebConf 15 in Heidelberg and also coordinated a work session to discuss our plans for Wheezy. Have a look at the video recordings: DebConf 15 I attended DebConf 15 with great pleasure after having missed DebConf 14 last year. While I did not do lots of work there, I participated in many discussions and I certainly came back with a renewed motivation to work on Debian. That s always good. :-) For the concrete work I did during DebConf, I can only claim two schroot uploads to fix the lack of support of the new overlay filesystem that replaces aufs in the official Debian kernel, and some Distro Tracker work (fixing an issue that some people had when they were logged in via Debian s SSO). While the numerous discussions I had during DebConf can t be qualified as work , they certainly contribute to build up work plans for the future: As a Kali developer, I attended multiple sessions related to derivatives (notably the Debian Derivatives Panel). I was also interested by the Debian in the corporate IT BoF led by Michael Meskes (Credativ s CEO). He pointed out a number of problems that corporate users might have when they first consider using Debian and we will try to do something about this. Expect further news and discussions on the topic. Martin Kraff, Luca Filipozzi, and me had a discussion with the Debian Project Leader (Neil) about how to revive/transform the Debian s Partner program. Nothing is fleshed out yet, but at least the process initiated by the former DPL (Lucas) is again moving forward. Other Debian work Sponsorship. I sponsored an NMU of pep8 by Daniel Stender as it was a requirement for prospector which I also sponsored since all the required dependencies are now available in Debian. \o/ Packaging. I NMUed libxml2 2.9.2+really2.9.1+dfsg1-0.1 fixing 3 security issues and a RC bug that was breaking publican. Since there s no upstream fix for more than 8 months, I went back to the former version 2.9.1. It s in line with the new requirement of release managers a package in unstable should migrate to testing reasonably quickly, it s not acceptable to keep it unfixed for months. With this annoying bug fixed, I could again upload a new upstream release of publican so I prepared and uploaded 4.3.2-1. It was my first source only upload. This release was more work than I expected and I filed no less than 3 bug to upstream (new bash-completion install path, request to provide sources of a minified javascript file, drop a .po file for an invalid language code). GPG issues with smartcard. Back from DebConf, when I wanted to sign some key, I stumbled again upon the problem which makes it impossible for me to use my two smartcards one after the other without first deleting the stubs for the private key. It s not a new issue but I decided that it was time to report it upstream, so I did it: #2079 on bugs.gnupg.org. Some research helped me to find a way to work-around the problem. Later in the month, after a dist-upgrade and a reboot, I was no longer able to use my smartcard as a SSH authentication key again it was already reported but there was no clear analysis, so I tried to do my own one and added the results of my investigation in #795368. It looks like the culprit is pinentry-gnome3 not working when started by the gpg-agent which is started before the DBUS session. Simple fix is to restart the gpg-agent in the session but I have no idea yet of what the proper fix should be (letting systemd manage the graphical user session and start gpg-agent would be my first answer, but that doesn t solve the issue for users of other init systems so it s not satisfying). Distro Tracker. I merged two patches from Orestis Ioannou fixing some bugs tagged newcomer. There are more such bugs (I even filed two: #797096 and #797223), go grab them and do a first contribution to Distro Tracker like Orestis just did! I also merged a change from Christophe Siraut who presented Distro Tracker at DebConf. I implemented in Distro Tracker the new authentication based on SSL client certificates that was recently announced by Enrico Zini. It s working nice, and this authentication scheme is far easier to support. Good job, Enrico! tracker.debian.org broke during DebConf, it stopped being updated with new data. I tracked this down to a problem in the archive (see #796892). Apparently Ansgar Burchardt changed the set of compression tools used on some jessie repositorie, replacing bz2 by xz. He dropped the old Packages.bz2 but missed some Sources.bz2 which were thus stale and APT reported Hashsum mismatch on the uncompressed content. Misc. I pushed some small improvement to my Salt formulas: schroot-formula and sbuild-formula. They will now auto-detect which overlay filesystem is available with the current kernel (previously aufs was hardcoded). Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

31 August 2015

Russ Allbery: Review: Through Struggle, The Stars

Review: Through Struggle, The Stars, by John J. Lumpkin
Series: Human Reach #1
Publisher: John J. Lumpkin
Copyright: July 2011
ISBN: 1-4611-9544-6
Format: Kindle
Pages: 429
Never let it be said that I don't read military SF. However, it can be said that I read books and then get hellishly busy and don't review them for months. So we'll see if I can remember this well enough to review it properly. In Lumpkin's future world, mankind has spread to the stars using gate technology, colonizing multiple worlds. However, unlike most science fiction futures of this type, it's not all about the United States, or even the United States and Russia. The great human powers are China and Japan, with the United States relegated to a distant third. The US mostly maintains its independence from either, and joins the other lesser nations and UN coalitions to try to pick up a few colonies of its own. That's the context in which Neil and Rand join the armed services: the former as a pilot in training, and the latter as an army grunt. This is military SF, so of course a war breaks out. But it's a war between Japan and China: improved starship technology and the most sophisticated manufacturing in the world against a much larger economy with more resources and a larger starting military. For reasons that are not immediately clear, and become a plot point later on, the United States president immediately takes an aggressive tone towards China and pushes the country towards joining the war on the side of Japan. Most of this book is told from Neil's perspective, following his military career during the outbreak of war. His plans to become a pilot get derailed as he gets entangled with US intelligence agents (and a bad supervisor). The US forces are not in a good place against China, struggling when they get into ship-to-ship combat, and Neil's ship goes on a covert mission to attempt to complicate the war with political factors. Meanwhile, Rand tries to help fight off a Chinese invasion of one of the rare US colony worlds. Through Struggle, The Stars does not move quickly. It's over 400 pages, and it's a bit surprising how little happens. What it does instead is focus on how the military world and the war feels to Neil: the psychological experience of wanting to serve your country but questioning its decisions, the struggle of working with people who aren't always competent but who you can't just make go away, the complexities of choosing a career path when all the choices are fraught with politics that you didn't expect to be involved in, and the sheer amount of luck and random events involved in the progression of one's career. I found this surprisingly engrossing despite the slow pace, mostly because of how true to life it feels. War is not a never-ending set of battles. Life in a military ship has moments when everything is happening far too quickly, but other moments when not much happens for a long time. Lumpkin does a great job of reflecting that. Unfortunately, I thought there were two significant flaws, one of which means I probably won't seek out further books in this series. First, one middle portion of the book switches away from Neil to follow Rand instead. The first part of that involves the details of fighting orbiting ships with ground-based lasers, which was moderately interesting. (All the technological details of space combat are interesting and thoughtfully handled, although I'm not the sort of reader who will notice more subtle flaws. But David Weber this isn't, thankfully.) But then it turns into a fairly generic armed resistance story, which I found rather boring. It also ties into the second and more serious flaw: the villain. The overall story is constructed such that it doesn't need a personal villain. It's about the intersection of the military and politics, and a war that may be ill-conceived but that is being fought anyway. That's plenty of conflict for the story, at least in my opinion. But Lumpkin chose to introduce a specific, named Chinese character in the villain role, and the characterization is... well. After he's humiliated early in the story by the protagonists, Li Xiao develops an obsession with killing them, for his honor, and then pursues them throughout the book in ways that are sometimes destructive to the Chinese war efforts. It's badly unrealistic compared to the tone of realism taken by the rest of the story. Even if someone did become this deranged, it's bizarre that a professional military (and China's forces are otherwise portrayed as fairly professional) would allow this. Li reads like pure caricature, and despite being moderately competent apart from his inexplicable (but constantly repeated) obsession, is cast in a mustache-twirling role of personal villainy. This is weirdly out of place in the novel, and started irritating me enough that it took me out of the story. Through Struggle, The Stars is the first book of a series, and does not resolve much by the end of the novel. That plus its length makes the story somewhat unsatisfying. I liked Neil's development, and liked him as a character, and those who like the details of combat mixed with front-lines speculation about politics may enjoy this. But a badly-simplified mustache-twirling victim and some extended, uninteresting bits mar the book enough that I probably won't seek out the sequels. Followed by The Desert of Stars. Rating: 6 out of 10

30 August 2015

Antonio Terceiro: DebConf15, testing debian packages, and packaging the free software web

This is my August update, and by the far the coolest thing in it is Debconf. Debconf15 I don t get tired of saying it is the best conference I ever attended. First it s a mix of meeting both new people and old friends, having the chance to chat with people whose work you admire but never had a chance to meet before. Second, it s always quality time: an informal environment, interesting and constructive presentations and discussions. This year the venue was again very nice. Another thing that was very nice was having so many kids and families. This was no coincidence, since this was the first DebConf in which there was organized childcare. As the community gets older, this a very good way of keeping those who start having kids from being alienated from the community. Of course, not being a parent yet I have no idea how actually hard is to bring small kids to a conference like DebConf. ;-) I presented two talks: There was also the now traditional Ruby BoF, where discussed the state and future of the Ruby ecosystem in Debian; and an in promptu Ruby packaging workshop where we introduced the basics of packaging in general, and Ruby packaging specifically. Besides shak, I was able to hack on a few cool things during DebConf: Miscellaneous updates

Philipp Kern: Automating the 3270 part of a Debian System z install

If you try to install Debian on System z within z/VM you might be annoyed at the various prompts it shows before it lets you access the network console via SSH. We can do better. From within CMS copy the default EXEC and default PARMFILE:

COPYFILE DEBIAN EXEC A DEBAUTO EXEC A
COPYFILE PARMFILE DEBIAN A PARMFILE DEBAUTO A

Now edit DEBAUTO EXEC A and replace the DEBIAN in 'PUNCH PARMFILE DEBIAN * (NOHEADER' with DEBAUTO. This will load the alternate kernel parameters file into the card reader, while still loading the original kernel and initrd files.

Replace PARMFILE DEBAUTO A's content with this (note the 80 character column limit):

ro locale=C
s390-netdevice/choose_networktype=qeth s390-netdevice/qeth/layer2=true
s390-netdevice/qeth/choose=0.0.fffc-0.0.fffd-0.0.fffe
netcfg/get_ipaddress=<IPADDR> netcfg/get_netmask=255.255.255.0
netcfg/get_gateway=<GW> netcfg/get_nameservers=<FIRST-DNS>
netcfg/confirm_static=true netcfg/get_hostname=debian
netcfg/get_domain=
network-console/authorized_keys_url=http://www.kern.lc/id_rsa.pub
preseed/url=http://www.kern.lc/preseed-s390x.cfg

Replace <IPADDR>, <GW>, and <FIRST-DNS> to suit your local network config. You might also need to change the netmask, which I left in for clarity about the format. Adjust the device address of your OSA network card. If it's in layer 3 mode (very likely) you should set layer2=false. Note that mixed case matters, hence you will want to SET CASE MIXED in xedit.

Then there are the two URLs that need to be changed. The authorized_keys_url file contains your SSH public key and is fetched unencrypted and unauthenticated, so be careful what networks you traverse with your request (HTTPS is not supported by debian-installer in Debian).

preseed/url is needed for installation parameters that do not fit the parameters file - there is an upper character limit that's about two lines longer than my example. This is why this example only contains the bare minimum for the network part, everything else goes into this preseeding file. It file can optionally be protected with a MD5 checksum in preseed/url/checksum.

Both URLs need to be very short. I thought that there was a way to specify a line continuation, but in my tests I was unable to produce one. Hence it needs to fit on one line, including the key. You might want to use an IPv4 as the hostname.

To skip the initial boilerplate prompts and to skip straight to the user and disk setup you can use this as preseed.cfg:

d-i debian-installer/locale string en_US
d-i debian-installer/country string US
d-i debian-installer/language string en
d-i time/zone US/Eastern
d-i mirror/country manual
d-i mirror/http/mirror string httpredir.debian.org
d-i mirror/http/directory string /debian
d-i mirror/http/proxy string

I'm relatively certain that the DASD disk setup part cannot be automated yet. But the other bits of the installation should be preseedable just like on non-mainframe hardware.

Dirk Eddelbuettel: RcppGSL 0.3.0

A new version of RcppGSL just arrived on CRAN. The RcppGSL package provides an interface from R to the GNU GSL using our Rcpp package. Following on the heels of an update last month we updated the package (and its vignette) further. One of the key additions concern memory management: Given that our proxy classes around the GSL vector and matrix types are real C++ object, we can monitor their scope and automagically call free() on them rather then insisting on the user doing it. This renders code much simpler as illustrated below. Dan Dillon added const correctness over a series on pull request which allows us to write more standard (and simply nicer) function interfaces. Lastly, a few new typedef declaration further simply the use of the (most common) double and int vectors and matrices. Maybe a code example will help. RcppGSL contains a full and complete example package illustrating how to write a package using the RcppGSL facilities. It contains an example of computing a column norm -- which we blogged about before when announcing an much earlier version. In its full glory, it looks like this:
#include <RcppGSL.h>
#include <gsl/gsl_matrix.h>
#include <gsl/gsl_blas.h>
extern "C" SEXP colNorm(SEXP sM)  
  try  
        RcppGSL::matrix<double> M = sM;     // create gsl data structures from SEXP
        int k = M.ncol();
        Rcpp::NumericVector n(k);           // to store results
        for (int j = 0; j < k; j++)  
            RcppGSL::vector_view<double> colview = gsl_matrix_column (M, j);
            n[j] = gsl_blas_dnrm2(colview);
         
        M.free() ;
        return n;                           // return vector
    catch( std::exception &ex )  
        forward_exception_to_r( ex );
    catch(...)  
        ::Rf_error( "c++ exception (unknown reason)" );
   
  return R_NilValue; // -Wall
 
We manually translate the SEXP coming from R, manually cover the try and catch exception handling, manually free the memory etc pp. Well in the current version, the example is written as follows:
#include <RcppGSL.h>
#include <gsl/gsl_matrix.h>
#include <gsl/gsl_blas.h>
// [[Rcpp::export]]
Rcpp::NumericVector colNorm(const RcppGSL::Matrix & G)  
    int k = G.ncol();
    Rcpp::NumericVector n(k);           // to store results
    for (int j = 0; j < k; j++)  
        RcppGSL::VectorView colview = gsl_matrix_const_column (G, j);
        n[j] = gsl_blas_dnrm2(colview);
     
    return n;                           // return vector
 
This takes full advantage of Rcpp Attributes automagically creating the interface and exception handler (as per the previous release), adds a const & interface, does away with the tedious and error-pronce free() and uses the shorter-typedef forms for RcppGSL::Matrix and RcppGSL::VectorViews using double variables. Now the function is short and concise and hence easier to read and maintain. The package vignette has more details on using RcppGSL. The NEWS file entries follows below:
Changes in version 0.3.0 (2015-08-30)
  • The RcppGSL matrix and vector class now keep track of object allocation and can therefore automatically free allocated object in the destructor. Explicit x.free() use is still supported.
  • The matrix and vector classes now support const reference semantics in the interfaces (thanks to PR #7 by Dan Dillon)
  • The matrix_view and vector_view classes are reorganized to better support const arguments (thanks to PR #8 and #9 by Dan Dillon)
  • Shorthand forms such as Rcpp::Matrix have been added for double and int vectors and matrices including views.
  • Examples such as fastLm can now be written in a much cleaner and shorter way as GSL objects can appear in the function signature and without requiring explicit .free() calls at the end.
  • The included examples, as well as the introductory vignette, have been updated accordingly.
Courtesy of CRANberries, a summary of changes to the most recent release is available. More information is on the RcppGSL page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Ben Hutchings: Securing www.decadent.org.uk

Sven Hoexter replied to my previous entry to say that WoSign also provides free DV TLS certificates. What's more, they allow up to 10 alternate names, unlike StartSSL. So I've gone ahead with a new certificate for www.decadent.org.uk and other virtual servers including git.decadent.org.uk. WoSign sensibly mandates a key length of 2048 bits, and together with the default TLS configuration for Apache in Debian 'jessie' this resulted in a A- rating from Qualys SSL Server Test. I then disabled non-PFS and otherwise weak cipher suites in /etc/apache2/mods-enabled/ssl.conf:
SSLCipherSuite HIGH:!aNULL:!kRSA:!3DES
This resulted in an A rating. Finally, I added redirection of all plaintext HTTP connections to HTTP-S (which is easier than working out how to make the virtual server work with and without TLS, anyway). I enabled HSTS for each VirtualHost:
Header always set Strict-Transport-Security "max-age=31536000; includeSubDomains"
This resulted in an A+ rating. These web sites will now be inaccessible to Java 6 and IE on Windows XP, but that's no great loss (1 in 1500 hits over the past few weeks).

29 August 2015

Norbert Preining: Kobo Glo and GloHD firmware 3.17.3 mega update (KSM, nickel patch, ssh, fonts)

I have updated my mega-update for Kobo to the latest firmware 3.17.3. Additionally, I have not built (and tested) updates for both Mark4 hardware (Glo) and Mark6 hardware (GloHD). Please see the previous post for details on what is included. The only difference that is important is the update to KSM (Kobo Start Menu) version 8, which is still in testing phase (thus a warning: the layout and setup of KSM8 might change till release). This is an important update as all version up to V7 could create database corruptions (which I have seen several times!) when used with Calibre and the Kepub driver. Kobo Logo Other things that are included are as usual: Metazoa firmware patches for the Glo (non HD) version I have activated the compact layout patch; koreader, pbchess, coolreader, the ssh part of kobohack, custom dictionaries support, and some side-loaded fonts. Again, for details please see the previous post You can check for database corruption by selecting tools - nickel diverse.msh - db chk integrity.sh in the Kobo Start Menu. If it returns ok, then all is fine. Otherwise you might see problems. I solved the corruption of my database by first dumping the database to an sql file, and reloading it into a new database. Assuming that you have the file KoboReader.sqlite, what I did is:
$ sqlite3  KoboReader.sqlite 
SQLite version 3.8.11.1 2015-07-29 20:00:57
Enter ".help" for usage hints.
sqlite> PRAGMA integrity_check;
*** in database main ***
Page 5237: btreeInitPage() returns error code 11
On tree page 889 cell 1: 2nd reference to page 5237
Page 4913 is never used
Page 5009 is never used
Error: database disk image is malformed
sqlite> .output foo.sql
sqlite> .dump
sqlite> .quit
$ sqlite3 KoboReader.sqlite-NEW
SQLite version 3.8.11.1 2015-07-29 20:00:57
Enter ".help" for usage hints.
sqlite> .read foo.sql
sqlite> .quit
The first part shows that the database is corrupted. Fortunately dumping succeeded and then reloading it into a new database, too. Finally I replaced (after backup) the sqlite on the device with the new database. Download Mark6 Kobo GloHD firmware: Kobo 3.17.3 for GloHD Mega update: Kobo-3.17.3-combined/Mark6/KoboRoot.tgz Mark4 Kobo Glo, Auro HD firmware: Kobo 3.17.3 for Glo and AuroHD Mega update: Kobo-3.17.3-combined/Mark4/KoboRoot.tgz Enjoy.

28 August 2015

Dimitri John Ledkov: Go enjoy Python3

Given a string, get a truncated string of length up to 12.

The task is ambiguous, as it doesn't say anything about whether or not 12 should include terminating null character or not. None the less, let's see how one would achieve this in various languages.
Let's start with python3

import sys
print(sys.argv[1][:12])

Simple enough, in essence given first argument, print it up to length 12. As an added this also deals with unicode correctly that is if passed arg is , it will correctly print . (note these are just random Unicode strings to me, no idea what they stand for).

In C things are slightly more verbose, but in essence, I am going to use strncpy function:

#include <stdio.h>
#include <string.h>
void main(int argc, char *argv[])
char res[12];
strncpy(res,argv[1],12);
printf("%s\n",res);

This treats things as byte-array instead of unicode, thus for unicode test it will end up printing just . But it is still simple enough.
Finally we have Go
package main

import "os"
import "fmt"
import "math"

func main()
fmt.Printf("%s\n", os.Args[1][:int(math.Min(12, float64(len(os.Args[1]))))])

This similarly treats argument as a byte array, and one needs to cast the argument to a rune to get unicode string handling. But there are quite a few caveats. One cannot take out of bounds slices. Thus a na ve os.Args[1][:12] can result in a runtime panic that slice bounds are out of range. Or if a string is known at compile time, a compile time error. Hence one needs to calculate length, and do a min comparison. And there lies the next caveat, math.Min() is only defined for float64 type, and slice indexes can only be integers and thus we end up writing ]))))])...

12 points for python3, 8 points for C, and Go receives nul points Eurovision style.

EDIT: Andreas R ssland and James Hunt are full of win. Both suggesting fmt.Printf("%.12s\n", os.Args[1]) for go. I like that a lot, as it gives simplicity & readability without compromising the default safety against out of bounds access. Hence the scores are now: 14 points for Go, 12 points for python3 and 8 points for C.

EDIT2: I was pointed out much better C implementation by Keith Thompson - http://pastebin.com/5i7rFmMQ in essence it uses strncat() which has much better null termination semantics. And Ben posted a C implementation which handles wide characters http://www.decadent.org.uk/ben/blog/truncating-a-string-in-c.html. I regret to inform you that this blog post got syndicated onto hacker news and has now become the top viewed post on my blog of all time, overnight. In retrospect, I regret awarding points at the end of the blog post, as that's just was merely an expression of opinion and is highly subjective measure. But this problem statement did originate from me reviewing go code that did "if/then/else" comparison and got it wrong to truncate a string and I thought surely one can just do [:12] which has lead me down the rabbit hole of discovering a lot about Go; it's compile and runtime out of bounds access safeguards; lack of universal Min() function; runes vs strings handling and so on. I'm only a beginner go programmer and I am very sorry for wasting everyone's time on this. I guess people didn't have much to do on a Throwback Thursday.

The postings on this site are my own and don't necessarily represent Intel s positions, strategies, or opinions.

Lucas Nussbaum: DebConf 15

I attended DebConf 15 last week. After being on semi-vacation from Debian for the last few months, recovering after the end of my second DPL term, it was great to be active again, talk to many people, and go back to doing technical work. Unfortunately, I caught the debbug quite early in the week, so I was not able to make it as intense as I wanted, but it was great nevertheless. I still managed to do quite a lot:
  • I rewrote a core part of UDD, which will make it easier to monitor data importer scripts and reduce the cron-spam
  • with DSA members, I worked on finding a suitable workaround for the storage performance issues that have been plaguing UDD for the last few months. fsyncs() will now longer hang for 15 minutes, yay!
  • I added a DUCK importer to UDD, and added that information to the Debian Maintainer Dashboard
  • I worked a bit on cleaning up the status of my packages, including digging into a strange texlive issue (that showed up in developers-reference), that is now fixed in unstable
  • I worked a bit on improving git-buildpackage documentation (more to come in that area)
  • Last but not least, I played Mao for the first time in years, and it was a lot of fun. (even if my brain is still slowly recovering)
DC15 was a great DebConf, probably one of the two bests I ve attended so far. I m now looking forward to DC16 in Cape Town!

Ben Hutchings: Securing git imap-send in Debian

I usually send patches from git via git imap-send, which gives me a chance to edit and save them through my regular mail client. Obviously I want to make a secure connection to the IMAP server. The upstream code now supports doing this with OpenSSL, but git is under GPL and it seems that not all relevant contributors have given the extra permission to link with OpenSSL. So in Debian you still need to use an external program to provide a TLS tunnel. The commonly used TLS tunnelling programs, openssl s_client and stunnel, do not validate server certificates in a useful way - at least by default. Here's how I've configured git imap-send and stunnel to properly validate the server certificate. If you use the PLAIN or LOGIN authentication method with the server, you will still see the warning:
*** IMAP Warning *** Password is being sent in the clear
The server does see the clear-text password, but it is encrypted on the wire and git imap-send just doesn't know that. ~/.gitconfig
[imap]
	user = ben
	folder = "drafts"
	tunnel = "stunnel ~/.git-imap-send/stunnel.conf"
~/.git-imap-send/stunnel.conf
debug = 3
foreground = yes
client = yes
connect = mail.decadent.org.uk:993
sslVersion = TLSv1.2
renegotiation = no
verify = 2
; Current CA for the IMAP server.
; If you don't want to pin to a specific CA certificate, use
; /etc/ssl/certs/ca-certificates.crt instead.
CAfile = /etc/ssl/certs/StartCom_Certification_Authority.pem
checkHost = mail.decadent.org.uk
If stunnel chokes on the checkHost variable, it doesn't support certificate name validation. Unfortunately no Debian stable release has this feature - only testing/unstable. I'm wondering whether it would be worthwhile to backport it or even to make a stable update to add this important security feature.

Ben Hutchings: Securing git imap-send in Debian

I usually send patches from git via git imap-send, which gives me a chance to edit and save them through my regular mail client. Obviously I want to make a secure connection to the IMAP server. The upstream code now supports doing this with OpenSSL, but git is under GPL and it seems that not all relevant contributors have given the extra permission to link with OpenSSL. So in Debian you still need to use an external program to provide a TLS tunnel. The commonly used TLS tunnelling programs, openssl s_client and stunnel, do not validate server certificates in a useful way - at least by default. Here's how I've configured git imap-send and stunnel to properly validate the server certificate. If you use the PLAIN or LOGIN authentication method with the server, you will still see the warning:
*** IMAP Warning *** Password is being sent in the clear
The server does see the clear-text password, but it is encrypted on the wire and git imap-send just doesn't know that. ~/.gitconfig
[imap]
	user = ben
	folder = "drafts"
	tunnel = "stunnel ~/.git-imap-send/stunnel.conf"
~/.git-imap-send/stunnel.conf
debug = 3
foreground = yes
client = yes
connect = mail.decadent.org.uk:993
sslVersion = TLSv1.2
renegotiation = no
verify = 2
; Current CA for the IMAP server.
; If you don't want to pin to a specific CA certificate, use
; /etc/ssl/certs/ca-certificates.crt instead.
CAfile = /etc/ssl/certs/StartCom_Certification_Authority.pem
checkHost = mail.decadent.org.uk
If stunnel chokes on the checkHost variable, it doesn't support certificate name validation. Unfortunately no Debian stable release has this feature - only testing/unstable. I'm wondering whether it would be worthwhile to backport it or even to make a stable update to add this important security feature.

Ben Hutchings: Securing git imap-send in Debian

I usually send patches from git via git imap-send, which gives me a chance to edit and save them through my regular mail client. Obviously I want to make a secure connection to the IMAP server. The upstream code now supports doing this with OpenSSL, but git is under GPL and it seems that not all relevant contributors have given the extra permission to link with OpenSSL. So in Debian you still need to use an external program to provide a TLS tunnel. The commonly used TLS tunnelling programs, openssl s_client and stunnel, do not validate server certificates in a useful way - at least by default. Here's how I've configured git imap-send and stunnel to properly validate the server certificate. If you use the PLAIN or LOGIN authentication method with the server, you will still see the warning:
*** IMAP Warning *** Password is being sent in the clear
The server does see the clear-text password, but it is encrypted on the wire and git imap-send just doesn't know that. ~/.gitconfig
[imap]
	user = ben
	folder = "drafts"
	tunnel = "stunnel ~/.git-imap-send/stunnel.conf"
~/.git-imap-send/stunnel.conf
debug = 3
foreground = yes
client = yes
connect = mail.decadent.org.uk:993
sslVersion = TLSv1.2
renegotiation = no
verify = 2
; Current CA for the IMAP server.
; If you don't want to pin to a specific CA certificate, use
; /etc/ssl/certs/ca-certificates.crt instead.
CAfile = /etc/ssl/certs/StartCom_Certification_Authority.pem
checkHost = mail.decadent.org.uk
If stunnel chokes on the checkHost variable, it doesn't support certificate name validation. Unfortunately no Debian stable release has this feature - only testing/unstable. I'm wondering whether it would be worthwhile to backport it or even to make a stable update to add this important security feature.

26 August 2015

Dirk Eddelbuettel: RProtoBuf 0.4.3

A new maintenance release 0.4.3 of RProtoBuf is now on CRAN. RProtoBuf provides R bindings for the Google Protocol Buffers ("Protobuf") data encoding library used and released by Google, and deployed as a language and operating-system agnostic protocol by numerous projects. This release comes upon the request of CRAN and adds additional import statements to the NAMESPACE file. While we were at it, a few more things got cleaned up and edited---but no new code was added. Full details are below.
Changes in RProtoBuf version 0.4.3 (2015-08-25)
  • Declare additional imports from methods in NAMESPACE.
  • Travis CI tests now run faster as all CRAN dependencies are installed as binaries.
  • The tools/winlibs.R script now tests for R (< 3.3.0) before calling the (soon-to-be phased out) setInternet2() function.
  • Several small edits were made to DESCRIPTION to clarify library dependencies, provide additonal references and conform to now-current R packaging standards.
CRANberries also provides a diff to the previous release. The RProtoBuf page has a package vignette, a a 'quick' overview vignette, and a unit test summary vignette. Questions, comments etc should go to the GitHub issue tracker off the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

24 August 2015

Iustin Pop: Finally, systemd!

Even though Debian has moved to systemd as default a long while ago now, I've stayed with sysv as I have somewhat custom setups (self-built trimmed down kernels, separate /usr not pre-mounted by initrd, etc.). After installing a new system with Jessie and playing a bit with systemd on it a couple of months ago, I said it's finally time to upgrade. Easier said than starting to actually do it . The first system I upgraded was a recent (~1 year old) install. It was a trimmed-down system with Debian's kernel, so everything went smoothly. So smoothly that I soon forgot I made the change, and didn't do any more switches for a while. Systemd was therefore out of my mind until this recent Friday when I got a bug report about mt's rcS init script and shipping a proper systemd unit. The first step should be to actually start using systemd, so I said - let's convert some more things! During the weekend I upgraded one system, still a reasonably small install, but older - probably 6-7 years. First reboot into systemd flagged the fact that I had some forced-load modules which no longer exist, fact that was too easy to ignore with sysv. Nice! The only downside was that there seems to be some race condition between and ntp, as it fails to start on boot (port listen conflict). I'll see if it repeats. Another small issue is that systemd doesn't like duplicate fstab entries (i.e. two devices which both refer to the same mount point), while this works fine for mount itself (when specifying the block device). I said that after that system, I'll wait a while until to upgrade the next. But so it happened that today another system had an issue and I had to reboot it (damn lost uptimes!). The kernel was old so I booted into a newer one (this time compiled with the required systemd options), so I had a though - what if I take the opportunity and also switch to systemd on this system? Caution said to wait, since this was the oldest system - installed sometime during or before 2004. Plus it doesn't use an initrd (long story), and it has a split /usr. Caution excitement caution lost and I proceeded. It turns out that systemd does warn about split /usr but itself has no problems. I learned that I also had very old sysfs entries that no longer exist, and which I didn't know about as sysv doesn't make it obvious. I also had a crypttab entry which was obsolete, and I forgot about it, until I met the nice red moving ASCII bar which fortunately had a timeout. To be honest, I believed I'll have to rescue boot and fix things on this "always-unstable" machine, on which I install and run random things, and which has a hackish /etc/fstab setup. I'm quite surprised it just worked. On unstable. So thanks a lot to the Debian systemd team. It was much simpler than I thought, and now, on to exploring systemd! P.S.: the sad part is that usually I'm a strong proponent of declarative configuration, but for some reason I was reluctant to migrate to systemd also on account on losing the "power" of shell scripts. Humans

Next.

Previous.