Search Results: "ag"

4 August 2014

Ian Donnelly: New Release: Elektra 0.8.7

Hi Everybody! I am very proud to inform you all that Elektra has just shipped a new release, version 0.8.7, with many great features and fixes! First of all, I want to let you all know that a lot of work from my Google Summer of Code Project has made its way into this release. Elektra now includes support for a three way merge of KeySets! A special and sincere thanks goes out to Felix Berlakovich for helping me test this new merge feature and adding some great features to allow for different merge strategies and dealing with meta keys. You can try out the new merge features using the kdb merge command or by using the Elektra API. There is still work to be done with Merging and improving documentation (also look for some posts on this blog soon about the feature). Additionally, thanks to Felix, we have technical previews for some new plug-ins. The new plug-ins are keytometa and ini. In short the keytometa plugin allows to convert normal keys to meta keys during the get operation and reverting this conversion during the set operation. The ini plugin is basically a rewrite of the simpleini plugin and makes use of the inih library. Also there have been many improvements made to the glob plug-in. He even found some time to add a new script for bash tab completion which is located under scripts/kdb-bash-completion. To use it on debian just copy it to /etc/bash_completion.d/ and make sure it is executable. Moreover, we fixed a lot of things with this newest release. Pino Toscano has been working on fixing up the Debian packages for Elektra but he has also fixed many other things along the way including fixing a lot of spelling errors, simplifying the RPATH setting, improvements to respecting $HOME and $TMPDIR, and improvements to some test cases. The kdb tool now does a better job of checking for subfolders that aren t allow and it now makes sure to output warnings before errors so errors can more easily be seen. We have also improved some tests for kdb tool and some plugins as well as fixed compiler warnings on clang and gcc 4.9. We also made some fixes to kdb import and export for some storage plugins and fixed some bugs so that kdb run_all now works flawlessly. There have also been a few tweaks to the API for this release, specifically in the C++ bindings. There is now a delMeta() function for C++. The reason for this is that contrary to the C API, calling

key.setMeta("metaname", NULL)

does not delete the metadata, but stores the value 0 . Additionally, we changed the arguments for isBelow, isDirectBelow, and isBelowSame for the C++ binding to be easier to understand and be more natural to use. Before this change, the C++ binding closely mirrored the C API which lead to an unintuitive behaviour. Before the change the API did the following:

Key ( user/config/key/below ).isBelow (Key ( user/config )) == false
Key ( user/config/key/below ).isBelow (Key ( user/config/key/below/deeper )) == true

That is because the first argument in the C API is the object itself in the C++ API.
The attribute of being below the key in question (the object) refers to the second key in the C API.
While this makes some sense for the C API, it definitely does not for the C++ API. Now the API behaves as follows (as intuitively expected):

Key ( user/config/key/below ).isBelow (Key ( user/config )) == true
Key ( user/config/key/below ).isBelow (Key ( user/config/key/below/deeper )) == false

We even had time for a bunch of documentation changes. We now have a tutorial for contextual values to GitHub so developers can start using contextual values with Elektra. We also included a specification for metadata and a better specification for contracts. There is even a little bit of extra news to share. We now use GitHub for active development of Elektra. We have adopted its issue tracker for issues. Also, now pull requests automatically get built by the server to see if the merge would brake the build and whether it passes all the tests. We are also in the process of updating a lot of our documentation and READMEs to use Markdown so they can be viewed easily on GitHub. Also, Raffael Pancheri has been making really great progress on a qt-gui for Elektra. There is still work to be done but it looks great and is coming along nicely. You can download the release now from Markus site: http://www.markus-raab.org/ftp/elektra/releases/elektra-0.8.7.tar.gz size: 1566800
md5sum: 4996df62942791373b192c793d912b4c
sha1: 00887cc8edb3dea1bc110f69ea64f6b700c29402
sha256: 698ebd41d540eb0c6427c17c13a6a0f03eef94655fbd40655c9b42d612ea1c9b Also there are packages already ready for some distributions: There is a lot of ongoing work to fix the Debian packages and I will post about it on this blog when they are good to go! Enjoy the new release!
-Ian S. Donnelly

Konstantinos Margaritis: SIMD book, "Sponsored by ARM"!

Ok, took a while but I got the final word about this and can announce that the sponsor who donated 500 EUR to the Indiegogo campaign was ARM itself! I have to thank my friends at ARM@Cambridge and especially Dr Monika Biddulph, General Manager, Partner Enablement Group at ARM. When the book goes to print you can be sure it will include "Sponsored by ARM" somewhere! :) Also a friendly reminder that even if the campaign is over, I still welcome the support in the form of preorders/sponsorships.

Russ Allbery: Review: Parasite

Review: Parasite, by Mira Grant
Series: Parasitology #1
Publisher: Orbit
Copyright: October 2013
ISBN: 0-316-21893-6
Format: Kindle
Pages: 504
It's 2027, and a company named Symbogen has revolutionized medicine and health. They manufacture a genetically-engineered parasite based on a tapeworm that can stabilize and protect the health of just about anyone. It can synthesize medication, fix chronic medical problems, and be adapted to different conditions. (Yes, I know biological systems don't work this way. That's not going to be the only suspension of disbelief problem.) This has made Symbogen one of the most powerful corporations in the country, aided by the skill at marketing and self-promotion shown by one of the founders. Sal, the protagonist of the novel, is one of Symbogen's most famous success stories. Sally was the victim of a horrible car crash that put her into an apparently irreversible coma. But as her family was debating whether to turn off life support, she woke up. She had no memory of her previous life at all, and had to relearn fine motor control, reading, and many other skills. She was essentially a new person. But she was alive; her symbiont had saved her. When the story proper starts, Sal is still a ward of her parents. She has generally adult skills despite still struggling with reading, but she still has occasional attacks and is under intensive monitoring by Symbogen. That means periodic mandatory appointments with Symbogen, which she hates, but she's otherwise started building a life for herself: a job in an animal shelter, an interest in exotic predatory plants, and, most notably, a boyfriend. There are things about her life she doesn't like, and she wants to be free of Symbogen, but she doesn't have a bad life. But then a mysterious illness begins sweeping through the population, causing people to go blank, apparently lose their minds, and then start attacking those near them. Some of you have doubtless already figured out the key plot revelation. It's not hard; even if you didn't from the summary, you will probably figure it out shortly into the book. And therein lies a large problem with this novel: it's hopelessly predictable. Creepy evil corporation that supposedly has your best interests at heart, check. Plucky mad scientist opposition who understands exactly what's going on, check. Well-meaning but heavy-handed government agents who try to get involved but mostly make everything worse, check. (Although it's unusual to have those agents as part of the protagonist's family, and I thought that added some additional depth.) Mostly clueless protagonist sucked into the plot and becoming critical to its resolution, check. Very few readers are going to be surprised by this story. This is not, by itself, a fatal flaw. Predictable story structures can carry satisfying variations, or introduce the reader to enjoyable characters. And I think Grant manages both here. Seanan McGuire, both as herself and under her Mira Grant pseudonym, tends to write damaged and struggling characters. Both her Newsflesh and October Daye series feature protagonists that have been hurt badly, but are coping and muddling through in their own ways. In Parasite, I think she takes a more daring and intriguing approach: a protagonist that other people in the story perceive as damaged and struggling, but who actually isn't. Sal is not a badly injured Sally, and she's quite a bit healthier than those around her think she is. Her thought processes don't work quite the same as those around her, but that's not because she's hurt. That's because she's a different person. This makes Parasite partly a novel about identity, about Sal claiming ownership of her own life. Grant drags this out longer than I wish she had, but I liked the idea. In Sal, she strikes a good balance between gratitude and genuine affection for her family and the need to become her own person unconstrained by other people's expectations. As with the Newsflesh series, Grant uses quotes and excerpts from interviews to fill in the world background: a few at the start of each chapter, and more around each part boundary. I like this technique, and Grant uses it well. By the end of the book, the Rolling Stone interview with the head of Symbogen has added a lot of insight into how Symbogen manages its public relations. Grant also throws in a few of her trademark dangerously off-beat characters: hyper-competent, wise-cracking, but eerily skewed. I loved those in Blackout and I loved Tansy and Dr. Cale here. (Adam was much less successful.) A whole book from Tansy's perspective wouldn't work, since she needs Sal as a straight woman, but I thought she stole every scene she was in. However, I agree wholeheartedly with Tansy on another point: Sal is remarkably, irritatingly dim about what is apparently intended to be the critical revelation of the book. I won't state it outright; given its significant presence in the final scene, apparently it is intended to be a spoiler. But I figured it out about 50 pages into the book. Grant telegraphs this revelation heavily, and Tansy considers it painfully obvious (with quite a bit of justification). But Sal doesn't figure it out for the entire book, ignores all the signs, and is apparently willfully blind. In a book written from the first-person perspective by an otherwise-reliable narrator, this is highly annoying. It significantly undermined my enjoyment of the book. I spent much of the novel ahead of the narrator in my understanding of the plot and waiting, in vain, for her to get on with it already. That unfortunately makes Parasite a mixed bag. I really liked many of the characters, and I think Grant did some interesting things with family dynamics and with claiming one's own identity. But this is undermined by a very predictable plot, the protagonist deciding to be dumber than a sack of hammers about a critical plot point, and some rather dubious world logic. (For example, why is Sal terrified of bad driving? It makes sense as a post-traumatic stress reaction... except it's a critical point to her characterization that she never went through that stress.) Sometimes I wanted to like this book and sometimes I wanted to shake it, and sometimes I felt both reactions at the same time. I like Grant's writing and characterization well enough that I will probably read the sequel, but this is more like the later books in the Newsflesh series than it is like the spectacular Feed. Worth reading, at least for me, but it could have been better. Rating: 7 out of 10

3 August 2014

Bits from Debian: DebConf14 - schedule available

Debconf14 will be held in three weeks in Portland, OR, USA and we're happy to announce that the schedule is already available. Of course, it is still possible for some minor changes to happen! DebConf will open on Saturday, August 23 with the Welcome talk followed by two highlighted talks: There will also be also a plethora of social events, such as our traditional cheese and wine party, our group photo and our day trip. The complete schedule can be found at: https://summit.debconf.org/debconf14/ DebConf talks will be broadcast live on the Internet when possible, and videos of the talks will be published on the web along with the presentation slides.

Dirk Eddelbuettel: Introducing sanitizers 0.1.0

A new package sanitizers is now on CRAN. It provides test cases for Address Sanitizers, and Undefined Behaviour Sanitizers. These are two recent features of both g++ and clang++, and described in the Checking Memory Access section of the Writing R Extension manual. I set up a new web page for the sanitizers package which illustrates their use case via pre-built Docker images, similar to what I presented at the end of my useR! 2014 keynote a few weeks ago. So instead of repeating this over here, I invite you to read the detailed discussion on the sanitizers page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Holger Levsen: 20140803-torbrowser-launcher

About torbrowser-launcher in all current Debian distros plus some thoughts and scripts for running it more securely So, torbrowser-launcher 0.1.2-1 is now in sid (only that version has the script examples discussed below), and 0.1.1-2(~bpo70+1) are in jessie and wheezy-backports. Originally Jacob Appelbaum packaged torbrowser-launcher, then Ulrike Uhlig stepped in and fixed some major bugs, I sponsored her uploads and somehow the idea emerged to team maintain the package, so pkg-anonymity-tools was founded. So far it's only used for having a mailing list which is used for the Maintainer: field of the torbrowser-launcher package. But we invite all maintainers of anonymity related packages to join the team! Currently there ain't even a Debian teams wiki page about it (it would be great if YOU could fix that!), so that will probably be the next thing that will happen. As for version control we intend to use the collab-maint project on alioth. So joining the team is not done by joining the alioth project (technically you can do this, but it's rather pointless), but rather by putting the pkg-anonymity-tools mailing list into the Maintainer: field of your package (and you and other people into the Uploaders: field) and subscribing to that very mailing list. Once more packages are maintained that way we'll need to see whether we'll need more mailing lists (eg one specific for commit notifications) or if we rely on client side filtering only or what else should be done. The example scripts (available in /usr/share/doc/torbrowser-launcher/examples in the package from sid or in git) show how to run torbrowser-launcher, confined with AppArmor, in Xephyr (a virtual Xserver running on another Xserver) as another user. This, using AppArmor and Xephyr, shall have two effects: Does that really help? Feedback welcome. Full quote of /usr/share/doc/torbrowser-launcher/examples/README:
torbrowser-launcher launcher scripts
====================================
These scripts are intended to run torbrowser-launcher (and thus torbrowser) as
another user in an Xephyr window server running inside your normal Xorg
session.
They assume the following packages are installed:
- torbrowser-launcher
- apparmor
- xserver-xephyr, awesome
- sudo, slay, psmisc
AppArmor should be enabled, but doesn't have to. I followed the HowTo from
https://wiki.debian.org/AppArmor, which can be summed up as just adding one
parameter to the kernel to enable it, followed by a reboot.
Using Apparmor has the advantage that the browser process cannot most of the
filesystem, eg saving downloads only works in ~/.torbrowser/tbb/x86_64/tor-browser_en-US/Desktop/
On wheezy, I'm using backports for torbrowser-launcher and apparmor.
The scripts assume they have been copied to /usr/local/bin/ and that there is
a user called "foo" (for running the actuall torbrowser(-launcher) process,
and that the current user has sudo rights for the following commands:
- sudo -i -u foo /usr/local/bin/tbb-l-wrapper
- sudo slay foo
There are two scripts, tbb-in-xephyr and tbb-l-wrapper. Only tbb-in-xephyr is
to be called directly and will result in torbrowser running in Xephyr.
Known problems:
---------------
- dbus is not started, so some input methods won't work. (Personally I don't
  want/need dbus though, so I'm awaiting a solution to
  https://trac.torproject.org/projects/tor/ticket/10014)
- not everybody likes awesome as the window manager being used ;)
Ideas, questions and ToDo:
--------------------------
- maybe all of this functionality could be integrated into.
  torbrowser-launcher itself, just writing this in shell was so easy.
- or for the time being, merge these two scripts into one, doing both,
  depending on how its called. Also make them run from everywhere.
- run this in an unprivileged LXC container, which is also apparmor confined.
- (when) does this double confinement make sense?
- use a more sensible named default user (instead of foo).
- there should really be an option, so torbrowser-launcher doesn't detach
  itself, so that this "while;ps fax grep" hack can go away.
- ship an usable sudoers.d example too.
- support for more users / instances
Feedback welcome, especially accompanied by patches! 

Dimitri John Ledkov: What is net neutrality?

Sorry, the web page you have requested is not available through your internet connection.

We have received an order from the Courts requiring us to prevent access to this site in order to help protect against Lex Julia Majestatis infridgement.

If you are a home broadband customer, for more information on why certain web pages are blocked, please click here.
If you are a business customer, or are trying to view this page through your company's internet connection, please click here.

2 August 2014

Don Armstrong: ErgoDox keyboard assembly

I routinely use a Kinesis Advantage Pro keyboard, which is a split, ergonomic keyboard with thumb clusters that uses brown cherryMX switches. Over the thirteen years that I've been using it, I've become a huge fan of this style of keyboard. However, I have two major annoyances with the Kinesis. First, while the firmware is good, remapping the keys is complicated and producing more complicated keyboard layouts with layers and keycodes that are not present in the original layout is not possible. Secondly, the interconnect between the main key wells and the controller board in the middle occasionally fails, and requires disassembly and occasional re-tinning of the circuit board interconnect connector. 1 About a year ago, I became aware of the ErgoDox keyboard, which is a keyboard design which mimics the kinesis to some degree, but with completely separated key halves (useful, because I'm substantially bigger than the average human), programmable firmware (so I can finally have the layers and missing keys) and with slightly more elegant interconnects (TRRS cables). Unfortunately, at the time I first heard about it (and other custom keyboards), making it required sourcing circuit boards, parts, and finding someone to cut a case for the keyboard. Then, a few months ago, I learned about MassDrop, a company who puts together groups of people to do buys of products at near-wholesale level prices, and their offer of all of the parts to build an ErgoDox. After waiting for a group buy of the keyboard to become available, I put in an order, and received the parts two months later. Over a few hours yesterday, I learned how to do surface mount soldering of the 78 diodes (one for each key), and finished assembling and flashing the firmware. This morning, I fixed up the few key bindings that I needed to be productive, and viola, my laptop at home now has a brand new ergonomic keyboard.

1 August 2014

Raphaël Hertzog: My Free Software Activities in July 2014

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (548.59 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Distro Tracker Now that tracker.debian.org is live, people reported bugs (on the new tracker.debian.org pseudo-package that I requested) faster than I could fix them. Still I spent many, many hours on this project, reviewing submitted patches (thanks to Christophe Siraut, Joseph Herlant, Dimitri John Ledkov, Vincent Bernat, James McCoy, Andrew Starr-Bochicchio who all submitted some patches!), fixing bugs, making sure the code works with Django 1.7, and started the same with Python 3. I added a tox.ini so that I can easily run the test suite in all 4 supported environments (created by tox as virtualenv with the combinations of Django 1.6/1.7 and Python 2.7/3.4). Over the month, the git repository has seen 73 commits, we fixed 16 bugs and other issues that were only reported over IRC in #debian-qa. With the help of Enrico Zini and Martin Zobel, we enabled the possibility to login via sso.debian.org (Debian s official SSO) so that Debian developers don t even have to explicitly create their account. As usual more help is needed and I ll gladly answer your questions and review your patches. Misc packaging work Publican. I pushed a new upstream release of publican and dropped a useless build-dependency that was plagued by a difficult to fix RC bug (#749357 for the curious, I tried to investigate but it needs major work for make 4.x compatibility). GNOME 3.12. With gnome-shell 3.12 hitting unstable, I had to update gnome-shell-timer (and filed an upstream ticket at the same time), a GNOME Shell extension to start some run-down counters. Django 1.7. I packaged python-django 1.7 release candidate 1 in experimental (found a small bug, submitted a ticket with a patch that got quickly merged) and filed 85 bugs against all the reverse dependencies to ask their maintainers to test their package with Django 1.7 (that we want to upload before the freeze obviously). We identified a pain point in upgrade for packages using South and tried to discuss it with upstream, but after closer investigation, none of the packages are really affected. But the problem can hit administrators of non-packaged Django applications. Misc stuff. I filed a few bugs (#754282 against git-import-orig uscan, #756319 against wnpp to see if someone would be willing to package loomio), reviewed an updated package for django-ratelimit in #755611, made a non-maintainer upload of mairix (without prior notice) to update the package to a new upstream release and bring it to modern packaging norms (Mako failed to make an upload in 4 years so I just went ahead and did what I would have done if it were mine). Kali work resulting in Debian contributions Kali wants to switch from being based on stable to being based on testing so I did try to setup britney to manage a new kali-rolling repository and encountered some problems that I reported to debian-release. Niels Thykier has been very helpful and even managed to improve britney thanks to the very specific problem that the kali setup triggered. Since we use reprepro, I did write some Python wrapper to transform the HeidiResult file in a set of reprepro commands but at the same time I filed #756399 to request proper support of heidi files in reprepro. While analyzing britney s excuses file, I also noticed that the Kali mirrors contains many source packages that are useless because they only concern architectures that we don t host (and I filed #756523 against reprepro). While trying to build a live image of kali-rolling, I noticed that libdb5.1 and db5.1-util were still marked as priority standard when in fact Debian already switched to db5.3 and thus should only be optional (I filed #756623 against ftp.debian.org). When doing some upgrade tests from kali (wheezy based) to kali-rolling (jessie based) I noticed some problems that were also affecting Debian Jessie. I filed #756629 against libfile-fcntllock-perl (with a patch), and also #756618 against texlive-base (missing Replaces header). I also pinged Colin Watson on #734946 because I got a spurious base-passwd prompt during upgrade (that was triggered because schroot copied my unstable s /etc/passwd file in the kali chroot and the package noticed a difference on the shell of all system users). Thanks See you next month for a new summary of my activities.

One comment Liked this article? Click here. My blog is Flattr-enabled.

Ian Donnelly: How-To: Write a Plug-In (Part 1, The Basics)

Hi Everybody! For the first part of my Plug-In tutorial, I am going to cover the basic overview of an elektra plug-in. This post just explains the basic format and functions of an Elektra plug-in, specifically a storage plug-in (this will be the focus of my tutorial). All plug-ins use the same basic interface. This interface consists of five basic functions, elektraPluginOpen, elektraPluginGet, elektraPluginSet, elektraPluginError, and elektraPluginClose. The developer replaces Plugin with the name of their plugin. So in the case of my plugin, the names of these functions would be elektraLineOpen(), elektraLineGet(), elektraLineSet(), elektraLineError(), and elektraLineClose(). Additionally, there is one more function called ELEKTRA_PLUGIN_EXPORT(plugin), where once again Plugin should be replaced with the name of the plug-in, this time in lower-case. So for my line plugin this function would be ELEKTRA_PLUGIN_EXPORT(line). The KDB relies on the first five functions for interacting with configuration files stored in the key database. Calls for kdbGet() and kdbClose() will call the functions elektraPluginGet() and elektraPluginClose() respectively for the plugin that was used to mount the configuration data. kdbSet() calls elektraPluginSet() but also elektraPluginError() when an error occurs. elektraPluginOpen() is called before the first call to elektraPluginGet() or elektraPluginSet(). These functions serve different purposes that allow the plug-in to work: Most simply put: most plug-ins consist of five major functions, elektraPluginOpen(), elektraPluginClose(), elektraPluginGet(), elektraPluginSet(), and ELEKTRA_EXPORT_PLUGIN(Plugin). Because remembering all these functions can be cumbersome, we provide a skeleton plugin in order to easily create a new plugin. The skeleton plugin is called template and a new plugin can be created by calling the copy-template script. For example for my plugin I called

../../scripts/copy-template line

from within the plugins directory. Afterwards two important things are left to be done:
  1. remove all functions (and their exports) from the plugin that are not needed. For example not every plugin actually makes use of the elektraPluginOpen() function.
  2. provide a basic contract as described above
After these two steps your plugin is ready to be compiled, installed and mounted for the first time. Have a look at the mount tutorial for details on howto do this. So Part 1 of the tutorial covers the implementation details for a plug-in. Next up will be Part 2, which will cover the theory behind contracts in Elektra as well as how to implement them. Sincerely,
Ian S. Donnelly

Gunnar Wolf: Wow. Just rejected an editorial offer...

Yes, I've been bragging about the Operating Systems book all over... Today, a colleague handed me a phone call from somebody at Editorial Patria, a well known educational editorial in Mexico. They are looking for material similar to what I wrote, but need the material to be enfocado a competencias Focused on skills, a pedagogic fashion. I was more than interested, of course. As it currently stands, I am very happy that our book is being used already at three universities in three countries (by the different authors) and have heard other people saying they would recommend it, and of course I'm interested in making our work have as big an impact as possible. Of course, we'd have to modify several aspects of the book to cater to the skills focus... But it would be great to have the book available at commercial bookstores. After all, university editions are never as widely circulated as commercial ones. I had just one hard request to accept this: Our work must be distributed under a free licensing. Explicitly allow book photocopies and electronic distribution (didn't get into the "and modification" part, but I would eventually get there ;-) ) And... Of course, the negotiation immediately fell down. Editorials, this person says, live from selling individual books. She says she was turned down by another university professor and for another subject this same week. So, yes, I took the opportunity to explain things as I (and the people that think as I do Fortunately, not so few) see them. Yes, of course, editorials have to make a living. But text books are often photocopied as it is. Who buys a book? Whoever needs it. On one hand, if somebody will be using a book throughout a semester and it's reasonably priced (say, up to 3 cost of photocopies), they will probably buy it because it just works better (it is more comfortable to use and nicer to read). If a teacher likes the explanation for a particular topic, it should be completely legal for him to distribute photocopies (or digital copies) of the specific material And quite probably, among the students, more than one will end up appreciating the material enough to go look for the book in the library. And, as I have done throughout my life, if I read (in copies, electronically or in a library) a book I like... Quite probably I will go buy it. So... Of course, she insisted it was against their corporate policy. I insisted on my explanation. I hope they meet many stubborn teachers refusing to distribute books under a non-free licensing. I hope I contributed to making a dent in an industry that must change. Yes, a very very small dent, but one that helps them break free from their obsolete mindset ;-) (But yes, I don't know how long I will regret not being part of their very nice catalog of science and engineering books) ;-) )

Russell Coker: More BTRFS Fun

I wrote a BTRFS status report yesterday commenting on the uneventful use of BTRFS recently [1]. Early this morning the server that stores my email (which had 93 days uptime) had a filesystem related problem. The root filesystem became read-only and then the kernel message log filled with unrelated messages so there was no record of the problem. I m now considering setting up rsyslogd to log the kernel messages to a tmpfs filesystem to cover such problems in future. As RAM is so cheap it wouldn t matter if a few megs of RAM were wasted by that in normal operation if it allowed me to extract useful data when something goes really wrong. It s really annoying to have a system in a state where I can login as root but not find out what went wrong. After that I tried 2 kernels in the 3.14 series, both of which had kernel BUG assertions related to Xen networking and failed to network correctly, I filed Debian Bug #756714. Fortunately they at least had enough uptime for me to run a filesystem scrub which reported no errors. Then I reverted to kernel 3.13.10 but the reboot to apply that kernel change failed. Systemd was unable to umount the root filesystem (maybe because of a problem with Xen) and then hung the system instead of rebooting, I filed Debian Bug #756725. I believe that if asked to reboot a system there is no benefit in hanging the system with no user space processes accessible. Here are some useful things that systemd could have done:
  1. Just reboot without umounting (like reboot -nf does).
  2. Pause for some reasonable amount of time to give the sysadmin a possibility of seeing the error and then rebooting.
  3. Go back to a regular runlevel, starting daemons like sshd.
  4. Offer a login prompt to allow the sysadmin to login as root and diagnose the problem.
Options 1, 2, and 3 would have saved me a bit of driving. Option 4 would have allowed me to at least diagnose the problem (which might be worth the drive). Having a system on the other side of the city which has no remote console access just hang after a reboot command is not useful, it would be near the top of the list of things I don t want to happen in that situation. The best thing I can say about systemd s operation in this regard is that it didn t make the server catch fire. Now all I really know is that 3.14 kernels won t work for my server, 3.13 will cause problems that no-one can diagnose due to lack of data, and I m now going to wait for it to fail again. As an aside the server has ECC RAM and it s hardware is known to be good, so I m sure that BTRFS is at fault.

31 July 2014

Russell Coker: Links July 2014

Dave Johnson wrote an interesting article for Salon about companies ripping off the tax system by claiming that all their income is produced in low tax countries [1]. Seb Lee-Delisle wrote an insightful article about how to ask to get paid to speak [2]. I should do that. Daniel Pocock wrote an informative article about the reConServer simple SIP conferencing server [3]. I should try it out, currently most people I want to conference with are using Google Hangouts, but getting away from Google is a good thing. Fran ois Marier wrote an informative post about hardening ssh servers [4]. S. E. Smith wrote an interesting article I Am Tired of Hearing Programmers Defend Gender Essentialism [5]. Bert Archer wrote an insightful article about lazy tourism [6]. His initial example of love locks breaking bridges was a bit silly (it s not difficult to cut locks off a bridge) but his general point about lazy/stupid tourism is good. Daniel Pocock wrote an insightful post about new developments in taxis, the London Taxi protest against Uber, and related changes [7]. His post convinced me that Uber is a good thing and should be supported. I checked the prices and unfortunately Uber is more expensive than normal taxis for my most common journey. Cory Doctorow wrote an insightful article for The Guardian about the moral issues related to government spying [8]. The Verge has an interesting review of the latest Lytro Lightbox camera [9]. Not nearly ready for me to use, but interesting technology. Prospect has an informative article by Kathryn Joyce about the Protestant child sex abuse scandal in the US [10]. Billy Graham s grandson is leading the work to reform churches so that they protect children instead of pedophiles. Prospect also has an article by Kathryn Joyce about Christians home-schooling kids to try and program them to be zealots and how that hurts kids [11]. The Daily Beast has an interesting article about the way that the extreme right wing in the US are trying to kill people, it s the right wing death panel [12]. Jay Michaelson wrote an informative article for The Daily Beast about right-wing hate groups in the US who promote the extreme homophobic legislation in Russia and other countries [13]. It also connects to the Koch brothers who seem to be associated with most evil. Elias Isquith wrote an insightful article for Salon about the current right-wing obsession with making homophobic discrimination an issue of religious liberty will hurt religious people [14]. He also describes how stupid the right-wing extremists are in relation to other issues too. EconomixComix.com has a really great comic explaning the economics of Social Security in the US [15]. They also have a comic explaining the TPP which is really good [16]. They sell a comic book about economics which I m sure is worth buying. We need to have comics explaining all technical topics, it s a good way of conveying concepts. When I was in primary school my parents gave me comic books covering nuclear physics and other science topics which were really good. Mia McKenzie wrote an insightful article for BlackGirlDangerous.com about dealing with racist white teachers [17]. I think that it would be ideal to have a school dedicated to each minority group with teachers from that group.

Craig Small: Linux Capabilities

I was recently updating some code that uses fping. Initially it used exec() that was redirected to a temporary file but I changed it to use popen. While it had been a while since I ve done this sort of thing, I do recall there was an issue with running popen on setuid binary. A later found it is mainly around setuid scripts which are very problematic and there are good reasons why you don t do this. Anyhow, the program worked fine which surprised me. Was fping setuid root to get the raw socket?
$ ls -l /usr/bin/fping
-rwxr-xr-x 1 root root 31464 May  6 21:42 /usr/bin/fping
It wasn t which at first all I thought ok, so that s why popen is happy . The way that fping and other programs work is they bind to a raw socket. This socket sits below the normal type sockets such as the ones used for TCP and UDP and normal users cannot use them by default. So how did fping work it s magic and get access to this socket? It used Capabilities. Previously getting privileged features had a big problem; it was an all or nothing thing. You want access to a raw socket? Sure, be setuid but that means you also could, for example, read any file on the system or set passwords. Capabilites provide a way of giving programs some better level of access, but not a blank cheque. The tool getcap is the way of determining what capabilities are found on a file. These capabilities are attributes on the file which, when the file is run, turn into capabilities or extra permissions. fping has the capability cap_net_raw+ep applied to it. This gives access to the RAW and PACKET sockets which is what fping needs. The +ep after the capability name means it is an Effective and Permitted capability, which describes what happens with child processes and dropping privileges. I hadn t seen these Capabilities before. They are a nice way to give your programs the access they need, but limiting the risk of something going wrong and having a rouge program running as root.

Steve Kemp: luonnos viesti - 31 hein kuu 2014

Yesterday I spent a while looking at the Debian code search site, an enormously useful service allowing you to search the code contained in the Debian archives. The end result was three trivial bug reports:
#756565 - lives
Insecure usage of temporary files. A CVE-identifier should be requested.
#756566 - libxml-dt-perl
Insecure usage of temporary files. A CVE-identifier has been requested by Salvatore Bonaccorso, and will be added to my security log once allocated.
756600 - xcfa
Insecure usage of temporary files. A CVE-identifier should be requested.
Finding these bugs was a simple matter of using the code-search to look for patterns like "system.*>.*%2Ftmp". Perhaps tomorrow somebody else would like to have a go at looking for backtick-related operations (" "), or the usage of popen. Tomorrow I will personally be swimming in a loch, which is more fun than wading in code..

Russell Coker: BTRFS Status July 2014

My last BTRFS status report was in April [1], it wasn t the most positive report with data corruption and system hangs. Hacker News has a brief discussion of BTRFS which includes the statement Russell Coker s reports of his experiences with BTRFS give me the screaming heebie-jeebies, no matter how up-beat and positive he stays about it [2] (that s one of my favorite comments about my blog). Since April things have worked better. Linux kernel 3.14 solves the worst problems I had with 3.13 and it s generally doing everything I want it to do. I now have cron jobs making snapshots as often as I wish (as frequently as every 15 minutes on some systems), automatically removing snapshots (removing 500+ snapshots at once doesn t hang the system), balancing, and scrubbing. The fact that I can now expect that a filesystem balance (which is a type of defragment operation for BTRFS that frees some chunks ) from a cron job and expect the system not to hang means that I haven t run out of metadata chunk space. I expect that running out of metadata space can still cause filesystem deadlocks given a lack of reports on the BTRFS mailing list of fixes in that regard, but as long as balance works well we can work around that. My main workstation now has 35 days of uptime and my home server has 90 days of uptime. Also the server that stores my email now has 93 days uptime even though it s running Linux kernel 3.13.10. I am rather nervous about the server running 3.13.10 because in my experience every kernel before 3.14.1 had BTRFS problems that would cause system hangs. I don t want a server that s an hour s drive away to hang The server that runs my email is using kernel 3.13.10 because when I briefly tried a 3.14 kernel it didn t work reliably with the Xen kernel 4.1 from Debian/Wheezy and I had a choice of using the Xen kernel 4.3 from Debian/Unstable to match the Linux kernel or use an earlier Linux kernel. I have a couple of Xen servers running Debian/Unstable for test purposes which are working well so I may upgrade my mail server to the latest Xen and Linux kernels from Unstable in the near future. But for the moment I m just not doing many snapshots and never running a filesystem scrub on that server. Scrubbing In kernel 3.14 scrub is working reliably for me and I have cron jobs to scrub filesystems on every system running that kernel. So far I ve never seen it report an error on a system that matters to me but I expect that it will happen eventually. The paper An Analysis of Data Corruption in the Storage Stack from the University of Wisconsin (based on NetApp data) [3] shows that nearline disks (IE any disks I can afford) have an incidence of checksum errors (occasions when the disk returns bad data but claims it to be good) of about 0.42%. There are 18 disks running in systems I personally care about (as opposed to systems where I am paid to care) so with a 0.42% probability of a disk experiencing data corruption per year that would give a 7.3% probability of having such corruption on one disk in any year and a greater than 50% chance that it s already happened over the last 10 years. Of the 18 disks in question 15 are currently running BTRFS. Of the 15 running BTRFS 10 are scrubbed regularly (the other 5 are systems that don t run 24*7 and the system running kernel 3.13.10). Newer Kernels The discussion on the BTRFS mailing list about kernel 3.15 is mostly about hangs. This is correlated with some changes to improve performance so I presume that it has exposed race conditions. Based on those discussions I haven t felt inclined to run a 3.15 kernel. As the developers already have some good bug reports I don t think that I could provide any benefit by doing more testing at this time. I think that there would be no benefit to me personally or the Linux community in testing 3.15. I don t have a personal interest in RAID-5 or RAID-6. The only systems I run that have more data than will fit on a RAID-1 array of cheap SATA disks are ones that I am paid to run and they are running ZFS. So the ongoing development of RAID-5 and RAID-6 code isn t an incentive for me to run newer kernels. Eventually I ll test out RAID-6 code, but at the moment I don t think they need more bug reports in this area. I don t have a great personal interest in filesystem performance at this time. There are some serious BTRFS performance issues. One problem is that a filesystem balance and subtree removal seem to take excessive amounts of CPU time. Another is that there isn t much support for balancing IO to multiple devices (in RAID-1 every process has all it s read requests sent to one device). For large-scale use of a filesystem these are significant problems. But when you have basic requirements (such as a mail server for dozens of users or a personal workstation with a quad-core CPU and fast SSD storage) it doesn t make much difference. Currently all of my systems which use BTRFS have storage hardware that exceeds the system performance requirements by such a large margin that nothing other than installing Debian packages can slow the system down. So while there are performance improvements in newer versions of the BTRFS kernel code that isn t an incentive for me to upgrade. It s just been announced that Debian/Jessie will use Linux 3.16, so I guess I ll have to test that a bit for the benefit of Debian users. I am concerned that 3.16 won t be stable enough for typical users at the time that Jessie is released.

Wouter Verhelst: Using your Belgian eID with SSH mark two

Several years ago, I blogged about how to use a Belgian electronic ID card with SSH. I never really used it myself, but was interested in figuring out if it would still work. The good news is that since then, you don't need to recompile OpenSSH anymore to get PKCS#11 support; this is now compiled in by default. The slightly bad news is that there will be some more typework. Rather than entering ssh-add -D 0 (to access the PKCS#11 certificate in slot 0), you should now enter something along the lines of ssh-add -s /usr/lib/libbeidpkcs11.so.0. This will ask for your passphrase, but it isn't necessary to enter the correct pin code at this point in time. The first time you try to log on, you'll get a standard beid dialog box where you should enter your pin code; this will then work. The next time, you'll be logged on and you can access servers without having to enter a pin code. The worse news is that there seems to be a bug in ssh-agent, making it impossible to unload a PKCS#11 library. Doing ssh-add -D will remove your keys from the agent; the next time you try to add them again, however, ssh-agent will simply report SSH_AGENT_FAILURE. I suspect the dlopen()ed modules aren't being unloaded when the keys are removed. Unfortunately, the same (or at least, a similar) bug appears to occur when one removes the card from the cardreader. As such, I don't currently recommend trying to use this. Update: fix command-line options to ssh-add invocation above.

Petter Reinholdtsen: Debian Edu interview: Bernd Zeitzen

The complete and free out of the box software solution for schools, Debian Edu / Skolelinux, is used quite a lot in Germany, and one of the people involved is Bernd Zeitzen, who show up on the project mailing lists from time to time with interesting questions and tips on how to adjust the setup. I managed to interview him this summer. Who are you, and how do you spend your days? My name is Bernd Zeitzen and I'm married with Hedda, a self employed physiotherapist. My former profession is tool maker, but I haven't worked for 30 years in this job. 30 years ago I started to support my wife and become her officeworker and a few years later the administrator for a small computer network, today based on Ubuntu Server (Samba, OpenVPN). For her daily work she has to use Windows Desktops because the software she needs to organize her business only works with Windows . :-( In 1988 we started with one PC and DOS, then I learned to use Windows 98, 2000, XP, , 8, Ubuntu, MacOSX. Today we are running a Linux server with 6 Windows clients and 10 persons (teacher of children with special needs, speech therapist, occupational therapist, psychologist and officeworkers) using our Samba shares via OpenVPN to work with the documentations of our patients. How did you get in contact with the Skolelinux / Debian Edu project? Two years ago a friend of mine asked me, if I want to get a job in his school (Gymnasium Harsewinkel). They started with Skolelinux / Debian Edu and they were looking for people to give support to the teachers using the software and the network and teaching the pupils increasing their computer skills in optional lessons. I'm spending 4-6 hours a week with this job. What do you see as the advantages of Skolelinux / Debian Edu? The independence. First: Every person is allowed to use, share and develop the software. Even if you are poor, you are allowed to use the software included in Skolelinux/Debian Edu and all the other Free Software. Second: The software runs on old machines and this gives us the possibility to recycle computers, weeded out from offices. The servers and desktops are running for more than two years and they are working reliable. We have two servers (one tjener and one terminal server), 45 workstations in three classrooms and seven laptops as a mobile solution for all classrooms. These machines are all booting from the terminal server. In the moment we are installing 30 laptops as mobile workstations. Then the pupils have the possibility to work with these machines in their classrooms. Internet access is realized by a WLAN router, connected to the schools network. This is all done without a dedicated system administrator or a computer science teacher. What do you see as the disadvantages of Skolelinux / Debian Edu? Teachers and pupils are Windows users. <Irony on> And Linux isn't cool. It's software for freaks using the command line. <Irony off> They don't realize the stability of the system. Which free software do you use daily? Firefox, Thunderbird, LibreOffice, Ubuntu Server 12.04 (Samba, Apache, MySQL, Joomla!, and Skolelinux / Debian Edu) Which strategy do you believe is the right one to use to get schools to use free software? In Germany we have the situation: every school is free to decide which software they want to use. This decision is influenced by teachers who learned to use Windows and MS Office. They buy a PC with Windows preinstalled and an additional testing version of MS Office. They don't know about the possibility to use Free Software instead. Another problem are the publisher of school books. They develop their software, added to the school books, for Windows.

30 July 2014

Bits from Debian: Jessie will ship Linux 3.16

The Debian Linux kernel team has discussed and chosen the kernel version to use as a basis for Debian 8 'jessie'. This will be Linux 3.16, due to be released in early August. Release candidates for Linux 3.16 are already packaged and available in the experimental suite. If you maintain a package that is closely bound to the kernel version - a kernel module or a userland application that depends on an unstable API - please ensure that it is compatible with Linux 3.16 prior to the freeze date (5th November, 2014). Incompatible packages are very likely to be removed from testing and not included in 'jessie'.
  1. My kernel module package doesn't build on 3.16 and upstream is not interested in supporting this version. What can I do?
    The kernel team might be able to help you with forward-porting, but also try Linux Kernel Newbies or the mailing list(s) for the relevant kernel subsystem(s).
  2. There's an important new kernel feature that ought to go into jessie, but it won't be in 3.16. Can you still add it?
    Maybe - sometimes this is easy and sometimes it's too disruptive to the rest of the kernel. Please contact the team on the debian-kernel mailing list or by opening a wishlist bug.
  3. Will Linux 3.16 get long term support from upstream?
    The Linux 3.16-stable branch will not be maintained as a longterm branch at kernel.org. However, the Ubuntu kernel team will continue to maintain that branch, following the same rules for acceptance and review, until around April 2016. Ben Hutchings is planning to continue maintenance from then until the end of regular support for 'jessie'.

Konstantinos Margaritis: SIMD book, update 3! Addition/Subtraction mostly finished

Finally. Apologies for the delay, but it's been a busy month. This time I will hold true to my word and upload a PDF for people to see (attached to this page). So, what's new? Here is a list of things done: * Finished ALL addition/subtraction related instructions for all engines and major derivatives (SSE*/AVX, VMX/VSX, NEON/armv8 NEON). With diagrams (these were the reasons it has taken so long).
* Reorganized the structure (split the book into Parts I/II, the instruction index will be in Part II, Part I will carry the design analysis of each SIMD engine.
* Added an TOC/index.
* So far, with just Addition/Subtraction Chapter and the rest empty sections, it has reached 175 pages (B5, again I'm not fixed on the size, it might actually change)! My estimate is that the whole book will surpass 800 pages with everything included. TODO:

Next.

Previous.