Search Results: "nate"

5 March 2013

Tanguy Ortolo: Suspend your computer from a non-mainstream desktop environment

Battery icon UPower for regular users Major desktop environments usually provide a user-friendly menu to suspend your computer. Internally, if seems to use something called UPower, which uses something called PolicyKit, which in turn may or may not use another piece called ConsoleKit (do not ask me what all these pieces are and how they relate to each other, I do not know and I do not want to know). For the regular user, the result is the following: when they click on that button, it suspends their computer.UPower for command line users However, if you are using a more unusual desktop environment, for instance a simple, light window manager, you will not have such a button. And no, UPower does not provide a simple command line tool to suspend your computer (they probably consider that command line is out of fashion ). Well, in case you need it, here is a way to suspend your computer from a command line or whatever: just alias it or bind it to a key shortcut. It uses some dark DBus magic:
$ dbus-send --system --print-reply --dest='org.freedesktop.UPower' \
    /org/freedesktop/UPower org.freedesktop.UPower.Suspend
If you prefer to hibernate your computer, that is, to suspend it, not to RAM but to disk:
$ dbus-send --system --print-reply --dest='org.freedesktop.UPower' \
    /org/freedesktop/UPower org.freedesktop.UPower.Hibernate
As you can see, for some reason it requires some duplication of information: the name org.freedesktop.UPower appears three times in that command line: in the destination , object path and interface member , whatever that could mean. Again, do not ask me why, I know nothing about that and I do not want to. Notes
  1. Actually, UPower does provide a command line tool called upower, but the only thing it does (monitoring power sources) is not really useful for that matter.

Lisandro Damián Nicanor Pérez Meyer: My Debian freeze experience (so far)

This is the first freeze in which I'm involved with upload rights. And it turned out to be a quite interesting ride so far, so I thought it would be nice to write about it.

As some of you may know, I'm a part of the Qt/KDE team. Before the freeze I was mostly involved in leaf packages, with some patch here or there, nothing fancy. And then the freeze came...

Bugs in Qt

...and bugs appeared in Qt. But they didn't get solved, even if the patches were there. Due to personal reasons, the manpower in Qt/KDE land decreased below normal levels (which were already low).

I took the time to review them, apply them in a local branch, build and test the fixes. I did a Qt upload before, but it was a team-consented one. This time there was not much reaction in our IRC channel as it used to be, so I was doubting if going ahead or not. I asked Ana, my great friend and former sponsor, for an opinion on the subject, and she gave me a really important advice: the patches were looking good and there is one really big true: if something get's broken, it can be fixed with a later upload.

You might be asking yourself why I was that afraid of doing the upload. Well, when one maintains such a medular package for many users one has to be careful And I also got used that those "big ones" like Qt where normally handled by hand skilled people. Do not take me wrong here, it's not that those people where keeping them for themselves, it's knowing that one does not has the same skills nor experience as them.

But again, no one was able to upload and I had the chance and will to do another upload if needed, so off it went. That was Qt 4:4.8.2-2.

Then new experiences followed: asking a buildd maintainer for a giveback, asking the Release Team for an unblock (more on this later), etc. While sponsoring me, Ana gave me another excellent advice which I always keep in my mind:

You can't know **everything** about Debian.

And that also includes a not so technical skill: communicating with other teams. But finally we got this new version of Qt in testing. Cool :-)

Of course, new bugs appeared, and my lack of skills (and sometimes, time) where replaced by team work: Pino looking at patches and Sune contacting upstream. The eleven uploads that followed are a nice example of team work, even if I was the one who signed and did the uploads. Whoever uses Qt must know that these wonderful people (including those who are not so active nowadays like Modestas or Fathi) have done lots to bring the better to their users.Thank you guys!

Be careful, they might bite you back!

Coming back to the non-technical skills, sometimes you have to communicate with other teams in Debian. And each team is (naturally) a separate world: possibly different people, different goals, etc. Of course, we share the goal to make Debian the best experience we can, but we do not necessarily agree on the paths to achieve so.

During the freeze, there is a team that gets lots of pressure, and not by chance: the Release Team. They handle a very important task, which is to ride the freeze to get to a release. OK, that's what everyone knows. Now, one thing is knowing that and another is really understanding what does that means.

Of course I was in the first group. From the outside, communicating with the RT was a kind of "special art", and not an easy one. I have even been advised to not ask for more than one or two unblocks per weekend, as they might "bite me back". So I put on my flamesuit on and... launched reportbug release.debian.org.

Now I'm really happy to say that my experience was far from what I described above. And yes, I had the chance to even disagree on some stuff. But remember: non-technical skills, a.k.a. social skills. Once I started to know what was going on inside the RT (joining #debian-release was a big help for that) I learnt some nice tips to approach them. Please allow me to list some of them:

  • Remember: you are the maintainer of the package, they are like gatekeepers that are there to help us coordinate to do a release. But they don't maintain the code, you do that. So try to be verbose when needed, explain the changes and don't forget a nice diff. They need to understand what is going on: they can't read your mind.
  • They are human beings too: not everyday might be their best day (the same goes for you too!). And they are under the pressure of a release. Be patient, that finally pays off.
  • Does your changes seem not so clear? try to improve them.
  • The package has a lot of changes but you really feel they are needed? Try to explain that as good as you can.
  • Try to put yourself in their position: do we really want this? If in doubt, there is a nice way to know what they think: a pre-approval bug.
I want to make a stop in this last point. A pre-approval bug it's an unblock bug in which you edit the subject to add "pre-approval" in it. Easy, isn't it? It gives you the opportunity to know what the RT thinks before doing the upload. In other words: it gives you the chance to communicate and do things in the best possible way for all the parts involved.

I've have also seen pre-approval bugs that were really not needed. But to learn where the threshold of what can be directly uploaded and what deserves a pre-approval bug is you need to know the guidelines the RT gives you. Do you still have doubts? fire a pre-approval bug and try to be clear.

Of course, this are all fruits of my experience with the RT during this time. If the RT thinks different from what I'm writing here, please stand up: we are hear to listen to you and learn :-)

As a side note, I think I should file a wishlist bug to include the pre-approval bug option in reportbug. Yes, I'm lazy :-)

Summing up

Overall this was a very nice and positive experience. We are not done yet. Are we really done at some point? Let's hope not, because this is where the fun comes from :-)

3 March 2013

Lucas Nussbaum: DPL game

When Francesca started her DPL game, I too started think about possible candidates. Here is my shortlist of dream candidates: Seriously, if you are a DD, you have the right to run. There s no need for someone to nominate you. If you think that you could possibly say something interesting during the discussion period, and can spare the time to participate in the -vote@ discussions, please run. DPL campaigns used to be a great time where Debian visions, goals, politics and random stuff were discussed. The more candidates, the more interesting campaigns (8 candidates in 2007!). Also, there are already two three other candidates, so even if you don t want the job, it s not that risky. (Initially, I thought about nominating everyone, but security-wise, it might not be such a great idea.)

Gregor Herrmann: DPL game

when francesca started her DPL game, I also started thinking about people I'd like to encourage to stand as a DPL candidate. unfortunately my initial "short list" contained 18 names, which doesn't fit the game's rules very well. after waiting a bit, 9 of them were mentioned by others already, so I can leave them out here (which also means no public hug, sorry, please complain to the inventor of the game!). from the remaining 9, I've now chosen my Fantastic Four (in alphabtic order of first names): in my opinion, the DPL task is mostly about managing, & those four people are in my experience all very good at organising projects & dealing with people, & are therefore well suited for this task. let's see if they feel motivated to take on the challenge! (the nomination period has already started, so don't hestitate!)

Petter Reinholdtsen: Frikanalen - Complete TV station organised using the web

Do you want to set up your own TV station, schedule videos and broadcast them on the air? Using free software? With video on demand support using free and open standards? Included a web based video stream as well? And administrate it all in your web browser from anywhere in the world? A few years now the Norwegian public access TV-channel Frikanalen have been building a system to do just this. The source code for the solution is licensed using the GNU LGPL, and available from github. The idea is simple. You upload a video file over the web, and attach meta information to the file. You select a time slot in the program schedule, and when the time come it is played on the air and in the web stream. It is also made available in a video on demand solution for anyone to see it also outside its scheduled time. All you need to run a TV station - using your web browser. There are several parts to this web based solution. I'll mention the three most important ones. The first part is the database of videos and the schedule. This is written in Django and include a REST API. The current database is SQLite, but the plan is to migrate it to PostgreSQL. At the moment this system can be tested on beta.frikanalen.tv. The second part is the video playout, taking the schedule information from the database and providing a video stream to broadcast. This is done using CasparCG from SVT and Media Lovin' Toolkit. Video signal distribution is handled using Open Broadcast Encoder. The third part is the converter, handling the transformation of uploaded video files to a format useful for broadcasting, streaming and video on demand. It is still very much work in progress, so it is not yet decided what it will end up using. Note that the source of the latter two parts are not yet pushed to github. The lead author want to clean them up a bit more first. The development is coordinated on the #frikanalen IRC channel (irc.freenode.net), and discussed on the frikanalen mailing list. The lead developer is Benjamin Bruheim (phed on IRC). Anyone is welcome to participate in the development.

2 March 2013

Raphaël Hertzog: My Free Software Activities in February 2013

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (78.31 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Debian packaging I wanted to update publican to the latest upstream release but I stopped after a few hours of work during which I filed two bugs that a modicum of testing should have caught before release. So I decided to wait for the next minor release. I uploaded python-django 1.4.4 and 1.4.5, new upstream maintenance and security releases which thus went into wheezy. I also prepared a stable update of Django (1.2.3-3+squeeze5) which required me to backport the last 2 sets of security patches. I uploaded a new revision of wordpress to fix a problem with TinyMCE (#700289) and to update/add many translation files (#697208). Bug reporting and misc fixes Live-build issue. I experienced some intermittent failures when building HDD live images with live-build on armel. Daniel Baumann directed me to the problematic piece of code (the oversizing of the image size was not enough) so I committed a small fix by increasing the oversizing factor to 6%. Live-config issue. I also reported another issue that I diagnosed in live-config (#701788), namely that the script which setups sudo was failing when the default user is root. git-buildpackage issue. I filed #700411 after noticing that git-import-orig imported the debian directory provided by upstream. Those directories are not used with 3.0 (quilt) source package and their presence in the upstream branch is thus harmful: any change to the upstream debian directory will result in conflicts when you merge a new upstream release in your packaging branch. rubygems integration. Later I had to package a bunch of ruby applications that were using Bundler and I wanted to reuse as many packaged ruby modules that I could. But for this, those modules had to provide the required rubygems meta-information. I filed #700419 to request those on rake-compiler and with the help of C dric Boutillier (and others on #debian-ruby), we identified a bunch of ruby modules which could get those with a simple recompilation. I filed bin-nmu requests in #700605. Misc bugs. simple-cdd offers to select profiles to install but I noticed that the associated debconf template was not translated (#700915). The startup scripts (provided by initscripts) in charge of activating the swap are supposed to handle a noswap kernel command line option to disable swap. In #701301, I reported that the option was not working correctly if quiet was present first in the command line due to spurious break statements. Debian France Administrative work. We were late for some legal procedures so I wrote the report of the last general assembly and sent it to the Tribunal d instance of Sarreguemines to record the changes in the administrative board. I also completed the special register of the association, it s a notebook that is legally required and that must document any important change in the governance structure of the association (new members of the board, headquarters change, new bylaws, etc.). Galette developments. Debian France is funding a few enhancements to the Galette free software that we re using to manage the association. I am in touch with the Galette developer to answer his questions and ensure that his work will meet our needs. Librement I have been looking for talented developers who have a genuine interest in my Librement project. I want to fund the initial development of the project but I don t have the means to fund it entirely. So I really wanted to find developers who would find an interest beside the money that I would pay. I got in touch with the team of developers from Scopyleft and they look like very good candidates. But they re heavy users of the Scrum development method and asked me to play the role of product owner . So I started to describe the project with user stories (i.e. create the backlog in the Scrum jargon), you can have a look at them here on trello.com. If you re interested by the topic of free software funding, feel free to review and to send me your comments. My goal is clearly to have a minimal viable product with the first iteration(s) that I fund and then use the platform itself to fund further developments of the project. Thanks See you next month for a new summary of my activities.

5 comments Liked this article? Click here. My blog is Flattr-enabled.

Russ Allbery: Review: Asimov's, June 2011

Review: Asimov's Science Fiction, June 2011
Editor: Sheila Williams
Issue: Volume 35, No. 6
ISSN: 1065-2698
Pages: 112
Sheila Williams's editorial is about a new ebook-only anthology of Asimov's stories, with some discussion of electronic sales. This sort of business editorial is what she's best at writing, although in this case I'm far enough in my reading that I'm reading it through a bit of a time machine. Silverberg's essay is about popular bubbles and is mildly interesting, although I've read quite a lot about this at greater length. I think he misses the plot at the end, though, where he brings in the 2008 collapse and works on focusing the blame on speculators rather than those selling the fraudulent investments. James Patrick Kelly has a surprisingly graceful introduction to fanfic that does a better job at conveying the underlying enthusiasm than most mainstream coverage. The review column this month is Peter Heck's workman-like survey of recently released books of interest. "The Cold Step Beyond" by Ian R. MacLeod: Most of MacLeod's writing I've read has focused on lush world-building, usually with a mix of magic and industrial revolution. This felt like a departure: a character study of a heavily-modified warrior and monster hunter against a clearly science-fictional background. The protagonist has no memory of her past beyond her training in a religious institution, and is as much machine as woman. She is on one of her first missions to hunt dangerous monsters. But she's given almost no direction, and rather than encountering a monster, she encounters a feral woman with an ancient ray gun. MacLeod has a very SFnal twist waiting in the wings, but what I liked best were the character dynamics and the interplay of duty and self-determination. (8) "All the News That's Fit" by Carol Emshwiller: This story follows Emshwiller's normal pattern of telling the story of an outsider and representative of outsider culture trying to understand a society, but her protagonist here is both more alone and more proactive than her average protagonist. She lives in a distant mountain village only rarely visited by a traveling reporter who brings news from outside. When he doesn't come, she goes out looking for him, at first to rescue him if he ran into trouble and later to find the news herself. Much of the story has the reader worrying nervously for her as she observes the large city while clearly having no idea how to live there. The ending provides an interesting and (at least for me) unexpected twist on the rural person coming to the big city, but it was marred by a level of anger from the protagonist that I never thought was well-motivated. (6) "Walking Stick Fires" by Alan DeNiro: There is a science fiction subgenre that, for lack of a better term, I'll call post-apocalyptic surrealism. This is an entry in that strange subgenre, featuring two alien bikers having mostly unmotivated adventures with strange objects and even stranger creatures on an Earth that's apparently been invaded and mostly devastated by multiple parties. Like a lot of post-apocalyptic surrealism, it never makes much sense. It reminded me of Rucker but with a bit more characterization and less hand-waving math. (4) "Apocalypse Daily" by Felicity Shoulders: I really liked the premise for this story. The protagonist is a concept designer for a social-oriented MMORPG, but one where only scores and resources carry over rather than the whole world. The goal of the game is survival: the world ends regularly, globally or locally, in a huge variety of ways, and one scores points and earns resources for further iterations by finding ways to survive that catastrophe. The game is structured to encourage cooperation, at least at first. But Katrina gets into a mildly paranoid workplace conflict with someone else on her team, leading to some significant changes in the underlying expectations of the game. Unfortunately, while the background and idea are quite solid, the direction the story goes is predictable pessimism, the conclusion doesn't say much that I found interesting, and I ended up detesting the protagonist. I would have preferred a more unique psychological twist. (6) "The Fighter" by Colin P. Davies: This is a rather short story, essentially an idea study in the context of a police stop of a futuristic entertainment fighter. This is another one of those depressing takes on a common SF trope. I didn't entirely follow the world-building, but it was all dark and pointless enough that I didn't really want to dig further. (4) "Kiss Me Twice" by Mary Robinette Kowal: I've read a few police procedurals featuring AIs before, but this is by far the best one I've read. Huang is a detective, and like all the detectives he makes extensive use of the police department AI. Unlike most of the detectives, though, he treats her as a full partner and close to human, and she reciprocates with a Mae West persona and a closer relationship with him than most of the department. They're in the middle of investigating a murder when there is a raid on the police department and the AI herself is stolen. The rest of the story entangles their investigation with the theft of the AI. The police AI is, of course, restored from backup, but that's the first time that's ever happened to her, and she's shaken by it, and more disturbed by the knowledge that there are two copies of her running. The mystery angle of the story is reasonably satisfying, but the AI characterization and Huang's careful feeling-out of their relationship and appropriate behavior on his part is the best part of the story. This is a great story: long, satisfying, and thoroughly excellent. I'd happily read more fiction in this universe. (8) Rating: 7 out of 10

28 February 2013

Niels Thykier: Wheezy release progress (February)

About 5-6 weeks ago, I wrote about the Wheezy release progress, so it is about time for another update. According to UDD, we are down to 204 RC bugs (down from 249, since my last post). It is not quite the 2.4 RC bugs per day actually it is about 1.1 RC bug a day. Unfortunately, we do not appear to be fixing bugs faster than we are reporting them at the moment. If you look at Richard Hartmann s post from last week, then we had 206 RC bugs left. Even worse, we appear to have regressed between week 7 and 8 of this year (194 to 206). If you want the Wheezy release to happen soon, please consider helping us by providing bug fixes that comply with the Wheezy freeze policy. If the pace of RC bug fixes do not pick up, the alternative is that the release team deals with the bugs. Note that deals generally falls into one of 2 categories. Either we defer/ignore the problem for Wheezy[1] or we remove affected packages from Wheezy/testing. Particularly, if we have to remove packages, they may take reverse dependencies with them as collateral damage. I do not like these tools anymore than you do. But if the RC bugs fixes are not coming in, it is the only two tools we have left. [1] Meaning that at best the bug fix will occur at Wheezy point release at worst, not at all.

Erich Schubert: ELKI data mining in 2013

ELKI, the data mining framework I use for all my research, is coming along nicely, and will see continued progress in 2013. The next release is scheduled for SIGMOD 2013, where we will be presenting the novel 3D parallel coordinates visualization we recently developed. This release will bear the version number 0.6.0.
Version 0.5.5 of ELKI is in Debian unstable since december (Version 0.5.0 will be in the next stable release) and Ubuntu raring. The packaged installation can share the dependencies with other Debian packages, so they are smaller than the download from the ELKI web site.
If you are developing cluster analysis or outlier detection algorithm, I would love to see them contributed to ELKI. If I get a clean and well-integrated code by mid june, your algorithm could be included in the next release, too. Publishing your algorithms in source code in a larger framework such as ELKI will often give you more citations. Because it is easier to compare with your algorithm then and to try it on new problems. And, well, citations counts are a measure that administration loves to judge researchers ...
So what else is happening with ELKI:
  • The new book "Outlier Analysis" by C. C. Aggarwal mentions ELKI for visual evaluation of outlier results as well as in the "Resources for the Practioner" section and cites around 10 publications closely related to ELKI.
  • Some classes for color feature extraction of ELKI have been contributed to jFeatureLib, a Java library for feature detection in image data.
  • I'd love to participate in the Google Summer of Code, but I need a contact at Google to "vouch" for the project, otherwise it is hard to get in. I've been sending a couple of emails, but so far have not heard back much yet.
  • As the performance of SVG/Batik is not too good, I'd like to see more OpenGL based visualizations. This could also lead to an Android based version for use on tablets.
  • As I'm not an UI guy, I would love to have someone make a fancier UI that still exposes all the rich functions we have. The current UI is essentially an automatically generated command line builder - which is nice, as new functionality shows up without the need to modify UI code. It's good for experienced users like me, but hard for beginners to get started.
  • I'd love to see integration of ELKI with e.g. OpenRefine / Google Refine to make it easier to do appropriate data cleaning and preprocessing
  • There is work underway for a distributed version running on Hadoop/YARN.

Dirk Eddelbuettel: inline 0.3.11

A maintenance release of inline is now on CRAN, and is being uploaded to Debian. The release fixes two minor bugs kindly reported by users. As the two previous releases appear to not have been announced here, their NEWS entries are included as well.
Changes in inline version 0.3.11 (2013-02-26)
  • Fix bug in cfunction for .C convention with raw vectors.
  • Correct cfunction to use .Platform$dynlib.ext as the file extension for the library file (unless on Windows).
  • Allow rcpp wrapper to pass another plugin (as eg RcppArmadillo)
Changes in inline version 0.3.10 (2012-10-03)
  • getDynLib() error message corrected as suggested by Yasir Suhail
  • Added rcpp() wrapper for cxxfunction() which sets plugin="Rcpp"
  • Converted NEWS to NEWS.Rd
  • New maintainer, after having coordinated releases (along with Romain) since 0.3.5 in June 2010
Changes in inline version 0.3.9 (2012-10-02)
  • Uncoordinating hijacking of package by CRAN maintainers with a single word change in cfunction.R to prevent an error under an unreleased version of R
Courtesy of CRANberries, there is also a diffstat report for the most recent release. A few more details are available at the R-Forge page.

26 February 2013

Ingo Juergensmann: Xen problems with VMs on 2.6.32-5-xen-amd64?

On Saturday there were some updates for Squeeze/Stable, for example a new/updated version of Xen hypervisor and kernels were downloaded and installed:
gate:~# dir /var/cache/apt/archives/
base-files_6.0squeeze7_amd64.deb                 libxenstore3.0_4.0.1-5.6_amd64.deb
bind9-host_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb   linux-base_2.6.32-48_all.deb
dbus_1.2.24-4+squeeze2_amd64.deb                 linux-image-2.6.32-5-amd64_2.6.32-48_amd64.deb
dbus-x11_1.2.24-4+squeeze2_amd64.deb             linux-image-2.6.32-5-xen-amd64_2.6.32-48_amd64.deb
firmware-linux-free_2.6.32-48_all.deb            lock
gzip_1.3.12-9+squeeze1_amd64.deb                 openssh-client_1%3a5.5p1-6+squeeze3_amd64.deb
host_1%3a9.7.3.dfsg-1~squeeze9_all.deb           openssh-server_1%3a5.5p1-6+squeeze3_amd64.deb
libbind9-60_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb  openssl_0.9.8o-4squeeze14_amd64.deb
libcups2_1.4.4-7+squeeze3_amd64.deb              partial
libdbus-1-3_1.2.24-4+squeeze2_amd64.deb          perl_5.10.1-17squeeze5_amd64.deb
libdbus-glib-1-2_0.88-2.1+squeeze1_amd64.deb     perl-base_5.10.1-17squeeze5_amd64.deb
libdns69_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb     perl-modules_5.10.1-17squeeze5_all.deb
libisc62_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb     ssh_1%3a5.5p1-6+squeeze3_all.deb
libisccc60_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb   tzdata_2012g-0squeeze1_all.deb
libisccfg62_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb  xen-hypervisor-4.0-amd64_4.0.1-5.6_amd64.deb
libldap-2.4-2_2.4.23-7.3_amd64.deb               xen-linux-system-2.6.32-5-xen-amd64_2.6.32-48_amd64.deb
liblwres60_1%3a9.7.3.dfsg-1~squeeze9_amd64.deb   xenstore-utils_4.0.1-5.6_amd64.deb
libperl5.10_5.10.1-17squeeze5_amd64.deb          xen-utils-4.0_4.0.1-5.6_amd64.deb
libssl0.9.8_0.9.8o-4squeeze14_amd64.deb
Unfortunately this update appears to be problematic on my Xen hosting server. This night it happened the second time that some of the virtual network interfaces disappeared or turned out to be non-working. For example I have two VMs: one running the webserver and one running the databases. Between these two VMs there's a bridge on the dom0 and both VMs have a VIF to that (internal) bridge. What happens is that this bridge becomes inaccessible from within the webserver VM. Sadly there's not much to see in the log files. I just spotted this on dom0:
Feb 26 01:01:29 gate kernel: [12697.907512] vif3.1: Frag is bigger than frame.
Feb 26 01:01:29 gate kernel: [12697.907550] vif3.1: fatal error; disabling device
Feb 26 01:01:29 gate kernel: [12697.919921] xenbr1: port 3(vif3.1) entering disabled state
Feb 26 01:22:00 gate kernel: [13928.644888] vif2.1: Frag is bigger than frame.
Feb 26 01:22:00 gate kernel: [13928.644920] vif2.1: fatal error; disabling device
Feb 26 01:22:00 gate kernel: [13928.663571] xenbr1: port 2(vif2.1) entering disabled state
Feb 26 01:40:44 gate kernel: [15052.629280] vif7.1: Frag is bigger than frame.
Feb 26 01:40:44 gate kernel: [15052.629314] vif7.1: fatal error; disabling device
Feb 26 01:40:44 gate kernel: [15052.641725] xenbr1: port 6(vif7.1) entering disabled state
This corresponds to the number of VMs having lost their internal connection to the bridge. On the webserver VM I see this output:
Feb 26 01:59:01 vserv1 kernel: [16113.539767] IPv6: sending pkt_too_big to self
Feb 26 01:59:01 vserv1 kernel: [16113.539794] IPv6: sending pkt_too_big to self
Feb 26 02:30:54 vserv1 kernel: [18026.407517] IPv6: sending pkt_too_big to self
Feb 26 02:30:54 vserv1 kernel: [18026.407546] IPv6: sending pkt_too_big to self
Feb 26 02:30:54 vserv1 kernel: [18026.434761] IPv6: sending pkt_too_big to self
Feb 26 02:30:54 vserv1 kernel: [18026.434787] IPv6: sending pkt_too_big to self
Feb 26 03:39:16 vserv1 kernel: [22128.768214] IPv6: sending pkt_too_big to self
Feb 26 03:39:16 vserv1 kernel: [22128.768240] IPv6: sending pkt_too_big to self
Feb 26 04:39:51 vserv1 kernel: [25764.250170] IPv6: sending pkt_too_big to self
Feb 26 04:39:51 vserv1 kernel: [25764.250196] IPv6: sending pkt_too_big to self
Rebooting the VMs will result in a non-working VM as it will get paused on creation and Xen scripts complain about not working hotplug scripts and Xen logs shows this:
[2013-02-25 13:06:34 5470] DEBUG (XendDomainInfo:101)
XendDomainInfo.create(['vm', ['name', 'vserv1'], ['memory', '2048'],
['on_poweroff', 'destroy'], ['on_reboot', 'restart'], ['on_crash',
'restart'], ['on_xend_start', 'ignore'], ['on_xend_stop', 'ignore'],
['vcpus', '2'], ['oos', 1], ['bootloader', '/usr/lib/xen-4.0/bin/pygrub'],
['bootloader_args', ''], ['image', ['linux', ['root', '/dev/xvdb '],
['videoram', 4], ['tsc_mode', 0], ['nomigrate', 0]]], ['s3_integrity', 1],
['device', ['vbd', ['uname', 'phy:/dev/lv/vserv1-boot'], ['dev', 'xvda'],
['mode', 'w']]], ['device', ['vbd', ['uname', 'phy:/dev/lv/vserv1-disk'],
['dev', 'xvdb'], ['mode', 'w']]], ['device', ['vbd', ['uname',
'phy:/dev/lv/vserv1-swap'], ['dev', 'xvdc'], ['mode', 'w']]], ['device',
['vbd', ['uname', 'phy:/dev/lv/vserv1mirror'], ['dev', 'xvdd'], ['mode',
'w']]]])
[2013-02-25 13:06:34 5470] DEBUG (XendDomainInfo:2508)
XendDomainInfo.constructDomain
[2013-02-25 13:06:34 5470] DEBUG (balloon:220) Balloon: 2100000 KiB free;
need 16384; done.
[2013-02-25 13:06:34 5470] DEBUG (XendDomain:464) Adding Domain: 39
[2013-02-25 13:06:34 5470] DEBUG (XendDomainInfo:2818)
XendDomainInfo.initDomain: 39 256
[2013-02-25 13:06:34 5781] DEBUG (XendBootloader:113) Launching bootloader
as ['/usr/lib/xen-4.0/bin/pygrub', '--args=root=/dev/xvdb  ',
'--output=/var/run/xend/boot/xenbl.6040', '/dev/lv/vserv1-boot'].
[2013-02-25 13:06:39 5470] DEBUG (XendDomainInfo:2845)
_initDomain:shadow_memory=0x0, memory_static_max=0x80000000,
memory_static_min=0x0.
[2013-02-25 13:06:39 5470] INFO (image:182) buildDomain os=linux dom=39
vcpus=2
[2013-02-25 13:06:39 5470] DEBUG (image:721) domid	    = 39
[2013-02-25 13:06:39 5470] DEBUG (image:722) memsize	    = 2048
[2013-02-25 13:06:39 5470] DEBUG (image:723) image	    =
/var/run/xend/boot/boot_kernel.xj7W_t
[2013-02-25 13:06:39 5470] DEBUG (image:724) store_evtchn   = 1
[2013-02-25 13:06:39 5470] DEBUG (image:725) console_evtchn = 2
[2013-02-25 13:06:39 5470] DEBUG (image:726) cmdline	    =
root=UUID=ed71a39f-fd2e-4035-8557-493686baa151 ro root=/dev/xvdb
[2013-02-25 13:06:39 5470] DEBUG (image:727) ramdisk	    =
/var/run/xend/boot/boot_ramdisk.QavuAo
[2013-02-25 13:06:39 5470] DEBUG (image:728) vcpus	    = 2
[2013-02-25 13:06:39 5470] DEBUG (image:729) features	    =
[2013-02-25 13:06:39 5470] DEBUG (image:730) flags	    = 0
[2013-02-25 13:06:39 5470] DEBUG (image:731) superpages     = 0
[2013-02-25 13:06:40 5470] INFO (XendDomainInfo:2367) createDevice: vbd :
 'uuid': '04d99772-cf27-aecf-2d1b-c73eaf657410', 'bootable': 1, 'driver':
'paravirtualised', 'dev': 'xvda', 'uname': 'phy:/dev/lv/vserv1-boot',
'mode': 'w' 
[2013-02-25 13:06:40 5470] DEBUG (DevController:95) DevController: writing
 'virtual-device': '51712', 'device-type': 'disk', 'protocol':
'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend':
'/local/domain/0/backend/vbd/39/51712'  to
/local/domain/39/device/vbd/51712.
[2013-02-25 13:06:40 5470] DEBUG (DevController:97) DevController: writing
 'domain': 'vserv1', 'frontend': '/local/domain/39/device/vbd/51712',
'uuid': '04d99772-cf27-aecf-2d1b-c73eaf657410', 'bootable': '1', 'dev':
'xvda', 'state': '1', 'params': '/dev/lv/vserv1-boot', 'mode': 'w',
'online': '1', 'frontend-id': '39', 'type': 'phy'  to
/local/domain/0/backend/vbd/39/51712.
[2013-02-25 13:06:40 5470] INFO (XendDomainInfo:2367) createDevice: vbd :
 'uuid': 'e46cb89f-3e54-41d2-53bd-759ed6c690d2', 'bootable': 0, 'driver':
'paravirtualised', 'dev': 'xvdb', 'uname': 'phy:/dev/lv/vserv1-disk',
'mode': 'w' 
[2013-02-25 13:06:40 5470] DEBUG (DevController:95) DevController: writing
 'virtual-device': '51728', 'device-type': 'disk', 'protocol':
'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend':
'/local/domain/0/backend/vbd/39/51728'  to
/local/domain/39/device/vbd/51728.
[2013-02-25 13:06:40 5470] DEBUG (DevController:97) DevController: writing
 'domain': 'vserv1', 'frontend': '/local/domain/39/device/vbd/51728',
'uuid': 'e46cb89f-3e54-41d2-53bd-759ed6c690d2', 'bootable': '0', 'dev':
'xvdb', 'state': '1', 'params': '/dev/lv/vserv1-disk', 'mode': 'w',
'online': '1', 'frontend-id': '39', 'type': 'phy'  to
/local/domain/0/backend/vbd/39/51728.
[2013-02-25 13:06:40 5470] INFO (XendDomainInfo:2367) createDevice: vbd :
 'uuid': 'e2d61860-7448-1843-3935-6b63c5d2878e', 'bootable': 0, 'driver':
'paravirtualised', 'dev': 'xvdc', 'uname': 'phy:/dev/lv/vserv1-swap',
'mode': 'w' 
[2013-02-25 13:06:40 5470] DEBUG (DevController:95) DevController: writing
 'virtual-device': '51744', 'device-type': 'disk', 'protocol':
'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend':
'/local/domain/0/backend/vbd/39/51744'  to
/local/domain/39/device/vbd/51744.
[2013-02-25 13:06:40 5470] DEBUG (DevController:97) DevController: writing
 'domain': 'vserv1', 'frontend': '/local/domain/39/device/vbd/51744',
'uuid': 'e2d61860-7448-1843-3935-6b63c5d2878e', 'bootable': '0', 'dev':
'xvdc', 'state': '1', 'params': '/dev/lv/vserv1-swap', 'mode': 'w',
'online': '1', 'frontend-id': '39', 'type': 'phy'  to
/local/domain/0/backend/vbd/39/51744.
[2013-02-25 13:06:40 5470] INFO (XendDomainInfo:2367) createDevice: vbd :
 'uuid': 'd314a46e-1ce9-0e8d-b009-3f08e29735f5', 'bootable': 0, 'driver':
'paravirtualised', 'dev': 'xvdd', 'uname': 'phy:/dev/lv/vserv1mirror',
'mode': 'w' 
[2013-02-25 13:06:40 5470] DEBUG (DevController:95) DevController: writing
 'virtual-device': '51760', 'device-type': 'disk', 'protocol':
'x86_64-abi', 'backend-id': '0', 'state': '1', 'backend':
'/local/domain/0/backend/vbd/39/51760'  to
/local/domain/39/device/vbd/51760.
[2013-02-25 13:06:40 5470] DEBUG (DevController:97) DevController: writing
 'domain': 'vserv1', 'frontend': '/local/domain/39/device/vbd/51760',
'uuid': 'd314a46e-1ce9-0e8d-b009-3f08e29735f5', 'bootable': '0', 'dev':
'xvdd', 'state': '1', 'params': '/dev/lv/vserv1mirror', 'mode': 'w',
'online': '1', 'frontend-id': '39', 'type': 'phy'  to
/local/domain/0/backend/vbd/39/51760.
[2013-02-25 13:06:40 5470] DEBUG (XendDomainInfo:3400) Storing VM details:
 'on_xend_stop': 'ignore', 'shadow_memory': '0', 'uuid':
'04541225-6d3c-3cae-a4c4-0b6d4ccfac7a', 'on_reboot': 'restart',
'start_time': '1361794000.37', 'on_poweroff': 'destroy', 'bootloader_args':
'', 'on_xend_start': 'ignore', 'on_crash': 'restart', 'xend/restart_count':
'0', 'vcpus': '2', 'vcpu_avail': '3', 'bootloader':
'/usr/lib/xen-4.0/bin/pygrub', 'image': "(linux (kernel ) (args
'root=/dev/xvdb  ') (superpages 0) (tsc_mode 0) (videoram 4) (pci ())
(nomigrate 0) (notes (HV_START_LOW 18446603336221196288) (FEATURES
'!writable_page_tables pae_pgdir_above_4gb') (VIRT_BASE
18446744071562067968) (GUEST_VERSION 2.6) (PADDR_OFFSET 0) (GUEST_OS linux)
(HYPERCALL_PAGE 18446744071578882048) (LOADER generic) (SUSPEND_CANCEL 1)
(PAE_MODE yes) (ENTRY 18446744071584289280) (XEN_VERSION xen-3.0)))",
'name': 'vserv1' 
[2013-02-25 13:06:40 5470] DEBUG (XendDomainInfo:1804) Storing domain
details:  'console/ring-ref': '2143834', 'image/entry':
'18446744071584289280', 'console/port': '2', 'store/ring-ref': '2143835',
'image/loader': 'generic', 'vm':
'/vm/04541225-6d3c-3cae-a4c4-0b6d4ccfac7a',
'control/platform-feature-multiprocessor-suspend': '1',
'image/hv-start-low': '18446603336221196288', 'image/guest-os': 'linux',
'cpu/1/availability': 'online', 'image/virt-base': '18446744071562067968',
'memory/target': '2097152', 'image/guest-version': '2.6', 'image/pae-mode':
'yes', 'description': '', 'console/limit': '1048576', 'image/paddr-offset':
'0', 'image/hypercall-page': '18446744071578882048',
'image/suspend-cancel': '1', 'cpu/0/availability': 'online',
'image/features/pae-pgdir-above-4gb': '1',
'image/features/writable-page-tables': '0', 'console/type': 'xenconsoled',
'name': 'vserv1', 'domid': '39', 'image/xen-version': 'xen-3.0',
'store/port': '1' 
[2013-02-25 13:06:40 5470] DEBUG (DevController:95) DevController: writing
 'protocol': 'x86_64-abi', 'state': '1', 'backend-id': '0', 'backend':
'/local/domain/0/backend/console/39/0'  to
/local/domain/39/device/console/0.
[2013-02-25 13:06:40 5470] DEBUG (DevController:97) DevController: writing
 'domain': 'vserv1', 'frontend': '/local/domain/39/device/console/0',
'uuid': 'c8819aed-c78f-02b8-0ef7-1600abd15add', 'frontend-id': '39',
'state': '1', 'location': '2', 'online': '1', 'protocol': 'vt100'  to
/local/domain/0/backend/console/39/0.
[2013-02-25 13:06:40 5470] DEBUG (XendDomainInfo:1891)
XendDomainInfo.handleShutdownWatch
[2013-02-25 13:06:40 5470] DEBUG (DevController:139) Waiting for devices
vif2.
[2013-02-25 13:06:40 5470] DEBUG (DevController:139) Waiting for devices
vif.
[2013-02-25 13:06:40 5470] DEBUG (DevController:139) Waiting for devices
vscsi.
[2013-02-25 13:06:40 5470] DEBUG (DevController:139) Waiting for devices
vbd.
[2013-02-25 13:06:40 5470] DEBUG (DevController:144) Waiting for 51712.
[2013-02-25 13:06:40 5470] DEBUG (DevController:628) hotplugStatusCallback
/local/domain/0/backend/vbd/39/51712/hotplug-status.
From my point of view, either Xen hypervisor or the kernel seems to be broken, but it's hard to tell for me. Maybe it would be easier to update the system from Squeeze to Wheezy and get rid off this problem that way? On the other hand this would solve the problem. Are there any other people experiencing problems with that latest update of Xen and kernel? UPDATE: Bug #701744 filed. UPDATE 2: Downgrading kernel and hypervisor on dom0 to the following packages from snapshot.debian.org seems to have solved that problem.
  • xen-hypervisor-4.0-amd64_4.0.1-5.4_amd64.deb
  • linux-image-2.6.32-5-xen-amd64_2.6.32-46_amd64.deb
Note that I haven't tested yet with the newest kernel update from DSA-2632-1. UPDATE 3: After running with older versions of hypervisor and kernel I now have upgraded the hypervisor to xen-hypervisor-4.0-amd64_4.0.1-5.6_amd64.deb and rebooted. Let's see whether it's running stable or not. If not it's the hypervisor, if yes, it's the kernel. UPDATE 4: apparently it's the kernel which is buggy. And the kernel from DSA-2632-1 is affected as well. So, the current workaround is to downgrad to linux-image-2.6.32-5-xen-amd64_2.6.32-46_amd64.deb.
Kategorie:

25 February 2013

Russ Allbery: Review: The Hare with Amber Eyes

Review: The Hare with Amber Eyes, by Edmund de Waal
Publisher: Farrar, Straus and Giroux
Copyright: 2010
ISBN: 0-374-10597-9
Format: Kindle
Pages: 351
A netsuke ( ) is a miniature sculpture, often made from ivory or boxwood, generally rounded and no larger than a matchbox. They were a part of Japanese clothing starting from the 17th century: traditional garments had no pockets, but people needed a way to carry small objects around with them, so they wore pouches or small boxes attached to a cord. The cord could then be passed under the sash of the garment and be held in place there by a netsuke attached to the end of the cord, since the netsuke was too large to slip under the sash. The hare with amber eyes is one of 264 netsuke. It, and all the rest, belonged to Edmund de Waal's uncle Iggie. They sat in a vitrine in his home in Japan, where de Waal went for a two-year scholarship and was working on a book on japonisme: the passionate and creative Western misunderstanding of Japan. When Iggie later died in 1991, de Waal inherited the collection. This book is his attempt to tell their history. The Hare with Amber Eyes falls into the genre of family memoir or biography, but it's one told from an unusual angle. Edmund de Waal is a potter by profession: an artist who creates physical objects that then pass from his studio out into the world. The physicality of the netsuke their occupation of time and space and their passsage through the world fascinates him, and he tries to express that fascination to the reader and uses it as a focal point for the memoir. He tells, here, the story of his own family, but he does so via the netsuke, following them from their purchase in Paris in 1871 through one branch of the family and then another until they reach Iggie and, finally, him. Along the way, he tries to put them into context. At first, the context is primarily artistic, and the early parts of this book are something of a tour of late 19th century artistic culture in Paris. But, as the story continues, the connection becomes more emotional and the context becomes more complicated. Anyone who is a fan of the TV program Antiques Roadshow or any similar effort to trace the history of specific objects will see the immediate appeal in this presentation. Antiques are fascinating in their mute path through lives and history; attempting to reconstruct the rooms they've sat in and the conversations they've witnessed is something I've always found intriguing. But it's also very difficult to do. Most of our objects, and most owners of objects, simply don't have that sort of documentation. I was wondering, at the start of this book, how de Waal was going to manage to construct his history. The answer is that his family is, or at least was, famous enough to have letters, papers, and history in libraries throughout Europe. That's what makes the parts of the book before Iggie's living memory possible. It's a bit disappointing, if unsurprising; de Waal doesn't have any special technique to find hidden history, and most objects could never have this sort of history written about them. The first owner of the netsuke in his family is Charles Ephrussi, a critic, historian, and art historian who was a patron of the Impressionists and who was important enough in Parisian art culture that he is the inspiration of a character in Marcel Proust's Remembrance of Things Past. This means that quite a lot of this book is about people who moved through social circles and lived in ways that I find quite foreign, which made it hard to invest too much in the first part of the book. The Ephrussi family was originally from Odessa and made a fortune as grain merchants some time prior to the start of the history, and by the time de Waal opens their story, they have vast inherited wealth. There is no sign in the book that Charles needs to work in any way. The first part of the book paints a picture of the culture of art patronage partly via descriptions of Charles's rooms and personal collection, to which the netsuke were added during the surge in interest in Japanese art in Europe produced by the opening of Japan during the Meiji period. It's interesting, but for me in a distant and intellectual way. (People with a deeper interest in Impressionist art or other late 19th century art may have a different reaction.) De Waal writes at length at the start of this book about avoiding sentimentality and trying to take an honest and immersive look at the history of the netsuke. The emotional intensity that he brings to this goal seemed a bit odd, since little in the early part of the book seems to tempt towards sentimentality. But then the Dreyfus Affair enters Charles's story, followed by his gift of the vitrine and the netsuke to a cousin in Austria as a marriage gift, and I started to realize the implications of following a possession of a prominent Jewish family in Europe during the first half of the twentieth century. This is the sort of non-fiction book that you can spoil, so I won't give a detailed description of the rest of the book. But there is a specific event in the history of the netsuke that de Waal tells that would be the story most people would start with, and I now understand the sentimentality that de Waal was trying to avoid. It's a great story, but one of the things I like about this book is that de Waal puts it into a broader context and complicates the story. He includes the long periods of time when the netsuke were mostly ignored, he ties them into the history of western obsession with (and exploitation of) Japanese art, and he shows how the sturdy nature of the netsuke, meant for physical handling rather than display, led to them creating the emotional connection that made them important to the family. It's a book that deals with anti-Semitism with accuracy and specifics without being either lurid or overwhelming. It's a book that traces a line through one of the most overwritten and well-trod areas of history and manages to stay on its own path. It's a book that made me angry, and made me cry, and then made me step back and think about those emotions in a broader context and think about the benefits of leaving those emotions behind. I've never read a book quite like this before. It's slow in places (particularly if one is not very interested in art history), utterly compelling in others, and carefully avoids any simple themes or grand stories. It's entangled with some of the darkest events of recent history, but de Waal stays tightly focused on the family through the lens of the netsuke, which is valuable for keeping the story from drifting into well-worn and less unique material. I was not as completely taken by this book as the review that got me to buy it (it's an excellent review, better than this one). But I'm glad that I read it, and I feel like I now know more about the world. Rating: 7 out of 10

24 February 2013

Andrew Pollock: [tech] On owning a Nissan Leaf

I'll soon be disposing of the Nissan Leaf that we leased a few months ago, so I thought it a useful exercise to write about my experiences with it. I am not a car man. I am a gadget man. For me, driving is a means to an end, and I'm much more interested in what I call the "cabin" experience, than the "driving" experience, so this is going to slanted much more that way. That said, I found the Nissan Leaf a fun car to drive, from my limited experiences of having fun driving cars. I liked how responsive it was when you put your foot down. It has two modes of operation, "D" and "Eco". I've actually taken to driving it in "Eco" mode most of the time, as it squeezes more range out of the batteries, but occasionally I'll pop it back into "D" to have a bit of fun. The main difference between the two modes, from a driving perspective, is it seems to limit the responsiveness of the accelerator. In "Eco" mode it feels more like you're driving in molasses, whereas in "D" mode, when you put your foot down, it responses instantly. "D" is great for dragging people off at the lights. It's a very zippy little car in "D" mode. It feels lighter. I've noticed that it has a bit of a penchant for over steering. Or maybe that's just my driving. If I have floored it a bit to take a right turn into oncoming traffic, I've noticed slight over steering. That's about all the driving type "real car stuff" I'll talk about. Now to the driver's "cabin experience". It's absolutely fabulous. I love sitting in the driver's seat of this car. Firstly, the seat itself is heated (in fact all of them are). As is the steering wheel. Nissan has gone to great lengths to allow you to avoid needing to run the car's heating system to heat the car, as doing so immediately drops at least 10 miles off the range. Unfortunately I found the windscreen had a tendency to fog up in the winter rainy periods, so I'd have to intermittently fire up the air conditioning to defog the windscreen. Of course, in the summer months, you're going to want to run the AC to cool down, so the range hit in that situation is unavoidable. I've only had this car from late Autumn until late Winter so far, so that hasn't been an issue I've had to contend with. The dashboard is all digital, and looks relatively high tech, which appeals to my inner geek. It's a dashboard. It tells you stuff. The stuff you'd expect it to tell you. Enough said. The audio system is nice. It supports Bluetooth audio, so one can be streaming Pandora from one's phone, through the sound system, for example. Or listening to audio from the phone. There's also a USB port, and it will play various audio files from USB storage. I found the way it chose to sort the files on a USB stick to be slightly surprising though. I haven't invested the time to figure out how I should be naming files so that they play in the order I expect them to play. The ability to play from USB storage compensates nicely for the fact that it only has a single CD player. (We have a 6 disc stacker in our 2006 Prius). The car also came with a 3 month free trial of Sirius XM satellite radio. This was fun. The only dance music FM station in the Bay Area has a weak signal in Mountain View, and I hate dance music with static, whereas there was an excellent electronic music station that I could listen to in glorious high definition. As long as I wasn't driving under a bridge. There's no way I'd pay money for satellite radio, but it was a fun gimmick to try out. The navigation system is really, really good. I haven't bothered using Google Maps on my phone at all. It gives such good spoken directions, that you don't even need to have the map on the screen. It names all the streets. I couldn't figure out a way to get distances in metric. The telematics service, Carwings, is probably my favourite feature. This is what really makes it feel like a car of the future. Through the companion Android application, I can view the charging status (if it's plugged in) or just check the available range (if it's not plugged in). From a web browser, I can plan a route, and push the route to the vehicle's navigation system. If the car is plugged in, I can also remotely turn on the vehicle's climate control system, pre-warming or cooling the car. It's a little thing, but the door unlocking annoyed me a little bit. I'm used to the Prius, where if you unlock the boot (that's trunk, Americans), or the front passenger door, all the doors unlocked. This was a convenient way of unlocking the car for multiple people as you approached it. With the Leaf, unlocking the boot only unlocks the boot. Unlocking the front passenger door only unlocks that door. It requires a second unlock action to unlock all the doors. I've found this to be slightly cumbersome when trying to unlock the car for everyone all at once, quickly (like when it's raining). Another minor annoyance is the headlights. I've gotten into the habit of driving with the headlights on all the time, because I believe it's safer. In the Prius, one could just leave the headlights switched to the "on" position, and they'd turn off whenever the driver's door was opened after the car was switched off. If you try that in the Leaf, the car beeps at you to warn you you've left the headlights on. It has an "auto" mode, where the car will choose when to turn the headlights on, based on ambient light conditions. In that case, when you turn the car off, it'll leave the headlights on for a configurable period of time and then turn them off. This is actually slightly unsettling, because it makes you think you've left your headlights on. The default timeout was quite long as well, something like 60 seconds. The way multiple Bluetooth phones are handled is just as annoying in the Leaf as it is in the Prius, which disappoints me, given 6 years have passed. The way I'd like to see multiple phones handled is the car chooses to pair with whichever phone is in range, or if multiple phones are in range, it asks or uses the last one it paired with. In reality, it tries to pair with whatever it paired with last time, and one has to press far too many buttons to switch it to one of the other phones it knows about. Range anxiety is definitely something of a concern. It can be managed by using the GPS navigation for long or otherwise anxiety-inducing trips, and then one can compare the "miles remaining" on the GPS with the "miles remaining" on the battery range, and reassure oneself that they will indeed make it to where they're trying to go. The worst case I had was getting to within 5 miles of empty. The car started complaining at me. The charging infrastructure in the Bay Area is pretty good. There are plenty of charging stations in San Francisco and San Jose. I'm spoiled in that I have free charging available at work (including a building at the end of my street, so I never bothered with getting a home charger installed). I've almost never had to pay for charging, so it's been great while gas prices have been on the rise. The car's navigation system knows about some charging stations, so you can plan a route with the charging stations it knows about in mind. The only problem is it doesn't know if the charging stations are in use. If you use the ChargePoint Android app, you can see if the charging stations are in use, but then you have to do this cumbersome dance to find an available charging station and plug the address into the vehicle's navigation system. Of course, what can then happen is in the time you're driving to the charging station, someone else starts using it. I actually got bitten by this yesterday. Would I buy a Leaf again? Not as my sole car. It makes a perfect second/commuter car, but as a primary vehicle, it's too limited by its range. They're also ridiculously expensive in Australia, and Brisbane has absolutely no charging infrastructure.

23 February 2013

Guido G nther: Accelerometer and screen orientation in GNOME3

Following up on using Debian on the WeTab with GNOME Shell I had a look at automatically adjusting the screen orientation when rotating the device. The accelerometer is handled in the asus_laptop module since kernel 3.2, so this shows up in /proc/bus/input/devices:
I: Bus=0019 Vendor=0000 Product=0000 Version=0000
N: Name="Pegatron Lucid Tablet Accelerometer"
P: Phys=pega_accel/input0
S: Sysfs=/devices/platform/asus_laptop/input/input4
U: Uniq=
H: Handlers=event4 js0 
B: PROP=0
B: EV=9
B: ABS=7
Whenever the screen orientation changes the driver emits a udev event. The helper in /lib/udev/accelerometer processes it thanks to /lib/udev/rules.d/61-accelerometer.rules, determines the orientation and adds the ID_INPUT_ACCELEROMETER_ORIENTATION property to the event which can be checked with udevadm monitor --property:
UDEV  [5778.534171] change   /devices/platform/asus_laptop/input/input4 (input)
ABS=7
ACTION=change
DEVPATH=/devices/platform/asus_laptop/input/input4
EV=9
ID_FOR_SEAT=input-platform-asus_laptop
ID_INPUT=1
ID_INPUT_ACCELEROMETER=1
ID_INPUT_ACCELEROMETER_ORIENTATION=right-up
ID_PATH=platform-asus_laptop
ID_PATH_TAG=platform-asus_laptop
MODALIAS=input:b0019v0000p0000e0000-e0,3,kra0,1,2,mlsfw
NAME="Pegatron Lucid Tablet Accelerometer"
PHYS="pega_accel/input0"
PRODUCT=19/0/0/0
PROP=0
SEQNUM=1550
SUBSYSTEM=input
TAGS=:seat:
UDEV_LOG=3
USEC_INITIALIZED=6154288
This is captured by gnome-settings-daemon (>=3.2) that adjusts the screen rotation using xrandr2 accordingly. This can be checked by running gnome-settings-daemon with --debug. The old way of using xinput events is no longer supported. Unfortunately Debian's udev has a minor bug that misdetects the input device so this patch is needed to have the screen orientation changed automatically on the WeTab. This blog is flattr enabled.

Daniel Pocock: Checking authors.txt before migrating to github

Ensuring that contributors are correctly recognised for their work is a cornerstone of the free and open source software community. Here I present a convenient script to help. When projects migrate to git (and usually Github these days), there is usually quite a lot of enthusiasm to get up and running and the authors.txt file is not looked at closely, or not at all. Github does a great job of summarising the free software contributions of each member. To do this, it needs to cross-reference their commits with their email addresses. This possibility is not unique to Github though: as git is a distributed version control system, it is quite possible that other services will seek to mirror and report on code contributions in the future. authors.txt can be tedious For a large project with a long history, there may be more than 100 committers, many of them designated by SVN user IDs that are unique to the project and not easily mapped to the user email address. If the project is hosted on Google Code SVN, then Google often makes dummy gmail.com accounts for the committers - and many people don't even use those accounts for anything else, not even as an email account. Including these accounts in a git repository is a bad idea if it is not the person's preferred email address. If they don't have that address associated with their Github account, then their commits won't be easily attributed to them at all. Often, building an accurate authors.txt can be tedious, but here are some steps to simplify it:
  • Use a script like svn-authors-extract to quickly build a template file by scanning the SVN repository.
  • Scan email list history - if necessary, download the email archive as a mailbox, and use regular expressions to extract all email addresses
  • Find out if the SVN server keeps a list of email addresses with the SVN user IDs somewhere
  • Use searches in Google, LinkedIn and various other places to try and find out where the person is now
  • Finally, use the script below to test the results
Testing the results against Github automatically How do you know which email addresses are already matched to a Github account and which need further research? I've written a convenient script to help It creates a dummy repository with one commit per committer. Upload this dummy repository as a new project on Github, view the commit log page, and all the matched user IDs will be highlighted, so the missing ones will stand out and you can quickly focus your search for missing email addresses. I've tested this with the authors.txt file I'm building for the Sipdroid/Lumicall projects. This project comes from Google Code SVN and almost all the identities are obscured by gmail.com addresses that may or may not be the real/preferred email addresses. I created a dummy project on Github called sipdroid-users and pushed the dummy commits there so I can see how many of the contributors are correctly mapped to a Github account. A full copy of the script is below, you can also find it in the sync2git repository:
#!/bin/bash
set -e
if [ $# -lt 2 ];
then
  echo "$0 <authors file> <dest repo for push>"
  exit 1
fi
AUTHORS_FILE=" pwd /$1"
DEST_REPO="$2"
if [ ! -e "$ AUTHORS_FILE " ];
then
  echo "can't find $ AUTHORS_FILE , aborting"
  exit 1
fi
TMP_REPO= mktemp -d 
TARGET_FILE="testing.txt"
cd "$ TMP_REPO "
git init .
cat "$ AUTHORS_FILE "   while read ;
do
  full_info= echo "$ REPLY "   cut -f2 -d'='   cut -b2- 
  an=" echo "$ full_info "   sed -e "s/ <.*\$//\" "
  am=" echo "$ full_info "   sed -e "s/^.*</</\" "
  cn="$an"
  cm="$am"
  echo "$REPLY" >> "$ TARGET_FILE "
  git add "$ TARGET_FILE "
  export GIT_AUTHOR_NAME="$an"
  export GIT_AUTHOR_EMAIL="$am"
  export GIT_COMMITTER_NAME="$cn"
  export GIT_COMMITTER_EMAIL="$cm"
  git commit -m "Test adding $ REPLY "
done
git remote add origin "$ DEST_REPO "
git push -u origin master

22 February 2013

Francois Marier: Doing a fresh Debian/Ubuntu install without having to reconfigure everything

Taking advantage of a new hard drive, I decided to reinstall my Ubuntu (Precise 12.04.2) laptop from scratch so that I could easily enable full-disk encryption (a requirement for Mozilla work laptops). Reinstalling and reconfiguring everything takes a bit of time though, so here's the procedure I followed to keep the configuration to a minimum.

Install Ubuntu/Debian on the new drive While full-disk encryption is built into the graphical installer as of Ubuntu 12.10, it's only available through the alternate install CD in 12.04 and earlier. Using that CD, install Ubuntu on the new drive, making sure you select the "Encrypted LVM" option when you get to the partitioning step. (The procedure is the same if you use a Debian CD.) To make things easy, the first user you create should match exactly the one from your previous installation.

Copy your home directory Once the OS is installed on the new drive, plug the old one back in (in an external enclosure if you need to) and mount its root partition as a read-only drive on /mnt. Then log in as root (not just using sudo, actually login using the root user) and copy your home directory on the new drive:
rm -rf /home/*
cp -a /mnt/home/* /home/
Then you should be able to log in as your regular user.

Reinstall all packages The next step is to reinstall all of the packages you had installed on the old OS. But first of all, let's avoid having to answer all of the debconf questions we've already answered in the past:
rm -rf /var/cache/debconf
cp -a /mnt/var/cache/debconf /var/cache/
and set the debconf priority to the one you usually use (medium in my case):
dpkg-reconfigure debconf
Next, make sure you have access to all of the necessary repositories:
cp -a /mnt/etc/apt/sources.list /etc/apt/
cp -a /mnt/etc/apt/sources.list.d/* /etc/apt/sources.list.d/
apt-get update

Getting and setting the list of installed packages To get a list of the packages that were installed on the old drive, use dpkg on the old install:
mount -o remount,rw /mnt
chroot /mnt
dpkg --get-selections > /packages
exit
mount -o remount,ro -r /mnt
Use that list on the new install to reinstall everything:
dpkg --set-selections < /mnt/packages
apt-get dselect-upgrade

Selectively copy configuration files over Finally, once all packages are installed, you can selectively copy the config files from the old drive (in /mnt/etc) to the new (/etc/). In particular, make sure you include these ones:
cp -a /mnt/etc/alternatives/ /mnt/etc/default/ /etc/
(I chose not to just overwrite all config files with the old ones because I wanted to get rid of any cruft that had accumulated there and suspected that there might be some slight differences due to the fresh install of the distro.)

Reboot Once that's done, you should really give that box a restart to ensure that all service are using the right config files, but otherwise, that's it. Your personal data and config are back, all of your packages are installed and configured the way they were, and everything is fully encrypted!

20 February 2013

Vincent Bernat: lldpd 0.7.1

A few weeks ago, a new version of lldpd, a 802.1AB (aka LLDP) implementation for various Unices, has been released. LLDP is an industry standard protocol designed to supplant proprietary Link-Layer protocols such as EDP or CDP. The goal of LLDP is to provide an inter-vendor compatible mechanism to deliver Link-Layer notifications to adjacent network devices. In short, LLDP allows you to know exactly on which port is a server (and reciprocally). To illustrate its use, I have made a xkcd-like strip: xkcd-like strip for the use of LLDP If you would like more information about lldpd, please have a look at its new dedicated website. This blog post is an insight of various technical changes that have affected lldpd since its latest major release one year ago. Lots of C stuff ahead!

Version & changelog UPDATED: Guillem Jover told me how he met the same goals for libbsd :
  1. Save the version from git into .dist-version and use this file if it exists. This allows one to rebuild ./configure from the published tarball without losing the version. This also handles Thorsten Glaser s critic.
  2. Include CHANGELOG in DISTCLEANFILES variable.
Since this is a better solution, I have adopted the appropriate line of codes from libbsd. The two following sections are partly technically outdated.

Automated version In configure.ac, I was previously using a static version number that I had to increase when releasing:
AC_INIT([lldpd], [0.5.7], [bernat@luffy.cx])
Since the information is present in the git tree, this seems a bit redundant (and easy to forget). Taking the version from the git tree is easy:
AC_INIT([lldpd],
        [m4_esyscmd_s([git describe --tags --always --match [0-9]* 2> /dev/null   date +%F])],
        [bernat@luffy.cx])
If the head of the git tree is tagged, you get the exact tag (0.7.1 for example). If it is not, you get the nearest one, the number of commits since it and part of the current hash (0.7.1-29-g2909519 for example). The drawback of this approach is that if you rebuild configure from the released tarball, you don t have the git tree and the version will be a date. Just don t do that.

Automated changelog Generating the changelog from git is a common practice. I had some difficulties to make it right. Here is my attempt (I am using automake):
dist_doc_DATA = README.md NEWS ChangeLog
.PHONY: $(distdir)/ChangeLog
dist-hook: $(distdir)/ChangeLog
$(distdir)/ChangeLog:
        $(AM_V_GEN)if test -d $(top_srcdir)/.git; then \
          prev=$$(git describe --tags --always --match [0-9]* 2> /dev/null) ; \
          for tag in $$(git tag   grep -E '^[0-9]+(\.[0-9]+) 1, $$'   sort -rn); do \
            if [ x"$$prev" = x ]; then prev=$$tag ; fi ; \
            if [ x"$$prev" = x"$$tag" ]; then continue; fi ; \
            echo "$$prev [$$(git log $$prev -1 --pretty=format:'%ai')]:" ; \
            echo "" ; \
            git log --pretty=' - [%h] %s (%an)' $$tag..$$prev ; \
            echo "" ; \
            prev=$$tag ; \
          done > $@ ; \
        else \
          touch $@ ; \
        fi
ChangeLog:
        touch $@
Changelog entries are grouped by version. Since it is a bit verbose, I still maintain a NEWS file with important changes.

Core

C99 I have recently read 21st Century C which has some good bits and also handles the ecosystem around C. I have definitively adopted designated initializers in my coding style. Being a GCC extension since a long time, this is not a major compatibility problem. Without designated initializers:
struct netlink_req req;
struct iovec iov;
struct sockaddr_nl peer;
struct msghdr rtnl_msg;
memset(&req, 0, sizeof(req));
memset(&iov, 0, sizeof(iov));
memset(&peer, 0, sizeof(peer));
memset(&rtnl_msg, 0, sizeof(rtnl_msg));
req.hdr.nlmsg_len = NLMSG_LENGTH(sizeof(struct rtgenmsg));
req.hdr.nlmsg_type = RTM_GETLINK;
req.hdr.nlmsg_flags = NLM_F_REQUEST   NLM_F_DUMP;
req.hdr.nlmsg_seq = 1;
req.hdr.nlmsg_pid = getpid();
req.gen.rtgen_family = AF_PACKET;
iov.iov_base = &req;
iov.iov_len = req.hdr.nlmsg_len;
peer.nl_family = AF_NETLINK;
rtnl_msg.msg_iov = &iov;
rtnl_msg.msg_iovlen = 1;
rtnl_msg.msg_name = &peer;
rtnl_msg.msg_namelen = sizeof(struct sockaddr_nl);
With designated initializers:
struct netlink_req req =  
    .hdr =  
        .nlmsg_len = NLMSG_LENGTH(sizeof(struct rtgenmsg)),
        .nlmsg_type = RTM_GETLINK,
        .nlmsg_flags = NLM_F_REQUEST   NLM_F_DUMP,
        .nlmsg_seq = 1,
        .nlmsg_pid = getpid()  ,
    .gen =   .rtgen_family = AF_PACKET  
 ;
struct iovec iov =  
    .iov_base = &req,
    .iov_len = req.hdr.nlmsg_len
 ;
struct sockaddr_nl peer =   .nl_family = AF_NETLINK  ;
struct msghdr rtnl_msg =  
    .msg_iov = &iov,
    .msg_iovlen = 1,
    .msg_name = &peer,
    .msg_namelen = sizeof(struct sockaddr_nl)
 ;

Logging Logging in lldpd was not extensive. Usually, when receiving a bug report, I asked the reporter to add some additional printf() calls to determine where the problem was. This was clearly suboptimal. Therefore, I have added many log_debug() calls with the ability to filter out some of them. For example, to debug interface discovery, one can run lldpd with lldpd -ddd -D interface. Moreover, I have added colors when logging to a terminal. This may seem pointless but it is now far easier to spot warning messages from debug ones. logging output of lldpd

libevent In lldpd 0.5.7, I was using my own select()-based event loop. It worked but I didn t want to grow a full-featured event loop inside lldpd. Therefore, I switched to libevent. The minimal required version of libevent is 2.0.5. A convenient way to check the changes in API is to use Upstream Tracker, a website tracking API and ABI changes for various libraries. This version of libevent is not available in many stable distributions. For example, Debian Squeeze or Ubuntu Lucid only have 1.4.13. I am also trying to keep compatibility with very old distributions, like RHEL 2, which does not have a packaged libevent at all. For some users, it may be a burden to compile additional libraries. Therefore, I have included libevent source code in lldpd source tree (as a git submodule) and I am only using it if no suitable system libevent is available. Have a look at m4/libevent.m4 and src/daemon/Makefile.am to see how this is done.

Client

Serialization lldpctl is a client querying lldpd to display discovered neighbors. The communication is done through an Unix socket. Each structure to be serialized over this socket should be described with a string. For example:
#define STRUCT_LLDPD_DOT3_MACPHY "(bbww)"
struct lldpd_dot3_macphy  
        u_int8_t                 autoneg_support;
        u_int8_t                 autoneg_enabled;
        u_int16_t                autoneg_advertised;
        u_int16_t                mau_type;
 ;
I did not want to use stuff like Protocol Buffers because I didn t want to copy the existing structures to other structures before serialization (and the other way after deserialization). However, the serializer in lldpd did not allow to handle reference to other structures, lists or circular references. I have written another one which works by annotating a structure with some macros:
struct lldpd_chassis  
    TAILQ_ENTRY(lldpd_chassis) c_entries;
    u_int16_t        c_index;
    u_int8_t         c_protocol;
    u_int8_t         c_id_subtype;
    char            *c_id;
    int              c_id_len;
    char            *c_name;
    char            *c_descr;
    u_int16_t        c_cap_available;
    u_int16_t        c_cap_enabled;
    u_int16_t        c_ttl;
    TAILQ_HEAD(, lldpd_mgmt) c_mgmt;
 ;
MARSHAL_BEGIN(lldpd_chassis)
MARSHAL_TQE  (lldpd_chassis, c_entries)
MARSHAL_FSTR (lldpd_chassis, c_id, c_id_len)
MARSHAL_STR  (lldpd_chassis, c_name)
MARSHAL_STR  (lldpd_chassis, c_descr)
MARSHAL_SUBTQ(lldpd_chassis, lldpd_mgmt, c_mgmt)
MARSHAL_END;
Only pointers need to be annotated. The remaining of the structure can be serialized with just memcpy()1. I think there is still room for improvement. It should be possible to add annotations inside the structure and avoid some duplication. Or maybe, using a C parser? Or using the AST output from LLVM?

Library In lldpd 0.5.7, there are two possible entry points to interact with the daemon:
  1. Through SNMP support. Only information available in LLDP-MIB are exported. Therefore, implementation-specific values are not available. Moreover, SNMP support is currently read-only.
  2. Through lldpctl. Thanks to a contribution from Andreas Hofmeister, the output can be requested to be formatted as an XML document.
Integration of lldpd into a network stack was therefore limited to one of those two channels. As an exemple, you can have a look at how Vyatta made the integration using the second solution. To provide a more robust solution, I have added a shared library, liblldpctl, with a stable and well-defined API. lldpctl is now using it. I have followed those directions2:
  • Consistent naming (all exported symbols are prefixed by lldpctl_). No pollution of the global namespace.
  • Consistent return codes (on errors, all functions returning pointers are returning NULL, all functions returning integers are returning -1).
  • Reentrant and thread-safe. No global variables.
  • One well-documented include file.
  • Reduce the use of boilerplate code. Don t segfault on NULL, accept integer input as string, provide easy iterators,
  • Asynchronous API for input/output. The library delegates reading and writing by calling user-provided functions. Those functions can yield their effects. In this case, the user has to callback the library when data is available for reading or writing. It is therefore possible to integrate the library with any existing event-loop. A thin synchronous layer is provided on top of this API.
  • Opaque types with accessor functions.
Accessing bits of information is done through atoms which are opaque containers of type lldpctl_atom_t. From an atom, you can extract some properties as integers, strings, buffers or other atoms. The list of ports is an atom. A port in this list is also an atom. The list of VLAN present on this port is an atom, as well as each VLAN in this list. The VLAN name is a NULL-terminated string living in the scope of an atom. Accessing a property is done by a handful of functions, like lldpctl_atom_get_str(), using a specific key. For example, here is how to display the list of VLAN assuming you have one port as an atom:
vlans = lldpctl_atom_get(port, lldpctl_k_port_vlans);
lldpctl_atom_foreach(vlans, vlan)  
    vid = lldpctl_atom_get_int(vlan,
                               lldpctl_k_vlan_id));
    name = lldpctl_atom_get_str(vlan,
                                lldpctl_k_vlan_name));
    if (vid && name)
        printf("VLAN %d: %s\n", vid, name);
 
lldpctl_atom_dec_ref(vlans);
Internally, an atom is typed and reference counted. The size of the API is greatly limited thanks to this concept. There are currently more than one hundred pieces of information that can be retrieved from lldpd. Ultimately, the library will also enable the full configuration of lldpd. Currently, many aspects can only be configured through command-line flags. The use of the library does not replace lldpctl which will still be available and be the primary client of the library.

CLI Having a configuration file was requested since a long time. I didn t want to include a parser in lldpd: I am trying to keep it small. It was already possible to configure lldpd through lldpctl. Locations, network policies and power policies were the three items that could be configured this way. So, the next step was to enable lldpctl to read a configuration file, parse it and send the result to lldpd. As a bonus, why not provide a full CLI accepting the same statements with inline help and completion?

Parsing & completion Because of completion, it is difficult to use a YACC generated parser. Instead, I define a tree where each node accepts a word. A node is defined with this function:
struct cmd_node *commands_new(
    struct cmd_node *,
    const char *,
    const char *,
    int(*validate)(struct cmd_env*, void *),
    int(*execute)(struct lldpctl_conn_t*, struct writer*,
        struct cmd_env*, void *),
    void *);
A node is defined by:
  • its parent,
  • an optional accepted static token,
  • an help string,
  • an optional validation function and
  • an optional function to execute if the current token is accepted.
When walking the tree, we maintain an environment which is both a key-value store and a stack of positions in the tree. The validation function can check the environment to see if we are in the right context (we want to accept the keyword foo only once, for example). The execution function can add the current token as a value in the environment but it can also pop the current position in the tree to resume walk from a previous node. As an example, see how nodes for configuration of a coordinate-based location are registered:
/* Our root node */
struct cmd_node *configure_medloc_coord = commands_new(
    configure_medlocation,
    "coordinate", "MED location coordinate configuration",
    NULL, NULL, NULL);
/* The exit node.
   The validate function will check if we have both
   latitude and longitude. */
commands_new(configure_medloc_coord,
    NEWLINE, "Configure MED location coordinates",
    cmd_check_env, cmd_medlocation_coordinate,
    "latitude,longitude");
/* Store latitude. Once stored, we pop two positions
   to go back to the "root" node. The user can only
   enter latitude once. */
commands_new(
    commands_new(
        configure_medloc_coord,
        "latitude", "Specify latitude",
        cmd_check_no_env, NULL, "latitude"),
    NULL, "Latitude as xx.yyyyN or xx.yyyyS",
    NULL, cmd_store_env_value_and_pop2, "latitude");
/* Same thing for longitude */
commands_new(
    commands_new(
        configure_medloc_coord,
        "longitude", "Specify longitude",
        cmd_check_no_env, NULL, "longitude"),
    NULL, "Longitude as xx.yyyyE or xx.yyyyW",
    NULL, cmd_store_env_value_and_pop2, "longitude");
The definition of all commands is still a bit verbose but the system is simple enough yet powerful enough to cover all needed cases.

Readline When faced with a CLI, we usually expect some perks like completion, history handling and help. The most used library to provide such features is the GNU Readline Library. Because this is a GPL library, I have first searched an alternative. There are several of them: From an API point of view, the first three libraries support the GNU Readline API. They also have a common native API. Moreover, this native API also handles tokenization. Therefore, I have developed the first version of the CLI with this API3. Unfortunately, I noticed later this library is not very common in the Linux world and is not available in RHEL. Since I have used the native API, it was not possible to fallback to the GNU Readline library. So, let s switch! Thanks to the appropriate macro from the Autoconf Archive (with small modifications), the compilation and linking differences between the libraries are taken care of. Because GNU Readline library does not come with a tokenizer, I had to write one myself. The API is also badly documented and it is difficult to know which symbol is available in which version. I have limited myself to:
  • readline(), addhistory(),
  • rl_insert_text(),
  • rl_forced_update_display(),
  • rl_bind_key()
  • rl_line_buffer and rl_point.
Unfortunately, the various libedit libraries have a noop for rl_bind_key(). Therefore, completion and online help is not available with them. I have noticed that most BSD come with GNU Readline library preinstalled, so it could be considered as a system library. Nonetheless, linking with libedit to avoid licensing issues is possible and help can be obtained by prefixing the command with help.

OS specific support

BSD support Until version 0.7, lldpd was Linux-only. The rewrite to use Netlink was the occasion to abstract interfaces and to port to other OS. The first port was for Debian GNU/kFreeBSD, then for FreeBSD, OpenBSD and NetBSD. They all share the same source code:
  • getifaddrs() to get the list of interfaces,
  • bpf(4) to attach to an interface to receive and send packets,
  • PF_ROUTE socket to be notified when a change happens.
Each BSD has its own ioctl() to retrieve VLAN, bridging and bonding bits but they are quite similar. The code was usually adapted from ifconfig.c. The BSD ports have the same functionalities than the Linux port, except for NetBSD which lacks support for LLDP-MED inventory since I didn t find a simple way to retrieve DMI related information. They also offer greater security by filtering packets sent. Moreover, OpenBSD allows to lock the filters set on the socket:
/* Install write filter (optional) */
if (ioctl(fd, BIOCSETWF, (caddr_t)&fprog) < 0)  
    rc = errno;
    log_info("privsep", "unable to setup write BPF filter for %s",
        name);
    goto end;
 
/* Lock interface */
if (ioctl(fd, BIOCLOCK, (caddr_t)&enable) < 0)  
    rc = errno;
    log_info("privsep", "unable to lock BPF interface %s",
        name);
    goto end;
 
This is a very nice feature. lldpd is using a privileged process to open the raw socket. The socket is then transmitted to an unprivileged process. Without this feature, the unprivileged process can remove the BPF filters. I have ported the ability to lock a socket filter program to Linux. However, I still have to add a write filter.

OS X support Once FreeBSD was supported, supporting OS X seemed easy. I got sponsored by xcloud.me which provided a virtual Mac server. Making lldpd work with OS X took only two days, including a full hour to guess how to get Apple Xcode without providing a credit card. To help people installing lldpd on OS X, I have also written a lldpd formula for Homebrew which seems to be the most popular package manager for OS X.

Upstart and systemd support Many distributions propose upstart and systemd as a replacement or an alternative for the classic SysV init. Like most daemons, lldpd detaches itself from the terminal and run in the background, by forking twice, once it is ready (for lldpd, this just means we have setup the control socket). While both upstart and systemd can accommodate daemons that behave like this, it is recommended to not fork. How to advertise readiness in this case? With upstart, lldpd will send itself the SIGSTOP signal. upstart will detect this, resume lldpd with SIGCONT and assume it is ready. The code to support upstart is therefore quite simple. Instead of calling daemon(), do this:
const char *upstartjob = getenv("UPSTART_JOB");
if (!(upstartjob && !strcmp(upstartjob, "lldpd")))
    return 0;
log_debug("main", "running with upstart, don't fork but stop");
raise(SIGSTOP);
The job configuration file looks like this:
# lldpd - LLDP daemon
description "LLDP daemon"
start on net-device-up IFACE=lo
stop on runlevel [06]
expect stop
respawn
script
  . /etc/default/lldpd
  exec lldpd $DAEMON_ARGS
end script
systemd provides a socket to achieve the same goal. An application is expected to write READY=1 to the socket when it is ready. With the provided library, this is just a matter of calling sd_notify("READY=1\n"). Since sd_notify() has less than 30 lines of code, I have rewritten it to avoid an external dependency. The appropriate unit file is:
[Unit]
Description=LLDP daemon
Documentation=man:lldpd(8)
[Service]
Type=notify
NotifyAccess=main
EnvironmentFile=-/etc/default/lldpd
ExecStart=/usr/sbin/lldpd $DAEMON_ARGS
Restart=on-failure
[Install]
WantedBy=multi-user.target

OS include files Linux-specific include files were a major pain in previous versions of lldpd. The problems range from missing header files (like linux/if_bonding.h) to the use of kernel-only types. Those headers have a difficult history. They were first shipped with the C library but were rarely synced and almost always outdated. They were then extracted from kernel version with almost no change and lagged behind the kernel version used by the released distribution4. Today, the problem is acknowledged and is being solved by both the distributions which extract the headers from the packaged kernel and by kernel developers with a separation of kernel-only headers from user-space API headers. However, we still need to handle legacy. A good case is linux/ethtool.h:
  • It can just be absent.
  • It can use u8, u16 types which are kernel-only types. To work around this issue, type munging can be setup.
  • It can miss some definition, like SPEED_10000. In this case, you either define the missing bits and find yourself with a long copy of the original header interleaved with #ifdef or conditionally use each symbol. The latest solution is a burden by itself but it also hinders some functionalities that can be available in the running kernel.
The easy solution to all this mess is to just include the appropriate kernel headers into the source tree of the project. Thanks to Google ripping them for its Bionic C library, we know that copying kernel headers into a program does not create a derivative work.

  1. Therefore, the use of u_int16_t and u_int8_t types is a left-over of the previous serializer where the size of all members was important.
  2. For more comprehensive guidelines, be sure to check Writing a C library.
  3. Tokenization is not the only advantage of libedit native API. The API is also cleaner, does not have a global state and has a better documentation. All the implementations are also BSD licensed.
  4. For example, in Debian Sarge, the Linux kernel was a 2.6.8 (2004) while the kernel headers were extracted from some pre-2.6 kernel.

19 February 2013

Robert Collins: First experience implementing StreamResult

My last two blog posts were largely about the needs of subunit, but a key result of any protocol is how easy working with it in a high level language is. In the weekend and evenings I ve done an implementation of a new set of classes StreamResult and friends that provides: So far the code has been uniformly simple to write. I started with an API that included an estimate function, which I ve since removed I don t believe the complexity is justified; enumeration is not significantly more expensive than counting, and runners that want to be efficient can either not enumerate or remember the enumeration from prior runs. The documentation in the linked pull request is a good place to start to get a handle on the API; I d love feedback. Next steps for me are to do a subunit protocol revision that maps to the Python API, both parser and generator and see how it feels. One wrinkle there is that the reason for doing this is to fix intrinsic limits in the existing protocol so doing forward and backward wire protocol compatibility would defeat the point. However we can make the output side explicitly choose a protocol version, and if we can autodetect the protocol version in the parser, even if we cannot handle mixed streams we can get the benefits of the new protocol once data has been detected. That said, I think we can start without autodetection during prototyping, and add it later. Without autodetection, programs like TestRepository will need configuration options to control what protocol variant to expect. This could be done by requiring this new protocol and providing a stream filter that can be deployed when needed.

Ingo Juergensmann: LVM on RAID5 broken - how to fix?

Some time ago one of my disks in my Software RAID5 failed. No big problem as I had two spare A08U-C2412 available to replace that single 1 TB SATA disk. I can't remember any details but somewhat went wrong and I ended up with a non-booting system. I think I tried to add the HW-RAID as a physical volume to the LVM and thought of migrating the SW-RAID to the HW-RAID or do mirroring or such. Anyway: I booted into my rescue system that was on a RAID1 partition on those disks, but LVM didn't came up anymore, because the SW-RAID5 wasn't recognized during boot. So I re-created the md device and discovered that my PV was gone as well. =:-0 No, big deal, I thought, because I have a backup of that machine on another host. I restored /etc/lvm and tried to do a vgcfgrestore after I re-created the PV with pvcreate. First I didn't use the old UUID, so vgcfgrestore complained. After creating the proper PV with UUID LVM did recognize the PV, VG and LVs. Unfortunately I can't mount any of the LVs. Something seems to be broken:
hahn-rescue:~# mount /dev/vg/sys /mnt
mount: you must specify the filesystem type
Feb 19 07:50:02 hahn-rescue kernel: [748288.740949] XFS (dm-0): bad magic number
Feb 19 07:50:02 hahn-rescue kernel: [748288.741009] XFS (dm-0): SB validate failed
Running a gpart scan on my SW-RAID5 did gave me some results:
hahn-rescue:~# gpart /dev/md3
Begin scan...
Possible partition(SGI XFS filesystem), size(20470mb), offset(5120mb)
Possible partition(SGI XFS filesystem), size(51175mb), offset(28160mb)
Possible partition(SGI XFS filesystem), size(1048476mb), offset(117760mb)
Possible partition(SGI XFS filesystem), size(204787mb), offset(1168640mb)
Possible partition(SGI XFS filesystem), size(204787mb), offset(1418240mb)
Possible partition(SGI XFS filesystem), size(1048476mb), offset(1626112mb)
*** Fatal error: dev(/dev/md3): seek failure.
These is not the complete lists of VM as a comparison to the output of lvs shows:
hahn-rescue:~# lvs
  LV                 VG   Attr     LSize   Pool Origin Data%  Move Log Copy%  Convert
  storage1           lv   -wi-ao--   1.00t                                          
  AmigaSeagateElite3 vg   -wi-a---   3.00g                                          
  audio              vg   -wi-a---  70.00g                                          
  backup             vg   -wi-a---   1.00t                                          
  data               vg   -wi-a---  50.00g                                          
  hochzeit           vg   -wi-a---  40.00g                                          
  home               vg   -wi-a---   5.00g                                          
  pics               vg   -wi-a--- 200.00g                                          
  sys                vg   -wi-a---  20.00g                                          
  video              vg   -wi-a--- 100.00g                                          
  windata            vg   -wi-a--- 100.00g   
Please notice that /dev/lv/storage1 is my HW-RAID where I stored the images of /dev/vg/-LVs to run xfs_repair and such on. Anyway, the sizes of the recognized XFS partitions by gpart is mostly correct, but some a missing and xfs_repair can not do anything good on the backup images on storage1. Everything ends up in /lost+found, because the blocks seem to be mixed up somehow. What I figured out is, that my old RAID5 device was in metaformat=1.2 whereas the new one is in format 0.9. My best guess is now to recreate the RAID5 device with format 1.2, do a vgcfgrestore on that and have (hopefully!) a working LVM with working LVs back that I can then mount again. If there's anything else I might be able to try, dear Lazyweb, please tell me. Please see the attached config files/tarballs for a complete overview. Side note: except for AmigaSeagateElite3, which is a dd image of an old Amiga SCSI disk, I should have a fairly complete backup on my second backup location, so there's not much lost, but I would be a real timesaver when I would be able to recover the lost LVs. Both systems are behind DSL/cable and have a limit of 10 Mbps upstream. It would take weeks to transfer the data and sending an USB disk would be faster.
AttachmentSize
mdadm.conf_.txt954 bytes
hahn-lvm-restore.tar.gz14.7 KB
Kategorie:

18 February 2013

Jan Wagner: Airprint support broken in Debian wheezy?

Some of our customers are using central CUPS systems for managing their printer infrastructure. In the last years the demand for support printing (called Airprint) from mobile apple products increased. This worked well for us on Debian squeeze as documented here (updated scrips). I tried this on a fresh installed Debian wheezy amd64, but no printer was found on any IOS device. Hmm .... let's see if the printer is announces via avahi:
$ avahi-browse -a   grep -i print
+   eth0 IPv6 Kyocera FS-1020D @ service                    Internet Printer     local
+   eth0 IPv6 AirPrint Kyocera @ service                    Internet Printer     local
+   eth0 IPv4 Kyocera FS-1020D @ service                    Internet Printer     local
+   eth0 IPv4 AirPrint Kyocera @ service                    Internet Printer     local
WTF?!? Fortunately we have an Ubuntu 12.04 running in our office and printing from IOS devices works without problems (without copying any files to /etc/avahi/services/):
$ avahi-browse -a   grep -i print
+   eth2 IPv6 Ricoh Aficio MP C2800 @ printing              Internet Printer     local
+   eth2 IPv6 Ricoh Aficio MP 171 @ printing                Internet Printer     local
+   eth2 IPv4 Ricoh Aficio MP 171 @ printing                Internet Printer     local
+   eth2 IPv4 Ricoh Aficio MP C2800 @ printing              Internet Printer     local
I just copied the whole configuration over to my wheezy system, but it didn't worked out. I tried this all on a kfreebsd-i386 system again without success. Sorry, but I don't understand the source of this issue. Cups on wheezy has the same upstream version as on precise. Avahi-daemon on wheezy is just one minor version ahead off precise. Is this a bug/incompatibility in cups and/or avahi? A missing patch compared to the packages of precise? Is this a configuration problem? Update: Looking into LP #1054495 and the debdiff of cups 1.5.3-0ubuntu6 indicates, that there seems modifications beside simple changes of mime configurations files. Update: The fix for LP #1054495 does really fix the problem on wheezy too. With the help of Didier Raboud I was able test a binary package with this fixapplied on our test setup. The good news is, there are no extra modifications needed beside just configuring cups to export it's printers for enabling support for iOS devices. I opened #700961 and hopefully release managment will accept this fix for wheezy. Hints, rants and comments could be send to 'blog - at - waja - dot - info' or via @blogwajainfo.

Next.

Previous.