Search Results: "ed"

11 July 2012

Michal Čihař: New Weblate widget to embed in your site

Weblate just got new feature, which might help you attracting new translators - embeddable widgets. You can put them to your website to show current state of the translation and attract new ones to simple introduction page. Right now, there is only one widget, but I hope to add more in future. However I'm far from being good in graphics, so if anybody could come up with some templates, I'd gladly use them :-). Anyway the current widget for Weblate itself looks like this:

Filed under: English Weblate 0 comments Flattr this!

Dirk Eddelbuettel: Getting numpy data into R -- Take Two

A couple of days ago, I had posted a short Python script to convert numpy files into a simple binary format which R can read quickly. Nice, but still needing an extra file. Shortly thereafter, I found Carl Rogers cnpy library which makes reading and writing numpy files from C++ a breeze, and I quickly wrapped this up into a new package RcppCNPy which was released a few days ago. This post will show a quick example, also summarized in the short pdf vignette describing the package, and provided as a demo within the package.
R> library(RcppCNPy)
Loading required package: Rcpp
R> library(rbenchmark)
R> 
R> n <- 1e5
R> k <- 50
R> 
R> M <- matrix(seq(1.0, n*k, by=1.0), n, k)
R> 
R> txtfile <- tempfile(fileext=".txt")
R> write.table(M, file=txtfile)
R> 
R> pyfile <- tempfile(fileext=".py")
R> npySave(pyfile, M)
R> 
R> pygzfile <- tempfile(fileext=".py")
R> npySave(pygzfile, M)
R> system(paste("gzip -9", pygzfile))
R> pygzfile <- paste(pygzfile, ".gz", sep="")
R> 
R> 
We first load the new package (as well as the rbenchmark package used for the benchmarking example) into R. We then create a large matrix of 100,000 rows and 50 columns. Not quite big data by any stretch, but large enough for ascii reading to be painfully slow. We also write two npy files and compress the second one. Next, we use the benchmark function to time the three approaches:
R> res <- benchmark(read.table(txtfile),
+                  npyLoad(pyfile),
+                  npyLoad(pygzfile),
+                  order="relative",
+                  columns=c("test", "replications", "elapsed", "relative"),
+                  replications=10)
R> print(res)
                 test replications elapsed relative
2     npyLoad(pyfile)           10   1.241  1.00000
3   npyLoad(pygzfile)           10   3.098  2.49637
1 read.table(txtfile)           10  96.744 77.95649
R> 
As shown by this example, loading a numpy file directly beats the pants off reading the data from ascii: it is about 78 times faster. Reading a compressed file is somewhat slower as the data stream has to be passed through the uncompressor provide by the zlib library. So instead of reading a binary blob in one go (once the file header has been parsed) we have to operate piecemeal---which is bound to be slower. It does however save in storage space (and users can make this tradeoff between speed and size) and is still orders of magnitude faster than parsing the ascii file. Finally, and not shown here, we unlink the temporary files. Summing up, this post demonstrated how the RcppCNPy package can be a useful to access data in numpy files (which may even be compressed). Data can also be written from R to be accessed later by numpy.

Paul Tagliamonte: The "hell-server"

I s pose a better name would be How to preform automated deep-source checks I ve pushed the code to paultag/qlight, and it s just for personal use at the moment. It s built to use paultag/sb around sbuild. Basically, it preforms a crudload of tests such as licensecheck, suspicious-source, pyflakes, bashisms, cppcheck, php in linter mode, and a ton of other checks. You can see a large majority of them in the source file. The batch runner is a singleton , so it s safely called more then once. It s running on a 5-minute cron, and I ve configured a dput target to push it into it s queue & run the enqueue script. Here s some example output from a random package I had lying around:
  ===== build status =====
sbo-run-clang  successful
sbo-run-gcc  successful
  ===== global status =====
Ran plugin extract, exit code 0
Ran plugin watch, exit code 0
Ran plugin licensecheck, exit code 0
Ran plugin validate-source, exit code 0
Ran plugin build, exit code 0
Ran plugin descr, exit code 0
Ran plugin lintian, exit code 0
  ===== descr =====
I: PT: Does this sound right? "gbemol is/are a(n) Graphical frontend for the Music Player Daemon (MPD)"
  ===== piuparts =====
   [last 20 lines]
  ===== licenses =====
BSD (3 clause) 
GPL (v2 or later) 
GPL (v2 or later) GENERATED FILE
GPL (v2 or later) (with incorrect FSF address) 
LGPL (v2 or later) (with incorrect FSF address) 
*No copyright* GPL (v2 or later) 
*No copyright* UNKNOWN
  ===== lintian =====
I: gbemol: spelling-error-in-binary usr/bin/gbemol Wheter Whether
W: gbemol: hardening-no-relro usr/bin/gbemol
I: gbemol: spelling-error-in-manpage usr/share/man/man1/gbemol.1.gz allows to allows one to
  ===== watch =====
-- Scanning for watchfiles in .
-- Found watchfile in ./debian
-- In debian/watch, processing watchfile line:
   http://sf.net/gbemol/gbemol-(.*)\.tar.gz
-- Found the following matching hrefs:
     gbemol-0.3.tar.gz
     gbemol-0.3.1.tar.gz
     gbemol-0.3.2.tar.gz
Newest version on remote site is 0.3.2, local version is 0.3.2
 => Package is up to date
-- Scan finished
  ===== source issues =====
suspicious-source:
bashisms:
script ./ltmain.sh does not appear to have a #! interpreter line;
you may get strange results
possible bashism in ./ltmain.sh line 368 ($RANDOM):
        my_tmpdir="$ my_template -$ RANDOM-0 $$"
error: ./ltmain.sh: Unterminated quoted string found, EOF reached. Wanted: <">
pyflakes:
cppcheck:
[src/gbemol-utils.c:167]: (error) Memory leak: path
[src/libmpdclient.c:1654]: (error) Possible null pointer dereference: output - otherwise it is redundant to check if output is null at line 1663
[src/libmpdclient.c:1655]: (error) Possible null pointer dereference: output - otherwise it is redundant to check if output is null at line 1663
[src/libmpdclient.c:1656]: (error) Possible null pointer dereference: output - otherwise it is redundant to check if output is null at line 1663
[src/libmpdclient.c:1664]: (error) Possible null pointer dereference: output - otherwise it is redundant to check if output is null at line 1663
phplint:
desktop validate:
.//.pc/desktop_file.patch/gbemol.desktop: warning: key "Encoding" in group "Desktop Entry" is deprecated
.//.pc/desktop_file.patch/gbemol.desktop: error: (will be fatal in the future): value "gbemol.png" for key "Icon" in group "Desktop Entry" is an icon name with an extension, but there should be no extension as described in the Icon Theme Specification if the value is not an absolute path
.//.pc/desktop_file.patch/gbemol.desktop: warning: value "Application;AudioVideo;Player;" for key "Categories" in group "Desktop Entry" contains a deprecated value "Application"
mp3check:
mp3val:
fontlint:
ftvalid:
msgfmt:
POFileChecker:
POFileSpell:
pngcheck:
jpeginfo:
ogginfo:
oggz-validate:
xmllint:
lacheck:
jslint:
check for environ calls to home:
check for use of /tmp:
./intltool-extract.in:    $OUTFILE     = "./tmp/$OUTFILE.h"
check for use of system():
./aclocal.m4:    lt_ld_extra= awk '/^include /   system(sprintf("cd /etc; cat %s 2>/dev/null", \[$]2)); skip = 1;     if (!skip) print \[$]0; skip = 0;  ' < /etc/ld.so.conf   $SED -e 's/#.*//;/^[  ]*hwcap[    ]/d;s/[:,  ]/ /g;s/=[^=]*$//;s/=[^= ]* / /g;/^$/d'   tr '\n' ' ' 
./intltool-merge.in:    return 1 if ((-x $file) or (system("$file -l >$devnull") == 0));
./intltool-merge.in:    system ("$iconv -f $encoding -t UTF-8 <$devnull 2>$devnull");
./intltool-update.in:    return 1 if ((-x $file) or (system("$file --version >$devnull") == 0));
./intltool-update.in:          system ($EXTRACT, "--update", "--srcdir=$SRCDIR",
./intltool-update.in:          system ($EXTRACT, "--update", "--type=$gettext_type", 
./intltool-update.in:    system ("$MSGMERGE", "-o", $outfile, $infile, "$MODULE.pot");
./intltool-update.in:    system ("$MSGFMT", "-o", "$devnull", "--verbose", $output_file);
./intltool-update.in:   system ("$MSGFMT", "-o", "$devnull", "--verbose", "$SRCDIR/$lang.po");
./configure:    lt_ld_extra= awk '/^include /   system(sprintf("cd /etc; cat %s 2>/dev/null", \$2)); skip = 1;     if (!skip) print \$0; skip = 0;  ' < /etc/ld.so.conf   $SED -e 's/#.*//;/^[   ]*hwcap[    ]/d;s/[:,  ]/ /g;s/=[^=]*$//;s/=[^= ]* / /g;/^$/d'   tr '\n' ' ' 
./intltool-extract.in:        system ("mkdir -p $dirname");
./intltool-extract.in:        system("mkdir tmp/"); 
./debian/gbemol.1:the \fBDebian\fP system (but may be used by others).  Permission is 
./debian/manpage.sgml:      the &debian; system (but may be used by others).  Permission is
  ===== sbo-run-clang.log warnings =====
update-binfmts: warning: unable to close /proc/sys/fs/binfmt_misc/register: Invalid argument
update-binfmts: warning: unable to enable binary format python2.6
dpkg-source: warning: extracting unsigned source package (gbemol_0.3.2-2.dsc)
dpkg-buildpackage: warning: using a gain-root-command while being root
gbemol-preferences.c:483:13: warning: 'gbemol_preferences_load_spin' defined but not used [-Wunused-function]
gbemol-net.c:60:11: warning: variable 'res' set but not used [-Wunused-but-set-variable]
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libfontconfig.so.1 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libatk-1.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against librt.so.1 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libgio-2.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libcairo.so.2 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libpango-1.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libpangocairo-1.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libfreetype.so.6 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libpangoft2-1.0.so.0 (it uses none of the library's symbols).
dpkg-gencontrol: warning: File::FcntlLock not available; using flock which is not NFS-safe
     + add misc-depends to fix lintian warning.
  ===== sbo-run-gcc.log warnings =====
dpkg-source: warning: extracting unsigned source package (gbemol_0.3.2-2.dsc)
dpkg-buildpackage: warning: using a gain-root-command while being root
gbemol-preferences.c:483:13: warning: 'gbemol_preferences_load_spin' defined but not used [-Wunused-function]
gbemol-net.c:60:11: warning: variable 'res' set but not used [-Wunused-but-set-variable]
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libfontconfig.so.1 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libatk-1.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against librt.so.1 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libgio-2.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libcairo.so.2 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libpango-1.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libpangocairo-1.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libfreetype.so.6 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libpangoft2-1.0.so.0 (it uses none of the library's symbols).
dpkg-gencontrol: warning: File::FcntlLock not available; using flock which is not NFS-safe
     + add misc-depends to fix lintian warning.
Not as interesting as it can get, but still pretty cool. Andrea, I swear I m not picking on you :) More to come later (hopefully) :)

10 July 2012

Arnaud Quette: Definitive solution to IPMI over LAN with Dell iDrac Express

I have this bunch of Dell R610, with iDrac6 Express management cards. I used these, among other things, for developing IPMI support in NUT and working on Infrastructure & Cloud power management. But that's the topic of another post (still, if you're interested in, check this and that). The thing is that this "IPMI" monitoring development has been limited to local support (Ie, power supplies can't be monitored remotely by the nut-ipmipsu driver), due to an issue : any attempt to enable IPMI access over the network was miserably failing! Well, these attempts were limited to a couple of 15 minutes runs, without plain motivation, almost a year ago. The various firmwares were up to date (iDrac 1.70, ...) , everything was running and configured fine, locally. But still... no IPMI available through the network! Looking on the Net, I've learned that many Dell customers with iDrac Express cards, were having the same issue. Dell support seems to have replaced tons of motherboards! There, I switched to other things, and time has passed.... A good year later (last week), I decided that it was time to get back on this. And I've found the solution there Incredible: this was due to a 'bug' in the Broadcom NetXtreme II LoM (LAN on Motherboad) firmware! I've not had time to dig this issue in depth, but here is a base explanation, for what it's worth: Some LoM initial self tests are failing. Thus, the LoM are not switched to the managed mode, and can't actually be available for BMC management (thus no IPMI over the network). In my case, the tests were wrongly failing at 'A07', a test which tries to establish a Gigabit connection! Strangely, all these servers are connected on a Gb switch! Not a fully satisfactory answer, but that said, there is a solution, and I've not much time to pour into this investigation (comments may always change my mind though!). So here is a comprehensive procedure to fix this, from your Linux system, and using FreeDOS:
$ mkfs.msdos /dev/sdX1
Note: 'X' is to be replaced by the exact name of your USB key. An hint: call 'tail -f /var/log/syslog" and unplug / replug your USB key. You will see some entries like "...sdb Attached SCSI removable disk". So, there, it's "sdb".
$ qemu -boot a -fda balder10.img -hda /dev/sdX A:\> sys c: A:\> xcopy /E /N a: c:
Note that you will need "root" privileges.
$ qemu -hda /dev/sdX
$ unzip Bcom_LAN_14.2.x_DOSUtilities_A03.exe
c:\ uxdiag -t abcd mfw 1
For what it's worth (again), I just hope that it will be useful to others... I will now prepare another post using using FreeIPMI to manage your servers, the GNU way... cheers,
-- Arno Thanks to Jordi Clariana, his enlightening post, Daniel for this one, Aur lien was motivating me again in solving this iDrac Express issue and Al Chu (FreeIPMI project leader) for all his invaluable help on IPMI.

Raphael Geissert: Experimental IPv6 support

For a few hours now, http.debian.net is also reachable over IPv6. The service is still experimental, as the code makes a few assumptions that are not yet automatically verified and the IPv6 GeoIP database is sort-of experimental too.

Note that the results between a request over IPv4 and one over IPv6 may vary quite a bit. Requests over IPv6 are redirected only to hosts that support it. A great number of mirrors, however, don't support IPv6.

Jon Dowland: Cited

This blog has been cited in an academic paper! On October 2011 I drafted a blog post about Infosuicide, or "Infocide" as I prefer, prompted by Mark Pilgrim's disappearance. I never got around to finishing the post up in particular I felt it was too long but my draft was exposed to the Internet and somebody noticed it. Assistant Professor Joseph Reagle stumbled across my draft and wrote an in-depth paper 410 Gone : Infocide in Open Content Communities citing a whole load of writings on the subject, including my own. I believe his paper has now been submitted to a conference proceedings (I'll update this when I find out which!) For that reason I've gone ahead and published the draft as-is.

Gunnar Wolf: From DebCamp to DebConf through cheese, wine and an intro track

One week. One long week. One beautiful week. One of the two major weeks of the year has passed since my previous post. Surely, we are in the middle of the two Major Weeks of the year, in the yearly schedule I have upheld for almost(!) ten years: DebConf+DebCamp. Yesterday, DebConf officially started. For the first time ever, we had a DebConf track targetted at the local (for a wide definition of local: All of the Central American countries) communities, which I chaired. We had the following talk lineup during this track: I believe it was a great success, and I hope the talks are useful in the future. They will be put online soon thanks to the tireless work of our work team. Today we sadly lost the presence of our DPL due to very happy circumstances he will surely announce himself. But DebConf will continue nevertheless - And proof of that is our anual, great, fun and inviting Cheese and Wine Party! After a series of organizational hiccups I hope nobody notices (oops, was I supposed not to say this?), today we had a beautiful, fun and most successful cheese and wine party, as we have had year after year since 2005. As many other people, we did our humble contribution for this party to be the success it deserves. There is lots of great cheeses, great wines, and much other great stuff we have to thank to each of the individuals who made this C&W party the success it was. Yes, it might be among the least-academic parts of our conference, but at the same time, it's one of its most cherished -and successful- traditions. And above all else, we have to thank our Great Leader^W^WCheeseMaster (who we still need to convince to play by our Great Leader's mandates - And no, I don't mean Zack here!) Hugs and thanks to my good and dear friend Christian Perrier for giving form to one of DebConf's social traditions that makes it so unique, so different from every other academic or communitary conference I have ever been part of. We still have most of the week to go. And if you are not in Managua (and are not coming soon), you can follow our activities following our video streams. Remember, debian/rules, now more than ever! And even given the (perpetual) heat in Managua: Wheezy is frozen, whee! [ all photos here taken by regina ]
AttachmentSize
100_2110.JPG1.53 MB
100_2119.JPG1.49 MB
100_2121.JPG1.52 MB
100_2135.JPG1.49 MB
100_2144.JPG1.11 MB
100_2146.JPG1.53 MB
100_2145.JPG1.08 MB
100_2122.JPG1.51 MB

Gerfried Fuchs: Mermaids Reloaded

It's almost seven years now since I wrote/published Mermaids. It was an important step in my life, confessing publicly what I've found out about myself, how I feel and identify myself. It definitely has been a certain turning point in my life. A year later, there was the Debconf6 in Mexico, which was just as important. I was wearing a skirt for the first time, and that for the whole duration of the two weeks. It was an enormous feeling of freedom, and I knew I was feeling at home. That was also the time I shaved off my beard. And since? Well, I sort of limited myself to these two weeks every year. Debconf is my haven, Debconf is my home. Here I allow myself to be myself. Debian is my family. And like in every family, there are people who won't understand, but that doesn't matter. I feel comfortable to express myself in this crowd in the way I feel. Being abroad seems to help with the confidence, and it seems like it also shines through. I haven't had lots of strange looks. To the contrary, when I went with a skirt to a homosexual acceptance event at home in Vienna, there was a group of gays who were pointing and laughing. Quite an interesting experience, one would assume that people who are wanting more acceptance would be more acceptable and tolerant themselves... But, I don't think I will ever take the step toward actually turning my body into a female one. It was an important step for me to find out about my inner self, and it actually managed to make me accept my male body; even though I can totally relate to people not being able to understand this both people not in the situation and people being in the situation and needing the physical adjustment to be able to be happy again. But this is what and how I am: Contents may vary from packaging. This though has an inherited problem: as I am not (usually) heading for a female appearance (besides my skirt during Debconf), people keep addressing me with a male pronoun, even, or rather especially when using my nick Rhonda. This is something that I consider a fair bit disturbing, especially when it comes from people that I consider to address with the term friends. One would assume that people who you share a fair bit of private life with would be the ones who can understand and relate better than others. But that's where we are, and I also can understand the troubles: I'm not giving them much visual help for fixing their thoughts. I found the confidence in my body, and like most of you know, I have a son I dearly love (and miss like hell these two weeks), so I know how to use the tools it comes with, frankly spoken. But I won't go the road to adjust my body to be a female one instead, just to convince people that I really am the female person that I identify as. It might be hard for you to understand that, it might be hard for you to accept it it is also hard for me, to fight for the acceptance that I thought are an inherent part of the term friendship. I have deepest respect for the people who feel the requirement and have the strength to adjust their body to their mind. It though just isn't the road for me. I already have enough uncertainties in my life to cope with, and I don't need another one to deal with, related to that I might not be able to accept my body after the transition than I am able to accept it now. It's just not important enough for me to find my place. My place is here, my home is Debconf.

/personal permanent link Comments: 4 Flattr this

Keith Packard: hotplug-displaylink

Getting Hotplugging working with a DisplayLink USB to DVI adapter I merged Dave Airlie s randr provider patches to the server and pushed them out in preparation for freezing the X server for the version 1.13 release. Before freezing, I figured I should at least test hotplugging my DisplayLink adapter. Well, that took all day Sources For the impatient, here s where all of the bits are. These work on my machine. Upstream bits:
  1. randrproto. git://anongit.freedesktop.org/git/xorg/proto/randrproto master
Bits from Dave Airlie s trees:
  1. libdrm. git://people.freedesktop.org/~airlied/drm.git prime
  2. libXrandr. git://people.freedesktop.org/~airlied/libXrandr.git prime
  3. xrandr. git://people.freedesktop.org/~airlied/xrandr.git prime
  4. xf86-video-intel. git://people.freedesktop.org/~airlied/xf86-video-intel prime
Bits from my trees:
  1. xf86-video-modesetting. git://people.freedesktop.org/~keithp/xf86-video-modesetting prime
  2. kernel. git://people.freedesktop.org/~keithp/linux prime
  3. X server. git://people.freedesktop.org/~keithp/xserver master
Kernel adventures The 3.5-rc6 bits have all of the necessary driver changes to support DisplayLink hot-plug. At least, they do if you re not running 32-bit user space atop a 64-bit kernel. If you are, then most of the DRM drivers don t work at all they re missing the .compat_ioctl entry in the file_operations structure, which makes all ioctl calls return -ENOTTY. In particular, the udl driver is missing this entry, so when you plug in the device, the server tries to talk to it and nothing works. X server adventures Once I got the kernel working, I discovered some nice crashing behaviour in the X server. If I started the server with the DisplayLink device plugged in, as soon as I tried to enable the output, the server would segfault. Turns out this cases requires special code in the server to ensure that the DisplayLink screen privates are adjust correctly while the intel driver loads. Modesetting driver fixup The modesetting driver was missing a rename that happened to a structure member in the last couple of days. I d bet Dave has the same patch sitting on a disk somewhere. xrandr changes The command line additions to xrandr are sufficient to get this stuff working, but I think it could use some improvements to make the interface a bit friendlier. In particular, requiring you to use XIDs to identify providers is a bit harsh. Schedules I ll be pushing the X server changes out this evening; review would be appreciated, but I don t think any of the patches I made there are scary. I sent the kernel patches necessary to lkml, but with Dave in transit, I m not sure who is minding the store. As for the rest of the bits, they re sitting in Dave s repositories, presumably they ll get pushed upstream soon. But, does it actually work? Almost. Everything says it s working, but I m not getting any signal to my DVI monitor. My DisplayLink device is ancient, and the kernel complains about it, saying
[drm:udl_parse_vendor_descriptor] *ERROR* Unrecognized vendor firmware descriptor".
So close, and yet

9 July 2012

Arnaud Quette: Hello Planet Debian

Hey fellows Debian lovers, and old friends reading this, after more than a decade serving Debian, our greatest and beloved OS, I've finally decided to start blogging. Please, bear with me, I'm still very new at blogging. Thus, constructive comments and mails are welcome ;-) So here we actually go with the usual 'quick introduction post': I'm Arnaud Quette, 36 years old, and Debian developer since 2001. I'm leading the upstream project called 'NUT' (Network UPS Tools project) since 2005, and I'm a Free Software hacker since 1997 (incomplete list of contribs and linkedIn profile). I'm working at Eaton, a Debian partner participating in the power protection of our Debian infrastructure (FTP masters and Alioth). If you're a Debian admin, and feel Eaton can help, drop me a mail. My main Opensource contribution, for some years, is to lead 'NUT', and the related power management developments and distribution. As such, I also ensure that all this work lands finely in Debian, and is useful and reliable to users. In the past, I've also been working on some Media Center packages (Coherence UPnP, moovida, LIRC, ...), Synaptics touchpads app and some more. Here is the current list of maintained and co-maintained packages, though not up to date. It's not always that easy to balance between upstream lead and the numerous developments on one hand, and Debian packaging and support on the other hand... Not counting real life and the many others responsibilities I have. Just forgive me for the time it has taken (and is still taking, sometimes) to update my various packages. Just remember that I'm always doing my best :-) So dear Planet Debian readers, I'll be bothering you with some news on power management and GreenIT in Debian. There will be both upstream and Debian developments and releases info, so that you all can react. There may probably be some diversion, on other Debian related topics, one way or another... All this will depend on your feedback, and the balance with my available time and pleasure to share with you these few tech bits. cheers,
Arnaud

Christian Perrier: DebConf running: take 7 (Managua "downtown")

For once, I went north. Which means going *into* the city but I also need to explore that part of Managua, right? Therefore, I went down north heading to the lake and Old Cathedral. Surroundings of the Old Cathedral and Presidential House are strange. There is nearly nobody around, few activity. Doesn't look like a city's "heart". I also reached the lakeshore. I already read that managua is more or less "turning its back" to its lake. It's really true. There's nearly no place where one can actually freely reach the lakeshore, at least where I was. Seems that, here also, the nasty habit of bars and restaurants to "privatize" the shores, as one can see in Italy, sometimes Mexico....is also happening. I'm happy we have laws against this in France, indeed. Fun to also see a small recreation park...just like Coney Island, in some way..:) Anyway, I could still reach the lakeshore at some point and take a picture of the Chiltepe Peninsula (guess what on it? A volcano, of course...). I finally went back up through a different neighbourhood than the one I went down. Often "not so nice" streets, in some poor neighbourhood but, again, I always felt very safe....and people are still saying "bon dia" when you meet them. GPS trace is here, as usual.

Jonathan McDowell: SVG::TT::Graph::TimeSeries and the Debian keyring

I'm at DebConf 12 and I've decided to use my time to clear out some minor bits and pieces I've been planning for a while. One of these was to do some graphing of the Debian keyrings over time. The bzr repository goes back to March 2008, but I've also got copies of keyrings for releases back to slink (February 1999). I've been a long time user of GD::Graph under Perl, but recently discovered SVG::TT::Graph and have been meaning to play with it. So I did. First up, number of keys in each keyring: ring-totals.png Most of the interesting data is towards the right, but we can also see the point where our v4 keyring overtook v3 keys back in 2001. More recently there's the end of our v3 support in 2010, and the steady increase of Debian Maintainers. The tiny green line is the Debian non-upload keyring. Next I looked at key size (limiting myself to the DDv4 keyring to make things simpler): key-sizes.png Here we can see the steady increase of 4096 bit keys since 2009, and to a lesser extent 2048 bit keys. There are a few other sizes - 1 10k key, 1 8k key and 2 3k keys (I suspect these are tied to OpenPGPv2 cards). We're up to 28% of the keyring being stronger keys, but there's still a long way to go. (Interestingly the Debian Maintainer keyring is much better with 65% of keys being 2k or larger. The non-upload keyring is all 2k or greater.) Finally I graphed key type, again limiting myself to the DDv4 keyring: key-types.png No real surprises here; DSA far and away the most common with RSA usage increasing as part of the move to larger key sizes. In the past we had a few Elgamal signing keys, but these were shown to be compromised thus disappeared entirely. What do these graphs show me? At least the following: (You should be able to click on the graphs for larger version.)

Julien Danjou: Logitech K750 keyboard and Unifying Receiver Linux support

A year ago, I bought a Logitech Wireless Solar Keyboard K750. I'm particularly picky on keyboards, but this one is good. It has an incredible useful feature: while being wireless, it has no need for disposable or rechargeable batteries, it uses solar power! Logitech Wireless Solar Keyboard K750 My problem is that there's obviously no way to know the battery status from Linux, the provided application only working on Windows. And one dark night, while fragging on QuakeLive, my keyboard stopped working: it had no battery left. This activity being quite energy consuming, it emptied the whole battery. Someone should write code to get the battery status and light meter from Linux: challenge accepted! How the keyboard works Logitech Unifying Receiver This keyboard, like many of the new wireless devices from Logitech, uses the Unifying interface. It's an USB receiver that can be attached up to 6 differents devices (mouse, keyboards ). On old Linux kernel, the Unifying receiver is recognized as only one keyboard and/or one mouse device. Recently, a driver called hid-logitech-dj has been added to the Linux kernel. With this driver, each device attached to the receiver is recognized as one different device. What the Logitech application does Logitech Solar App The Logitech application under Windows works that way: you launch it, and it displays the battery charge level. On the keyboard, there's a special "light" button (up right). When pressed, a LED will light up on the keyboard: green if the keyboard is receiving enough light and is charging, red if the keyboard does not receive enough light and is therefore discharging. Pushing this same button while the application is running will makes the light meter activated: the application will tell you how much lux your keyboard is receiving. Let's reverse engineer this As far as I know, there's nothing in the USB HID protocol that handles this kind of functionality (battery status, light meter ) in a standard way. So the first task to accomplish is, unfortunately, to reverse engineer the program. I discovered a bit too late that Drew Fisher did a good presentation on USB reverse engineering at 28c3. You might want to take a look at it if you want to reverse engineer on USB. I did not need it, but I learned a few things. Anyway, my plan was the following: run the Logitech application inside a virtual machine running Windows, give it direct access to the USB keyboard, and sniff what happens on the USB wire. To achieve that, you need a virtual machine emulator that can do USB pass-through. Both KVM and VirtualBox can do that, but VirtualBox works much better with USB and allow hot(un)plugging of devices, so I used it. To sniff what happens on the USB, you need to load the usbmon Linux kernel module. Simply doing modprobe usbmon will work. You can then use Wireshark which know how to use usbmon devices and understand the USB protocol. USB stuff you need to know You don't need to know much about USB to understand what I'll write about below, but for the sake of comprehensibility I'll write a couple of things here before jumping in. To communicate with an USB device, we communicate with one of its endpoints. Endpoints are regrouped into an interface. Interfaces are regrouped into a configuration. A device might contains one or several configurations. There's also several types of packets in the USB wire protocol, and at least two of them interest us there, they are: All of this and more is well (and better) explained in the chapter 13 of Linux Device Drivers, Third Edition. Sniffed data Once everything was set-up, I ran my beloved Wireshark. There's a an URB of type interrupt sent each time you press any key with some data in it. Therefore I advise you to plug another keyboard (or use the laptop keyboard if you're doing this on a laptop), otherwise you'll get crazy trying to sniff the keyboard you're typing on. At this point, just launching the application does a bunch of USB traffic. Pressing the "light" button on the keyboard makes even more USB packets coming in and out. Here's the interesting packets that I noticed once I excluded the noise: With all this, the next step was clear: understand the packets and reproduce that exchange under Linux. What the packets mean The "go for the light meter" packet The packet sent from the computer to the keyboard is the following. <figure>
Frame 17: 71 bytes on wire (568 bits), 71 bytes captured (568 bits)
    Frame Length: 71 bytes (568 bits)
    Capture Length: 71 bytes (568 bits)
USB URB
    URB id: 0xffff880161997240
    URB type: URB_SUBMIT ('S')
    URB transfer type: URB_CONTROL (0x02)
    Endpoint: 0x00, Direction: OUT
        0... .... = Direction: OUT (0)
        .000 0000 = Endpoint value: 0
    Device: 6
    URB bus id: 1
    Device setup request: relevant (0)
    Data: present (0)
    URB sec: 1340124450
    URB usec: 495643
    URB status: Operation now in progress (-EINPROGRESS) (-115)
    URB length [bytes]: 7
    Data length [bytes]: 7
    [Response in: 18]
    [bInterfaceClass: HID (0x03)]
    URB setup
        bmRequestType: 0x21
            0... .... = Direction: Host-to-device
            .01. .... = Type: Class (0x01)
            ...0 0001 = Recipient: Interface (0x01)
    bRequest: SET_REPORT (0x09)
    wValue: 0x0210
        ReportID: 16
        ReportType: Output (2)
    wIndex: 2
    wLength: 7
0000  40 72 99 61 01 88 ff ff 53 02 00 06 01 00 00 00   @r.a....S.......
0010  22 ad e0 4f 00 00 00 00 1b 90 07 00 8d ff ff ff   "..O............
0020  07 00 00 00 07 00 00 00 21 09 10 02 02 00 07 00   ........!.......
0030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040  10 01 09 03 78 01 00                              ....x..
</figure> What's here interesting is the last part representing the data. wLength says that the length of the data is 7 bytes, so let's take a look at those 7 bytes: 10 01 09 03 78 01 00. Well, actually, you can't decode them like that, unless you're a freak or a Logitech engineer. And I have actually no idea what they mean. But sending this to the keyboard will trigger an interesting thing: the keyboard will start sending URB interrupt with some data without you pressing any more key. The "light meter and battery values" packet This is most interesting packet. This is the one sent by the keyboard to the host and that contains the data we want to retrieve. <figure>
Frame 1467: 84 bytes on wire (672 bits), 84 bytes captured (672 bits)
    Frame Length: 84 bytes (672 bits)
    Capture Length: 84 bytes (672 bits)
USB URB
    URB id: 0xffff88010c43c380
    URB type: URB_COMPLETE ('C')
    URB transfer type: URB_INTERRUPT (0x01)
    Endpoint: 0x83, Direction: IN
        1... .... = Direction: IN (1)
        .000 0011 = Endpoint value: 3
    Device: 2
    URB bus id: 6
    Device setup request: not relevant ('-')
    Data: present (0)
    URB sec: 1334953309
    URB usec: 728740
    URB status: Success (0)
    URB length [bytes]: 20
    Data length [bytes]: 20
    [Request in: 1466]
    [Time from request: 0.992374000 seconds]
    [bInterfaceClass: Unknown (0xffff)]
Leftover Capture Data: 1102091039000c061d474f4f4400000000000000
0000  80 c3 43 0c 01 88 ff ff 43 01 83 02 06 00 2d 00   ..C.....C.....-.
0010  5d c5 91 4f 00 00 00 00 a4 1e 0b 00 00 00 00 00   ]..O............
0020  14 00 00 00 14 00 00 00 00 00 00 00 00 00 00 00   ................
0030  02 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00   ................
0040  11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00   ....9....GOOD...
0050  00 00 00 00                                       ....
</figure> This packets come in regularly (1 per second) on the wire for some time once you sent the "go for the light meter" packet. At one point they are emitted less often and do not contain the value for the light meter anymore, suggesting that the control packet sent earlier triggers the activation of the light meter for a defined period. Now you probably wonder where the data are in this. They're in the 20 bytes leftover in the capture data part, indicated by Wireshark, at the end of the packet: 11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00 00 00 00 00. Fortunately, it was easy to decode. Knowing we're looking for 2 values (battery charge and light meter), we just need to observe and compare the packet emitted on the wire with the values displayed by the Logitech Solar App. To achieve this, I looked both at the Logitech Solar App and Wireshark while bringing more and more light near the keyboard, increasing the lux value received by the meter on the Solar App, and saw that the fields represented in blue (see below) where changing in Wireshark. Since 2 bytes were changing, I guessed that it was coded on 16 bits, and therefore it was easy to correlate the value with the Solar App. <figure>
[ ....9....GOOD....... ]
11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00 00 00 00 00
4 bytes - 1 byte for battery charge - 2 bytes for light meter - 2 bytes - 4 bytes for GOOD - 7 bytes
</figure> In this example, the battery has a charge of 0x39 = 57 % and the light meter receives 0x0c = 12 lux of light. It's basically dark, and that makes sense: it was night and the light was off in my office, the only light being the one coming from my screen. I've no idea what the GOOD part of the packet is about, but it's present in every packet and it's actually very handy to recognize such a packet. Therefore I'm considering this as some sort of useful mark for now. For the other bytes, they were always the same (0x11 0x2 0x9 0x10 at the beginning, 7 times 0x00 at the end). The 2 bytes between the light meter and GOOD probably mean something, but I've no idea what for now. Building our solar app Now we've enough information to build our own very basic solar application. We know how to triggers the light meter, and we know how to decode the packets. We're going to write a small application using libusb. Here's a quick example. It's not perfect and does not check for error codes, be careful. <figure>
/* Written by Julien Danjou <julien> in 2012 */
#include <linux/hid.h>
#include <libusb.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(void)
 
    libusb_context *ctx;
    libusb_init(&ctx);
    libusb_set_debug(ctx, 3);
    /* Look at the keyboard based on vendor and device id */
    libusb_device_handle *device_handle = libusb_open_device_with_vid_pid(ctx, 0x046d, 0xc52b);
    fprintf(stderr, "Found keyboard 0x%p\n", device_handle);
    libusb_device *device = libusb_get_device(device_handle);
    struct libusb_device_descriptor desc;
    libusb_get_device_descriptor(device, &desc);
    for(uint8_t config_index = 0; config_index < desc.bNumConfigurations; config_index++)
     
        struct libusb_config_descriptor *config;
        libusb_get_config_descriptor(device, config_index, &config);
        /* We know we want interface 2 */
        int iface_index = 2;
        const struct libusb_interface *iface = &config->interface[iface_index];
        for (int altsetting_index = 0; altsetting_index < iface->num_altsetting; altsetting_index++)
         
            const struct libusb_interface_descriptor *iface_desc = &iface->altsetting[al>tsetting_index];
            if (iface_desc->bInterfaceClass == LIBUSB_CLASS_HID)
             
                libusb_detach_kernel_driver(device_handle, iface_index);
                libusb_claim_interface(device_handle, iface_index);
                unsigned char ret[65535];
                unsigned char payload[] = "\x10\x02\x09\x03\x78\x01\x00";
                if(libusb_control_transfer(device_handle,
                                           LIBUSB_REQUEST_TYPE_CLASS   LIBUSB_RECIPIENT_INTERFACE,
                                           HID_REQ_SET_REPORT,
                                           0x0210, iface_index, payload, sizeof(payload) - 1, 10000))
                 
                    int actual_length = 0;
                    while(actual_length != 20   strncmp((const char *) &ret[9], "GOOD", 4))
                        libusb_interrupt_transfer(device_handle,
                                                  iface_desc->endpoint[0].bEndpointAddress,
                                                  ret, sizeof(ret), &actual_length, 100000);
                    uint16_t lux = ret[5] << 8   ret[6];
                    fprintf(stderr, "Charge: %d %%\nLight: %d lux\n", ret[4], lux);
                 
                libusb_release_interface(device_handle, iface_index);
                libusb_attach_kernel_driver(device_handle, iface_index);
             
         
     
    libusb_close(device_handle);
    libusb_exit(ctx);
 
</figure> What the program is doing is the following: This gives the following:
Found keyboard 0x0x24ec8e0
Charge: 64 %
Light: 21 lux
Challenge accomplished! To be continued Unfortunately, this approach has at least one major drawback. We have to disconnect the Logitech Unifying Receiver from the kernel. That means that while we're waiting for the packet, we're dropping packets corresponding to other events from every connected device (key presses, pointer motions ). In order to solve that, I sent a request for help on the linux-input mailing list. That way, I learned that Logitech is actually using the HID++ protocol to communicate with the devices using the Unifying Receiver. Lars-Dominik Braun managed to get the HID++ specifications from Logitech and published them with their authorization. This opens a whole new world. With that document, I may be able to understand the part I reverse engineered and convert this to a more useful and generic library using the hidraw interface (so we don't have to disconnect the devices from the kernel driver).

Steve McIntyre: CD work at DebCamp

Great productive couple of days hacking on debian-cd with Joey. I don't do DebCamp often enough - it's great fun...

DebConf team: Many thanks and kudos to the videoteam! (Posted by Holger Levsen)

Thanks to the awesome work of a lot of people (some not even here), this time we did the setup in one day less then previous years, and IMHO it was mostly rather very relaxed and efficient as well!
As you probably will know already, the streams are linked on the DebConf12 webpage and the recordings will be linked there, too, when they are ready. The scheduled release for that is tomorrow evening localtime, we ll see if we can deliver :-)
To make this happen, there will be a video training session tomorrow morning at 11:00 in talkroom 2, Roberto Ter n. The training will cover both the live session setup and the reviewing workflow and is definitly still open for new members!
So once again (one cannot say it enough, so please express your thanks as well) muchisimas gracias to all members of the dc12 videoteam! As Biella once said, you re ninjas!
Last and definitly not least, many thanks to our equipment sponsors too (HP, IRILL, Debian and individuals)!
cheers, a very happy h01ger!

Benjamin Mako Hill: Wiki Conferencing

I am in Berlin for the Wikipedia Academy, a very cool hybrid free culture community plus refereed academic conference organized, in part, by Wikimedia Deutschland. On Friday, I was very excited to have been invited to give the conference's opening keynote based on my own hybrid take on learning from failures in peer production and incorporating a bunch of my own research. Today, I was on a panel at the conference about free culture and sharing practices. I'll post talks materials and videos when the conference puts them online. I will be in Berlin for the next week or so before I head to directly to Washington, DC for Wikimania between the 11th and 15th. I'll be giving three talks there: Between then and now, I'm taking the next week in Berlin to catch up on work, and with friends. If you're in either place and want to meet up, please get in touch and lets try to arrange something.

Benjamin Mako Hill: The Global Iron Blogger Network

Since last November, I've been participating in and coordinating Iron Blogger: a drinking club where you pay $5 to a "beer" pool if you fail to blog weekly. The revival of Iron Blogger in Boston has been a big success. Even more exciting, however, is that Iron Blogger concept has spread. There are now two other Iron Blogger instances: in San Francisco coordinated by Parker Higgens, and in Berlin run by Nicole Ebber and Michelle Thorne. Yesterday, we convened a virtual meeting of the Global Iron Blogger Council (i.e., an email thread) and we all agreed a new on iron blogger rule that might sweeten the deal for jet-setting prospective Iron Bloggers: any paid-up member of any Iron Blogger club can attend meet-ups in any other Iron Blogger cities if they happen to be in town for one. Because We Are One. If you want to join us in Boston, we have some room through attrition. Rust bloggers, perhaps? If you'd like to join, you should contact me. And if you'd like to set up your own in a different city, the code is in git. One warning, however. As those of us that have set it up have figured out, the documentation for the software to run Iron Blogger is between poor and non-existent. If you do want to set up your own instance, please get in touch. I'm happy to give you some pointers that you'll probably need but, more importantly, I'd like to work with the next brave soul to put together documentation of the setup process along the way.

Russell Coker: Breaking SATA Connectors

I ve just broken my second SATA connector. This isn t a lot considering the number of hard drives I ve worked with, but it s still really annoying as I generally don t break things. The problem is that unplugging a SATA cable requires pushing a little clip, this isn t overly difficult but it unfortunately doesn t fit well with habits formed from previous hardware. The power cables used for hard drives based on the ST-506 interface which was copied for the IDE interface was large and had a fairly tight fit. Removing such a cable requires a significant amount of force which is about the same as the amount of force required to break a SATA connector. When I first started using PCs a reasonably configured AT system cost over $5,000 (maybe something like $10,000 in today s money). With that sort of price hardly anyone had a set of test PCs. When hardware prices dropped such that hard drives of reasonable size became reasonably affordable on the second-hand market I bought more disks and used some for extra storage and some for testing software. As there was nothing like VMWare for testing OS images the way to test a new OS was to plug in a different hard drive and boot it. So I got a lot of practice at removing IDE power cables with as much force as was necessary. Now I own a pile of test PCs, SATA disks less than 100G are free, I use Xen for a lot of my testing, and generally I have much less need to swap hard drives around. In most situations in which I would swap hard drives in the 90 s I will now swap PCs and I have piles of PCs ready for this purpose. So I haven t had enough practice with SATA disks to develop habits for safely removing them. So far this lack of habit development has resulted in damaging two disks due to changing drives while not concentrating enough. Fortunately duct-tape works well for holding a SATA connector in place when the plastic that attaches to the clip is broken. Related posts:
  1. Strange SATA Disk Performance Below is a GNUPlot graph of ZCAV output from a...
  2. Vibration and Strange SATA Performance Almost two years ago I blogged about a strange performance...
  3. Dell PowerEdge T105 Today I received a Dell PowerEDGE T105 for use by...

Russell Coker: Postfwd and Local Only Email

Over a year ago when I was considering my first Android phone purchase I setup a test account on my mail server so that I could test email clients on phones and tablets. I used a short password because I didn t want to type a lot on small screens and because typing a password into a random system owned by someone else isn t particularly secure anyway. Then I forgot about the account until I noticed that my mail server was sending out spam. Next time I setup such a test account I ll put rules similar to the following in my Postfwd [1] configuration to stop Postfix from sending such messages. That will prevent the test account from receiving mail from outside or sending mail out of the server. The former is optional (getting a few thousand spam messages in an unused test account is no big deal) but the latter is needed to prevent getting my server blacklisted. id=R_test_recipient ; recipient==test@coker.com.au ; sender!~.*@coker.com.au ; action=REJECT
id=R_test_sender ; sender==test@coker.com.au ; recipient!~.*@coker.com.au ; action=REJECT Related posts:
  1. email disclaimers Andre Pang blogs about the annoyance of email disclaimers. For...
  2. Email Passwords I was doing some routine sysadmin work for a client...
  3. Some Postfix Scripts for dealing with Outbound Spamming I ve just written some small scripts to help me manage...

Joey Hess: debian-cd work at DebCamp

I've been working the past two days on debian-cd, which was the main thing (besides git-annex assistant) I planned to work on at DebCamp. Yesterday, Steve McIntyre and I cleaned up some cruft in debian-cd's package lists. This freed somewhere in the area of 30 mb. I also took a grep through d-i and made sure the CDs include packages that d-i can optionally install in some situations. Today, I investigated how Recommends are ordered on the CD, and concluded it's as close to optimal as can be easily achieved. So was not able to save space there, but I did find a way to reorganize the desktop task that avoids needing to include a lot of printing stuff and some other stuff on the first CD. While that helped some, it still didn't get either Gnome or Kde to entirely fit, so getting there will probably involve rebuilding 100 or so packages with xz, if someone decides to do that. So it's stil TBD whether Gnome or Kde will fit on a single CD in Debian wheezy. At this point I think most of us are getting tired of fighting this increasingly losing battle every release, and so other options like only having a desktop DVD are looking more appealing, as they solve the problem long-term. This would also free up the first CD for other interesting use cases, perhaps xfce, or perhaps a CD targeted at server users, and/or containing high-popularity non-desktop packages.

Next.

Previous.