Search Results: "reg"

13 July 2012

Dirk Eddelbuettel: RcppArmadillo 0.3.2.4

Conrad released version 3.2.4 of Armadillo yesterday. It contains a workaround for g++ 4.7.0 and 4.7.1 which have a regression triggered by the Armadillo codebase for small fixed-sized matrices. The corresponding RcppArmadillo package 0.3.2.4 arrived on CRAN earlier today. The short NEWS entry follows below.
0.3.2.4  2012-07-11
    o   Upgraded to Armadillo release 3.2.4
          * workaround for a regression (bug) in GCC 4.7.0 and 4.7.1
Courtesy of CRANberries, there is also a diffstat report for 0.3.2.4 relative to 0.3.2.3 As always, more detailed information is on the RcppArmadillo page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

12 July 2012

Christian Perrier: DebConf running: stages 8 and 9

I failed. I didn't go running the day after the Cheese and Wine party (tuesday). I wonder why...:-) Indeed, I was feeling well, but slightly tired and I woke up at 8am, which means that I would have to go out quite late, in the heat...and run while feeling tired. And something I have learnt in all my running activities is that when one feels not in shape, better not insist. This is where injuries can come. Moreover, the next day was meant to be the Day Trip, with a goal for me: climb the Cerro negro volcano as fast as I can, training with my sticks for difficult climbs, in the heat...in the perspective of future races I would like to do in French Alps, in the upcoming years (such as La Montagn'hard : I recommend watching the video). Sadly, a landslide has cut the road 2km away from the starting point and, despite my desperate attempts to still form a group of "strong people" to go there and do the extra 4km walk anyway....we later learned that the entire road to Cerro Negro is closed and we'd have no chance to go on it. That infortunate, though I hopefully have another chance in the upcoming two weeks, where I'm supposed to come back and climb the volcano with Elizabeth (but, I of course will not leave my beloved one alone on a volcano!). So, I ended up staying on the beach at Las Pe itas. And, as I explained to No l, on days where I *decided* to run, I *need* to run. It's like a drug as you'd guess. So, I went running on the beach..:-). In the early afternoon, during a very hot day (probably the best weather we ever had, which makes me even more regret "my" volcano), about 34 C, no shade...and some wind, and sand...:-) Running on the beach is not as hard as one would imagine as long as you stay at the limit of the water, where the sand is compact. Indeed, this is even good for articulations as the sand soil is of course soft and absorbs impacts. The only drawback is that this beach is quite not so flat and this is sometimes like running across a hill all time long. Anyway, I went north up to Poneloya, up to the point where a river blocks the path along the beach, then I headed back. I had originally the intent to continue south-east, past the place where we were, up to the start of Juan Venaco island. However, the face wind when comign back, as well as the very hot sun, really achieved me....and I decided it would be wiser to stop. I however couldn't resist running IN the waves, which means I now have very humid shoes (but I have two pairs of those...:-)) Today (Thursday), was a more standard run in Managua, with No l and Ralf. We once again went to the South suburbs hills. All three of us were suite tired as it seems, so we chose a quite slow pace and, still, the climb was not that easy. We went the same way I went with Ralf a few days ago, but we didn't enter the "dogs path" this time, but rather tried to continue the road south. After 4.5 km, we had however to stop because the path didn't go further and, anyway, I think it was enough for all of us..:-) The way back, we tried another way, which was not particularly touristic (close to the National Soccer Stadium....a not so impressive stadium, but soccer is not the national sport here in Nicaragua, where people are more in base-ball or boxing). Both GPS traces: See you tomorrow for yet another GPS trace..:-)

Ritesh Raj Sarraf: Reporting bugs with Apport

So yet another bug reporting tool. :-)When I prepared Apport for Debian, I wasn't sure how it will look going forward. If you look at the current one lying in experimental, it just detects your crashes and pops up in your systray. It doesn't have the mechanism to interact with BTS.So, like the title of this post says, this is the first worthy feature for Apport in the context of Debian.Apport Detail ReportNothing special here. You would have seen a similar window before if you have installed apport. What changes with this release is, that now, when you check the Send an error report to help fix this problem, it will really file a bug report on the BTS.Here's what the emailed bug report will look like:
Package: leafnode
Version: 2.0.0.alpha20090406a-1
=============================
ProblemType: Crash
Architecture: amd64
Date: Tue Jul  3 00:08:02 2012
Dependencies:
 adduser 3.113+nmu3
 base-passwd 3.5.24
 cron 3.0pl1-123
 debconf 1.5.44
 debianutils 4.3.1
 dpkg 1.16.4.3
 gcc-4.7-base 4.7.1-2
 libbz2-1.0 1.0.6-3
 libc-bin 2.13-33
 libc6 2.13-33
 libclass-isa-perl 0.36-3
 libdb5.1 5.1.29-4
 libfile-copy-recursive-perl 0.38-1
 libgcc1 1:4.7.1-2
 libgdbm3 1.8.3-11
 liblzma5 5.1.1alpha+20120614-1
 libpam-modules 1.1.3-7.1
 libpam-modules-bin 1.1.3-7.1
 libpam-runtime 1.1.3-7.1
 libpam0g 1.1.3-7.1
 libpcre3 1:8.30-5
 libpopt0 1.16-7
 libselinux1 2.1.9-5
 libsemanage-common 2.1.6-6
 libsemanage1 2.1.6-6
 libsepol1 2.1.4-3
 libswitch-perl 2.16-2
 libustr-1.0-1 1.0.4-3
 libwrap0 7.6.q-23
 logrotate 3.8.1-4
 lsb-base 4.1+Debian7 [modified: lib/lsb/init-functions]
 multiarch-support 2.13-33
 netbase 5.0
 openbsd-inetd 0.20091229-2
 passwd 1:4.1.5.1-1
 perl 5.14.2-12
 perl-base 5.14.2-12
 perl-modules 5.14.2-12
 sensible-utils 0.0.7
 tar 1.26-4
 tcpd 7.6.q-23
 update-inetd 4.43
 zlib1g 1:1.2.7.dfsg-13
Disassembly:
 => 0x7f8e69738475 <*__GI_raise+53>:	cmp    $0xfffffffffffff000,%rax
    0x7f8e6973847b <*__GI_raise+59>:	ja     0x7f8e69738492 <*__GI_raise+82>
    0x7f8e6973847d <*__GI_raise+61>:	repz retq 
    0x7f8e6973847f <*__GI_raise+63>:	nop
    0x7f8e69738480 <*__GI_raise+64>:	test   %eax,%eax
    0x7f8e69738482 <*__GI_raise+66>:	jg     0x7f8e69738465 <*__GI_raise+37>
    0x7f8e69738484 <*__GI_raise+68>:	test   $0x7fffffff,%eax
    0x7f8e69738489 <*__GI_raise+73>:	jne    0x7f8e697384a2 <*__GI_raise+98>
    0x7f8e6973848b <*__GI_raise+75>:	mov    %esi,%eax
    0x7f8e6973848d <*__GI_raise+77>:	nopl   (%rax)
    0x7f8e69738490 <*__GI_raise+80>:	jmp    0x7f8e69738465 <*__GI_raise+37>
    0x7f8e69738492 <*__GI_raise+82>:	mov    0x34e97f(%rip),%rdx        # 0x7f8e69a86e18
    0x7f8e69738499 <*__GI_raise+89>:	neg    %eax
    0x7f8e6973849b <*__GI_raise+91>:	mov    %eax,%fs:(%rdx)
    0x7f8e6973849e <*__GI_raise+94>:	or     $0xffffffff,%eax
    0x7f8e697384a1 <*__GI_raise+97>:	retq
DistroRelease: Debian 7.0
ExecutablePath: /usr/sbin/fetchnews
ExecutableTimestamp: 1265584779
Package: leafnode 2.0.0.alpha20090406a-1
PackageArchitecture: amd64
ProcCmdline: /usr/sbin/fetchnews
ProcCwd: /
ProcEnviron:
 LANGUAGE=en_US:en
 LC_TIME=en_IN.UTF-8
 LC_MONETARY=en_IN.UTF-8
 PATH=(custom, no user)
 LC_ADDRESS=en_IN.UTF-8
 LANG=en_US.UTF-8
 LC_TELEPHONE=en_IN.UTF-8
 LC_NAME=en_IN.UTF-8
 SHELL=/bin/sh
 LC_MEASUREMENT=en_IN.UTF-8
 LC_NUMERIC=en_IN.UTF-8
 LC_PAPER=en_IN.UTF-8
ProcMaps:
 00400000-00421000 r-xp 00000000 08:06 4464934                            /usr/sbin/fetchnews
 00621000-00622000 rw-p 00021000 08:06 4464934                            /usr/sbin/fetchnews
 00622000-00623000 rw-p 00000000 00:00 0 
 00be4000-00c05000 rw-p 00000000 00:00 0                                  [heap]
 7f8e68edc000-7f8e68eef000 r-xp 00000000 08:06 1179776                    /lib/x86_64-linux-gnu/libresolv-2.13.so
 7f8e68eef000-7f8e690ee000 ---p 00013000 08:06 1179776                    /lib/x86_64-linux-gnu/libresolv-2.13.so
 7f8e690ee000-7f8e690ef000 r--p 00012000 08:06 1179776                    /lib/x86_64-linux-gnu/libresolv-2.13.so
 7f8e690ef000-7f8e690f0000 rw-p 00013000 08:06 1179776                    /lib/x86_64-linux-gnu/libresolv-2.13.so
 7f8e690f0000-7f8e690f2000 rw-p 00000000 00:00 0 
 7f8e690f2000-7f8e690f7000 r-xp 00000000 08:06 1180036                    /lib/x86_64-linux-gnu/libnss_dns-2.13.so
 7f8e690f7000-7f8e692f6000 ---p 00005000 08:06 1180036                    /lib/x86_64-linux-gnu/libnss_dns-2.13.so
 7f8e692f6000-7f8e692f7000 r--p 00004000 08:06 1180036                    /lib/x86_64-linux-gnu/libnss_dns-2.13.so
 7f8e692f7000-7f8e692f8000 rw-p 00005000 08:06 1180036                    /lib/x86_64-linux-gnu/libnss_dns-2.13.so
 7f8e692f8000-7f8e692fa000 r-xp 00000000 08:06 1183392                    /lib/libnss_mdns4_minimal.so.2
 7f8e692fa000-7f8e694f9000 ---p 00002000 08:06 1183392                    /lib/libnss_mdns4_minimal.so.2
 7f8e694f9000-7f8e694fa000 rw-p 00001000 08:06 1183392                    /lib/libnss_mdns4_minimal.so.2
 7f8e694fa000-7f8e69505000 r-xp 00000000 08:06 1180055                    /lib/x86_64-linux-gnu/libnss_files-2.13.so
 7f8e69505000-7f8e69704000 ---p 0000b000 08:06 1180055                    /lib/x86_64-linux-gnu/libnss_files-2.13.so
 7f8e69704000-7f8e69705000 r--p 0000a000 08:06 1180055                    /lib/x86_64-linux-gnu/libnss_files-2.13.so
 7f8e69705000-7f8e69706000 rw-p 0000b000 08:06 1180055                    /lib/x86_64-linux-gnu/libnss_files-2.13.so
 7f8e69706000-7f8e69883000 r-xp 00000000 08:06 1179700                    /lib/x86_64-linux-gnu/libc-2.13.so
 7f8e69883000-7f8e69a83000 ---p 0017d000 08:06 1179700                    /lib/x86_64-linux-gnu/libc-2.13.so
 7f8e69a83000-7f8e69a87000 r--p 0017d000 08:06 1179700                    /lib/x86_64-linux-gnu/libc-2.13.so
 7f8e69a87000-7f8e69a88000 rw-p 00181000 08:06 1179700                    /lib/x86_64-linux-gnu/libc-2.13.so
 7f8e69a88000-7f8e69a8d000 rw-p 00000000 00:00 0 
 7f8e69a8d000-7f8e69a95000 r-xp 00000000 08:06 1180031                    /lib/x86_64-linux-gnu/libcrypt-2.13.so
 7f8e69a95000-7f8e69c94000 ---p 00008000 08:06 1180031                    /lib/x86_64-linux-gnu/libcrypt-2.13.so
 7f8e69c94000-7f8e69c95000 r--p 00007000 08:06 1180031                    /lib/x86_64-linux-gnu/libcrypt-2.13.so
 7f8e69c95000-7f8e69c96000 rw-p 00008000 08:06 1180031                    /lib/x86_64-linux-gnu/libcrypt-2.13.so
 7f8e69c96000-7f8e69cc4000 rw-p 00000000 00:00 0 
 7f8e69cc4000-7f8e69cc6000 r-xp 00000000 08:06 1180047                    /lib/x86_64-linux-gnu/libdl-2.13.so
 7f8e69cc6000-7f8e69ec6000 ---p 00002000 08:06 1180047                    /lib/x86_64-linux-gnu/libdl-2.13.so
 7f8e69ec6000-7f8e69ec7000 r--p 00002000 08:06 1180047                    /lib/x86_64-linux-gnu/libdl-2.13.so
 7f8e69ec7000-7f8e69ec8000 rw-p 00003000 08:06 1180047                    /lib/x86_64-linux-gnu/libdl-2.13.so
 7f8e69ec8000-7f8e69ed5000 r-xp 00000000 08:06 1179893                    /lib/x86_64-linux-gnu/libpam.so.0.83.0
 7f8e69ed5000-7f8e6a0d4000 ---p 0000d000 08:06 1179893                    /lib/x86_64-linux-gnu/libpam.so.0.83.0
 7f8e6a0d4000-7f8e6a0d5000 r--p 0000c000 08:06 1179893                    /lib/x86_64-linux-gnu/libpam.so.0.83.0
 7f8e6a0d5000-7f8e6a0d6000 rw-p 0000d000 08:06 1179893                    /lib/x86_64-linux-gnu/libpam.so.0.83.0
 7f8e6a0d6000-7f8e6a112000 r-xp 00000000 08:06 1182916                    /lib/x86_64-linux-gnu/libpcre.so.3.13.1
 7f8e6a112000-7f8e6a312000 ---p 0003c000 08:06 1182916                    /lib/x86_64-linux-gnu/libpcre.so.3.13.1
 7f8e6a312000-7f8e6a313000 rw-p 0003c000 08:06 1182916                    /lib/x86_64-linux-gnu/libpcre.so.3.13.1
 7f8e6a313000-7f8e6a333000 r-xp 00000000 08:06 1180134                    /lib/x86_64-linux-gnu/ld-2.13.so
 7f8e6a503000-7f8e6a507000 rw-p 00000000 00:00 0 
 7f8e6a530000-7f8e6a532000 rw-p 00000000 00:00 0 
 7f8e6a532000-7f8e6a533000 r--p 0001f000 08:06 1180134                    /lib/x86_64-linux-gnu/ld-2.13.so
 7f8e6a533000-7f8e6a534000 rw-p 00020000 08:06 1180134                    /lib/x86_64-linux-gnu/ld-2.13.so
 7f8e6a534000-7f8e6a535000 rw-p 00000000 00:00 0 
 7fffbc06e000-7fffbc08f000 rw-p 00000000 00:00 0                          [stack]
 7fffbc1ff000-7fffbc200000 r-xp 00000000 00:00 0                          [vdso]
 ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
ProcStatus:
 Name:	fetchnews
 State:	S (sleeping)
 Tgid:	6872
 Pid:	6872
 PPid:	6871
 TracerPid:	0
 Uid:	9	9	9	9
 Gid:	9	9	9	9
 FDSize:	64
 Groups:	9 
 VmPeak:	   21440 kB
 VmSize:	   21276 kB
 VmLck:	       0 kB
 VmPin:	       0 kB
 VmHWM:	     984 kB
 VmRSS:	     984 kB
 VmData:	     380 kB
 VmStk:	     136 kB
 VmExe:	     132 kB
 VmLib:	    2132 kB
 VmPTE:	      64 kB
 VmSwap:	       0 kB
 Threads:	1
 SigQ:	0/23227
 SigPnd:	0000000000000000
 ShdPnd:	0000000000000000
 SigBlk:	0000000000000000
 SigIgn:	0000000000000000
 SigCgt:	0000000000000000
 CapInh:	0000000000000000
 CapPrm:	0000000000000000
 CapEff:	0000000000000000
 CapBnd:	ffffffffffffffff
 Cpus_allowed:	f
 Cpus_allowed_list:	0-3
 Mems_allowed:	00000000,00000001
 Mems_allowed_list:	0
 voluntary_ctxt_switches:	6
 nonvoluntary_ctxt_switches:	1
Registers:
 rax            0x0	0
 rbx            0x0	0
 rcx            0xffffffffffffffff	-1
 rdx            0x6	6
 rsi            0x1ad8	6872
 rdi            0x1ad8	6872
 rbp            0x0	0x0
 rsp            0x7fffbc08d4f8	0x7fffbc08d4f8
 r8             0x7f8e6a504700	140249645729536
 r9             0x6d6f642064656966	7885631562835126630
 r10            0x8	8
 r11            0x246	582
 r12            0x0	0
 r13            0x7fffbc08d920	140736348084512
 r14            0x0	0
 r15            0x0	0
 rip            0x7f8e69738475	0x7f8e69738475 <*__GI_raise+53>
 eflags         0x246	[ PF ZF IF ]
 cs             0x33	51
 ss             0x2b	43
 ds             0x0	0
 es             0x0	0
 fs             0x0	0
 gs             0x0	0
Signal: 6
SourcePackage: leafnode
Stacktrace:
 #0  0x00007f8e69738475 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
         pid = <optimized out>
         selftid = <optimized out>
 #1  0x00007f8e6973b6f0 in *__GI_abort () at abort.c:92
         act =  __sigaction_handler =  sa_handler = 0x7f8e69850d3d, sa_sigaction = 0x7f8e69850d3d , sa_mask =  __val =  140736348083740, 140249634747968, 0, 0, 140736348084512, 140249631083424, 140249645738440, 0, 4294967295, 206158430232, 1, 6427272, 0, 0, 0, 0 , sa_flags = 1781664242, sa_restorer = 0x1 
         sigs =  __val =  32, 0 <repeats 15 times> 
 #2  0x0000000000416292 in ?? ()
 No symbol table info available.
 #3  0x0000000000411c80 in ?? ()
 No symbol table info available.
 #4  0x0000000000406952 in ?? ()
 No symbol table info available.
 #5  0x00007f8e69724ead in __libc_start_main (main=<optimized out>, argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffbc08d918) at libc-start.c:228
         result = <optimized out>
         unwind_buf =  cancel_jmp_buf =  jmp_buf =  0, 498162418289285118, 4206480, 140736348084512, 0, 0, -498021567843461122, -435434157313288194 , mask_was_saved = 0 , priv =  pad =  0x0, 0x0, 0x418360, 0x7fffbc08d928 , data =  prev = 0x0, cleanup = 0x0, canceltype = 4293472 
         not_first_call = <optimized out>
 #6  0x0000000000402fb9 in ?? ()
 No symbol table info available.
 #7  0x00007fffbc08d918 in ?? ()
 No symbol table info available.
 #8  0x000000000000001c in ?? ()
 No symbol table info available.
 #9  0x0000000000000001 in ?? ()
 No symbol table info available.
 #10 0x00007fffbc08ee96 in ?? ()
 No symbol table info available.
 #11 0x0000000000000000 in ?? ()
 No symbol table info available.
StacktraceTop:
 ?? ()
 ?? ()
 ?? ()
 __libc_start_main (main=<optimized out>, argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffbc08d918) at libc-start.c:228
 ?? ()
ThreadStacktrace:
 .
 Thread 1 (LWP 6872):
 #0  0x00007f8e69738475 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64
         pid = <optimized out>
         selftid = <optimized out>
 #1  0x00007f8e6973b6f0 in *__GI_abort () at abort.c:92
         act =  __sigaction_handler =  sa_handler = 0x7f8e69850d3d, sa_sigaction = 0x7f8e69850d3d , sa_mask =  __val =  140736348083740, 140249634747968, 0, 0, 140736348084512, 140249631083424, 140249645738440, 0, 4294967295, 206158430232, 1, 6427272, 0, 0, 0, 0 , sa_flags = 1781664242, sa_restorer = 0x1 
         sigs =  __val =  32, 0 <repeats 15 times> 
 #2  0x0000000000416292 in ?? ()
 No symbol table info available.
 #3  0x0000000000411c80 in ?? ()
 No symbol table info available.
 #4  0x0000000000406952 in ?? ()
 No symbol table info available.
 #5  0x00007f8e69724ead in __libc_start_main (main=<optimized out>, argc=<optimized out>, ubp_av=<optimized out>, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffbc08d918) at libc-start.c:228
         result = <optimized out>
         unwind_buf =  cancel_jmp_buf =  jmp_buf =  0, 498162418289285118, 4206480, 140736348084512, 0, 0, -498021567843461122, -435434157313288194 , mask_was_saved = 0 , priv =  pad =  0x0, 0x0, 0x418360, 0x7fffbc08d928 , data =  prev = 0x0, cleanup = 0x0, canceltype = 4293472 
         not_first_call = <optimized out>
 #6  0x0000000000402fb9 in ?? ()
 No symbol table info available.
 #7  0x00007fffbc08d918 in ?? ()
 No symbol table info available.
 #8  0x000000000000001c in ?? ()
 No symbol table info available.
 #9  0x0000000000000001 in ?? ()
 No symbol table info available.
 #10 0x00007fffbc08ee96 in ?? ()
 No symbol table info available.
 #11 0x0000000000000000 in ?? ()
 No symbol table info available.
Title: fetchnews crashed with SIGABRT in __libc_start_main()
Uname: Linux 3.4-trunk-amd64 x86_64
UserGroups: 
The first 2 lines should be enough for the BTS server to file the bug report to the correct package and against the correct version.If you paid attention in the screen shot and the email report, you will notice that the email report does not include the CoreDump section. This was knocked off intentionally as we currently might not need it. If there is a need, the data is always available on the user's apport database, and the dev/user and seek it on a need basis. But in case, if a day comes when we really want it, that too is possible. The same email report will have the CoreDump seciton with a section like the following:
CoreDump: base64 
H4sICAAAAAAC/0NvcmVEdW1wAOx9C0AU1f7/7LLAggirouF7VDQ0xcVXWGqLoq4Kur6KMmV5CoqwwqJYVquggYqhlVm3B3mt7OWlbnXtZeubHtfobc9Lb6xUemiU5f7PmfkcmBl3VMzbvff3Px89fPb7nfOeM2fOOfOdMzeNT5pgNBgEBpMwRmiRBMEmnA6bEC8MbfYvI6eN4BePBfvX0zQC6Q9RJyULY6NKpuGC6A8rO56uE07wH44lI7YynINlc6cmnFE3n1L5nNB7j+Ssv8lPelbDaeEkuBCu8ag6HENjz9PCmZTh6sNz/aanVz4PS6+V4aoQTohQhxM1rK2XGoQTI/yn5xD814sX4VyacGerFxbOObh15atj6Z17OKl89Qjn0Qkn6pSvEeGqB59z+YKU4aqmtS6fQgDS0wlXo5NPC8I5HK07DyyczdW68yCy9FytK58V4Zw64eou8l8+G8JZy/2Xj51Abfmaw21Th7Np+LTrD+Fc21p5/SGcRxPOoX8dye0T4ep00vPotU92Hh5u3Xln4Wwvt6587A7jaGU4C8I5X/afz3qd/pqFs73WuutWZOm91rr20hzuu9aVz4pwru9adz3YEM6jE65Op3wOdh6OtO68s3C2tgtaVT4nS+/cw0nlc7H61AnnDfBfPg/Cie0WnOv5C1KFi2tdPqsQzqoTTtAZ91SzcI4Frepf6tlIbeaCc71PyyECcXxm686fBeGsrQwnIpytleGsCFcX9IJPVT6jmrXnwRHM2q863NnOn5MNbJ0+irOFo2EcBjn8uGkzxgvN/ZuMID8t7lAXQXiPuHfhLjTE983qMmswAVVmhpydm5UpZhYsEiaq254PCFLkW8nfQF9cViT1HUM18Te9LefjEhahJv6zgV3DtI7LMaJhdSzMwMFo+cIIgdPWbXaWOyMnP2tpEbwPLi4qHFyUnps/uPmI+EfqOhBD/SBF3nq19J1S+VmT8Z389iVVm8CBdhAzwQHNl 7xNNReLV6SpPH/NUzH0OWZNHkPAVYPl+Npo9GEaua1GDtfIF2ni7wz+5QP5fOOOIXxxTJYjWPgP/bfLQHSLRkUd3hTgv749PWm7FP7r0FcDP15U/cmO9fL4jLFNgzkaiBr4zwWL//QaYtey3vH/NtBrxEbbBHFJk6bOTuFt4v9cmwj4N/vn4ODg4ODg4ODg4OD4/wE3aZ7/G/H8n60B2aDfFmVs9kOf/5vJ325CV2n+HSgol59tKvYiasZmxRzNJCeIhG0q7gY1Y4OCA1UlsKl4b5hRxWxhu2Vdmq0Dp6vYi8UxS5SgCmdEuBiEi4F

Now, Is this going to be useful? Will we get flooded with bug reports?
 
Usefulness: I think this will uncover many bugs that might be getting unnoticed today. Take this example itself. This bug was detect against the leafnode package's fetchnews binary. That binary does a quiet job in the background, fetching news. I never was aware that it had been crashing. As a system-wide monitoring tool, apport was able to detect, trap and inform the user of the crash. So I think this will be useful and improve the overall quality.
Abuse/Flood: This is something I can't predict. Perhaps it will bring in challenges if people just blindly click on the Send Report button. One option can be to have the "Send Report" checkbox unchecked by default. That'll hopefully lower down the possibilities.
 
What do you think? Let me know your comments.
This change will soon be pushed to apport in experimental. Hopefully this weekend.
 
I want to wrap up this post with a Thank You note to Martin Pitt and his team who created Apport, and with such simplicity. It took me 30 lines of code to adapt it to Debian. My intent with Apport is to keep the changes to the minimum so that we can always leverage new features and fixes that Apport brings in. So, as far as I'll try, there will be no drift from its original shape.
Categories:
Keywords:

11 July 2012

Paul Tagliamonte: The "hell-server"

I s pose a better name would be How to preform automated deep-source checks I ve pushed the code to paultag/qlight, and it s just for personal use at the moment. It s built to use paultag/sb around sbuild. Basically, it preforms a crudload of tests such as licensecheck, suspicious-source, pyflakes, bashisms, cppcheck, php in linter mode, and a ton of other checks. You can see a large majority of them in the source file. The batch runner is a singleton , so it s safely called more then once. It s running on a 5-minute cron, and I ve configured a dput target to push it into it s queue & run the enqueue script. Here s some example output from a random package I had lying around:
  ===== build status =====
sbo-run-clang  successful
sbo-run-gcc  successful
  ===== global status =====
Ran plugin extract, exit code 0
Ran plugin watch, exit code 0
Ran plugin licensecheck, exit code 0
Ran plugin validate-source, exit code 0
Ran plugin build, exit code 0
Ran plugin descr, exit code 0
Ran plugin lintian, exit code 0
  ===== descr =====
I: PT: Does this sound right? "gbemol is/are a(n) Graphical frontend for the Music Player Daemon (MPD)"
  ===== piuparts =====
   [last 20 lines]
  ===== licenses =====
BSD (3 clause) 
GPL (v2 or later) 
GPL (v2 or later) GENERATED FILE
GPL (v2 or later) (with incorrect FSF address) 
LGPL (v2 or later) (with incorrect FSF address) 
*No copyright* GPL (v2 or later) 
*No copyright* UNKNOWN
  ===== lintian =====
I: gbemol: spelling-error-in-binary usr/bin/gbemol Wheter Whether
W: gbemol: hardening-no-relro usr/bin/gbemol
I: gbemol: spelling-error-in-manpage usr/share/man/man1/gbemol.1.gz allows to allows one to
  ===== watch =====
-- Scanning for watchfiles in .
-- Found watchfile in ./debian
-- In debian/watch, processing watchfile line:
   http://sf.net/gbemol/gbemol-(.*)\.tar.gz
-- Found the following matching hrefs:
     gbemol-0.3.tar.gz
     gbemol-0.3.1.tar.gz
     gbemol-0.3.2.tar.gz
Newest version on remote site is 0.3.2, local version is 0.3.2
 => Package is up to date
-- Scan finished
  ===== source issues =====
suspicious-source:
bashisms:
script ./ltmain.sh does not appear to have a #! interpreter line;
you may get strange results
possible bashism in ./ltmain.sh line 368 ($RANDOM):
        my_tmpdir="$ my_template -$ RANDOM-0 $$"
error: ./ltmain.sh: Unterminated quoted string found, EOF reached. Wanted: <">
pyflakes:
cppcheck:
[src/gbemol-utils.c:167]: (error) Memory leak: path
[src/libmpdclient.c:1654]: (error) Possible null pointer dereference: output - otherwise it is redundant to check if output is null at line 1663
[src/libmpdclient.c:1655]: (error) Possible null pointer dereference: output - otherwise it is redundant to check if output is null at line 1663
[src/libmpdclient.c:1656]: (error) Possible null pointer dereference: output - otherwise it is redundant to check if output is null at line 1663
[src/libmpdclient.c:1664]: (error) Possible null pointer dereference: output - otherwise it is redundant to check if output is null at line 1663
phplint:
desktop validate:
.//.pc/desktop_file.patch/gbemol.desktop: warning: key "Encoding" in group "Desktop Entry" is deprecated
.//.pc/desktop_file.patch/gbemol.desktop: error: (will be fatal in the future): value "gbemol.png" for key "Icon" in group "Desktop Entry" is an icon name with an extension, but there should be no extension as described in the Icon Theme Specification if the value is not an absolute path
.//.pc/desktop_file.patch/gbemol.desktop: warning: value "Application;AudioVideo;Player;" for key "Categories" in group "Desktop Entry" contains a deprecated value "Application"
mp3check:
mp3val:
fontlint:
ftvalid:
msgfmt:
POFileChecker:
POFileSpell:
pngcheck:
jpeginfo:
ogginfo:
oggz-validate:
xmllint:
lacheck:
jslint:
check for environ calls to home:
check for use of /tmp:
./intltool-extract.in:    $OUTFILE     = "./tmp/$OUTFILE.h"
check for use of system():
./aclocal.m4:    lt_ld_extra= awk '/^include /   system(sprintf("cd /etc; cat %s 2>/dev/null", \[$]2)); skip = 1;     if (!skip) print \[$]0; skip = 0;  ' < /etc/ld.so.conf   $SED -e 's/#.*//;/^[  ]*hwcap[    ]/d;s/[:,  ]/ /g;s/=[^=]*$//;s/=[^= ]* / /g;/^$/d'   tr '\n' ' ' 
./intltool-merge.in:    return 1 if ((-x $file) or (system("$file -l >$devnull") == 0));
./intltool-merge.in:    system ("$iconv -f $encoding -t UTF-8 <$devnull 2>$devnull");
./intltool-update.in:    return 1 if ((-x $file) or (system("$file --version >$devnull") == 0));
./intltool-update.in:          system ($EXTRACT, "--update", "--srcdir=$SRCDIR",
./intltool-update.in:          system ($EXTRACT, "--update", "--type=$gettext_type", 
./intltool-update.in:    system ("$MSGMERGE", "-o", $outfile, $infile, "$MODULE.pot");
./intltool-update.in:    system ("$MSGFMT", "-o", "$devnull", "--verbose", $output_file);
./intltool-update.in:   system ("$MSGFMT", "-o", "$devnull", "--verbose", "$SRCDIR/$lang.po");
./configure:    lt_ld_extra= awk '/^include /   system(sprintf("cd /etc; cat %s 2>/dev/null", \$2)); skip = 1;     if (!skip) print \$0; skip = 0;  ' < /etc/ld.so.conf   $SED -e 's/#.*//;/^[   ]*hwcap[    ]/d;s/[:,  ]/ /g;s/=[^=]*$//;s/=[^= ]* / /g;/^$/d'   tr '\n' ' ' 
./intltool-extract.in:        system ("mkdir -p $dirname");
./intltool-extract.in:        system("mkdir tmp/"); 
./debian/gbemol.1:the \fBDebian\fP system (but may be used by others).  Permission is 
./debian/manpage.sgml:      the &debian; system (but may be used by others).  Permission is
  ===== sbo-run-clang.log warnings =====
update-binfmts: warning: unable to close /proc/sys/fs/binfmt_misc/register: Invalid argument
update-binfmts: warning: unable to enable binary format python2.6
dpkg-source: warning: extracting unsigned source package (gbemol_0.3.2-2.dsc)
dpkg-buildpackage: warning: using a gain-root-command while being root
gbemol-preferences.c:483:13: warning: 'gbemol_preferences_load_spin' defined but not used [-Wunused-function]
gbemol-net.c:60:11: warning: variable 'res' set but not used [-Wunused-but-set-variable]
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libfontconfig.so.1 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libatk-1.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against librt.so.1 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libgio-2.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libcairo.so.2 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libpango-1.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libpangocairo-1.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libfreetype.so.6 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libpangoft2-1.0.so.0 (it uses none of the library's symbols).
dpkg-gencontrol: warning: File::FcntlLock not available; using flock which is not NFS-safe
     + add misc-depends to fix lintian warning.
  ===== sbo-run-gcc.log warnings =====
dpkg-source: warning: extracting unsigned source package (gbemol_0.3.2-2.dsc)
dpkg-buildpackage: warning: using a gain-root-command while being root
gbemol-preferences.c:483:13: warning: 'gbemol_preferences_load_spin' defined but not used [-Wunused-function]
gbemol-net.c:60:11: warning: variable 'res' set but not used [-Wunused-but-set-variable]
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libfontconfig.so.1 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libatk-1.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against librt.so.1 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libgio-2.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libcairo.so.2 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libpango-1.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libpangocairo-1.0.so.0 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libfreetype.so.6 (it uses none of the library's symbols).
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/gbemol/usr/bin/gbemol was not linked against libpangoft2-1.0.so.0 (it uses none of the library's symbols).
dpkg-gencontrol: warning: File::FcntlLock not available; using flock which is not NFS-safe
     + add misc-depends to fix lintian warning.
Not as interesting as it can get, but still pretty cool. Andrea, I swear I m not picking on you :) More to come later (hopefully) :)

10 July 2012

Gunnar Wolf: From DebCamp to DebConf through cheese, wine and an intro track

One week. One long week. One beautiful week. One of the two major weeks of the year has passed since my previous post. Surely, we are in the middle of the two Major Weeks of the year, in the yearly schedule I have upheld for almost(!) ten years: DebConf+DebCamp. Yesterday, DebConf officially started. For the first time ever, we had a DebConf track targetted at the local (for a wide definition of local: All of the Central American countries) communities, which I chaired. We had the following talk lineup during this track: I believe it was a great success, and I hope the talks are useful in the future. They will be put online soon thanks to the tireless work of our work team. Today we sadly lost the presence of our DPL due to very happy circumstances he will surely announce himself. But DebConf will continue nevertheless - And proof of that is our anual, great, fun and inviting Cheese and Wine Party! After a series of organizational hiccups I hope nobody notices (oops, was I supposed not to say this?), today we had a beautiful, fun and most successful cheese and wine party, as we have had year after year since 2005. As many other people, we did our humble contribution for this party to be the success it deserves. There is lots of great cheeses, great wines, and much other great stuff we have to thank to each of the individuals who made this C&W party the success it was. Yes, it might be among the least-academic parts of our conference, but at the same time, it's one of its most cherished -and successful- traditions. And above all else, we have to thank our Great Leader^W^WCheeseMaster (who we still need to convince to play by our Great Leader's mandates - And no, I don't mean Zack here!) Hugs and thanks to my good and dear friend Christian Perrier for giving form to one of DebConf's social traditions that makes it so unique, so different from every other academic or communitary conference I have ever been part of. We still have most of the week to go. And if you are not in Managua (and are not coming soon), you can follow our activities following our video streams. Remember, debian/rules, now more than ever! And even given the (perpetual) heat in Managua: Wheezy is frozen, whee! [ all photos here taken by regina ]
AttachmentSize
100_2110.JPG1.53 MB
100_2119.JPG1.49 MB
100_2121.JPG1.52 MB
100_2135.JPG1.49 MB
100_2144.JPG1.11 MB
100_2146.JPG1.53 MB
100_2145.JPG1.08 MB
100_2122.JPG1.51 MB

9 July 2012

Julien Danjou: Logitech K750 keyboard and Unifying Receiver Linux support

A year ago, I bought a Logitech Wireless Solar Keyboard K750. I'm particularly picky on keyboards, but this one is good. It has an incredible useful feature: while being wireless, it has no need for disposable or rechargeable batteries, it uses solar power! Logitech Wireless Solar Keyboard K750 My problem is that there's obviously no way to know the battery status from Linux, the provided application only working on Windows. And one dark night, while fragging on QuakeLive, my keyboard stopped working: it had no battery left. This activity being quite energy consuming, it emptied the whole battery. Someone should write code to get the battery status and light meter from Linux: challenge accepted! How the keyboard works Logitech Unifying Receiver This keyboard, like many of the new wireless devices from Logitech, uses the Unifying interface. It's an USB receiver that can be attached up to 6 differents devices (mouse, keyboards ). On old Linux kernel, the Unifying receiver is recognized as only one keyboard and/or one mouse device. Recently, a driver called hid-logitech-dj has been added to the Linux kernel. With this driver, each device attached to the receiver is recognized as one different device. What the Logitech application does Logitech Solar App The Logitech application under Windows works that way: you launch it, and it displays the battery charge level. On the keyboard, there's a special "light" button (up right). When pressed, a LED will light up on the keyboard: green if the keyboard is receiving enough light and is charging, red if the keyboard does not receive enough light and is therefore discharging. Pushing this same button while the application is running will makes the light meter activated: the application will tell you how much lux your keyboard is receiving. Let's reverse engineer this As far as I know, there's nothing in the USB HID protocol that handles this kind of functionality (battery status, light meter ) in a standard way. So the first task to accomplish is, unfortunately, to reverse engineer the program. I discovered a bit too late that Drew Fisher did a good presentation on USB reverse engineering at 28c3. You might want to take a look at it if you want to reverse engineer on USB. I did not need it, but I learned a few things. Anyway, my plan was the following: run the Logitech application inside a virtual machine running Windows, give it direct access to the USB keyboard, and sniff what happens on the USB wire. To achieve that, you need a virtual machine emulator that can do USB pass-through. Both KVM and VirtualBox can do that, but VirtualBox works much better with USB and allow hot(un)plugging of devices, so I used it. To sniff what happens on the USB, you need to load the usbmon Linux kernel module. Simply doing modprobe usbmon will work. You can then use Wireshark which know how to use usbmon devices and understand the USB protocol. USB stuff you need to know You don't need to know much about USB to understand what I'll write about below, but for the sake of comprehensibility I'll write a couple of things here before jumping in. To communicate with an USB device, we communicate with one of its endpoints. Endpoints are regrouped into an interface. Interfaces are regrouped into a configuration. A device might contains one or several configurations. There's also several types of packets in the USB wire protocol, and at least two of them interest us there, they are: All of this and more is well (and better) explained in the chapter 13 of Linux Device Drivers, Third Edition. Sniffed data Once everything was set-up, I ran my beloved Wireshark. There's a an URB of type interrupt sent each time you press any key with some data in it. Therefore I advise you to plug another keyboard (or use the laptop keyboard if you're doing this on a laptop), otherwise you'll get crazy trying to sniff the keyboard you're typing on. At this point, just launching the application does a bunch of USB traffic. Pressing the "light" button on the keyboard makes even more USB packets coming in and out. Here's the interesting packets that I noticed once I excluded the noise: With all this, the next step was clear: understand the packets and reproduce that exchange under Linux. What the packets mean The "go for the light meter" packet The packet sent from the computer to the keyboard is the following. <figure>
Frame 17: 71 bytes on wire (568 bits), 71 bytes captured (568 bits)
    Frame Length: 71 bytes (568 bits)
    Capture Length: 71 bytes (568 bits)
USB URB
    URB id: 0xffff880161997240
    URB type: URB_SUBMIT ('S')
    URB transfer type: URB_CONTROL (0x02)
    Endpoint: 0x00, Direction: OUT
        0... .... = Direction: OUT (0)
        .000 0000 = Endpoint value: 0
    Device: 6
    URB bus id: 1
    Device setup request: relevant (0)
    Data: present (0)
    URB sec: 1340124450
    URB usec: 495643
    URB status: Operation now in progress (-EINPROGRESS) (-115)
    URB length [bytes]: 7
    Data length [bytes]: 7
    [Response in: 18]
    [bInterfaceClass: HID (0x03)]
    URB setup
        bmRequestType: 0x21
            0... .... = Direction: Host-to-device
            .01. .... = Type: Class (0x01)
            ...0 0001 = Recipient: Interface (0x01)
    bRequest: SET_REPORT (0x09)
    wValue: 0x0210
        ReportID: 16
        ReportType: Output (2)
    wIndex: 2
    wLength: 7
0000  40 72 99 61 01 88 ff ff 53 02 00 06 01 00 00 00   @r.a....S.......
0010  22 ad e0 4f 00 00 00 00 1b 90 07 00 8d ff ff ff   "..O............
0020  07 00 00 00 07 00 00 00 21 09 10 02 02 00 07 00   ........!.......
0030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040  10 01 09 03 78 01 00                              ....x..
</figure> What's here interesting is the last part representing the data. wLength says that the length of the data is 7 bytes, so let's take a look at those 7 bytes: 10 01 09 03 78 01 00. Well, actually, you can't decode them like that, unless you're a freak or a Logitech engineer. And I have actually no idea what they mean. But sending this to the keyboard will trigger an interesting thing: the keyboard will start sending URB interrupt with some data without you pressing any more key. The "light meter and battery values" packet This is most interesting packet. This is the one sent by the keyboard to the host and that contains the data we want to retrieve. <figure>
Frame 1467: 84 bytes on wire (672 bits), 84 bytes captured (672 bits)
    Frame Length: 84 bytes (672 bits)
    Capture Length: 84 bytes (672 bits)
USB URB
    URB id: 0xffff88010c43c380
    URB type: URB_COMPLETE ('C')
    URB transfer type: URB_INTERRUPT (0x01)
    Endpoint: 0x83, Direction: IN
        1... .... = Direction: IN (1)
        .000 0011 = Endpoint value: 3
    Device: 2
    URB bus id: 6
    Device setup request: not relevant ('-')
    Data: present (0)
    URB sec: 1334953309
    URB usec: 728740
    URB status: Success (0)
    URB length [bytes]: 20
    Data length [bytes]: 20
    [Request in: 1466]
    [Time from request: 0.992374000 seconds]
    [bInterfaceClass: Unknown (0xffff)]
Leftover Capture Data: 1102091039000c061d474f4f4400000000000000
0000  80 c3 43 0c 01 88 ff ff 43 01 83 02 06 00 2d 00   ..C.....C.....-.
0010  5d c5 91 4f 00 00 00 00 a4 1e 0b 00 00 00 00 00   ]..O............
0020  14 00 00 00 14 00 00 00 00 00 00 00 00 00 00 00   ................
0030  02 00 00 00 00 00 00 00 00 02 00 00 00 00 00 00   ................
0040  11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00   ....9....GOOD...
0050  00 00 00 00                                       ....
</figure> This packets come in regularly (1 per second) on the wire for some time once you sent the "go for the light meter" packet. At one point they are emitted less often and do not contain the value for the light meter anymore, suggesting that the control packet sent earlier triggers the activation of the light meter for a defined period. Now you probably wonder where the data are in this. They're in the 20 bytes leftover in the capture data part, indicated by Wireshark, at the end of the packet: 11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00 00 00 00 00. Fortunately, it was easy to decode. Knowing we're looking for 2 values (battery charge and light meter), we just need to observe and compare the packet emitted on the wire with the values displayed by the Logitech Solar App. To achieve this, I looked both at the Logitech Solar App and Wireshark while bringing more and more light near the keyboard, increasing the lux value received by the meter on the Solar App, and saw that the fields represented in blue (see below) where changing in Wireshark. Since 2 bytes were changing, I guessed that it was coded on 16 bits, and therefore it was easy to correlate the value with the Solar App. <figure>
[ ....9....GOOD....... ]
11 02 09 10 39 00 0c 06 1d 47 4f 4f 44 00 00 00 00 00 00 00
4 bytes - 1 byte for battery charge - 2 bytes for light meter - 2 bytes - 4 bytes for GOOD - 7 bytes
</figure> In this example, the battery has a charge of 0x39 = 57 % and the light meter receives 0x0c = 12 lux of light. It's basically dark, and that makes sense: it was night and the light was off in my office, the only light being the one coming from my screen. I've no idea what the GOOD part of the packet is about, but it's present in every packet and it's actually very handy to recognize such a packet. Therefore I'm considering this as some sort of useful mark for now. For the other bytes, they were always the same (0x11 0x2 0x9 0x10 at the beginning, 7 times 0x00 at the end). The 2 bytes between the light meter and GOOD probably mean something, but I've no idea what for now. Building our solar app Now we've enough information to build our own very basic solar application. We know how to triggers the light meter, and we know how to decode the packets. We're going to write a small application using libusb. Here's a quick example. It's not perfect and does not check for error codes, be careful. <figure>
/* Written by Julien Danjou <julien> in 2012 */
#include <linux/hid.h>
#include <libusb.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(void)
 
    libusb_context *ctx;
    libusb_init(&ctx);
    libusb_set_debug(ctx, 3);
    /* Look at the keyboard based on vendor and device id */
    libusb_device_handle *device_handle = libusb_open_device_with_vid_pid(ctx, 0x046d, 0xc52b);
    fprintf(stderr, "Found keyboard 0x%p\n", device_handle);
    libusb_device *device = libusb_get_device(device_handle);
    struct libusb_device_descriptor desc;
    libusb_get_device_descriptor(device, &desc);
    for(uint8_t config_index = 0; config_index < desc.bNumConfigurations; config_index++)
     
        struct libusb_config_descriptor *config;
        libusb_get_config_descriptor(device, config_index, &config);
        /* We know we want interface 2 */
        int iface_index = 2;
        const struct libusb_interface *iface = &config->interface[iface_index];
        for (int altsetting_index = 0; altsetting_index < iface->num_altsetting; altsetting_index++)
         
            const struct libusb_interface_descriptor *iface_desc = &iface->altsetting[al>tsetting_index];
            if (iface_desc->bInterfaceClass == LIBUSB_CLASS_HID)
             
                libusb_detach_kernel_driver(device_handle, iface_index);
                libusb_claim_interface(device_handle, iface_index);
                unsigned char ret[65535];
                unsigned char payload[] = "\x10\x02\x09\x03\x78\x01\x00";
                if(libusb_control_transfer(device_handle,
                                           LIBUSB_REQUEST_TYPE_CLASS   LIBUSB_RECIPIENT_INTERFACE,
                                           HID_REQ_SET_REPORT,
                                           0x0210, iface_index, payload, sizeof(payload) - 1, 10000))
                 
                    int actual_length = 0;
                    while(actual_length != 20   strncmp((const char *) &ret[9], "GOOD", 4))
                        libusb_interrupt_transfer(device_handle,
                                                  iface_desc->endpoint[0].bEndpointAddress,
                                                  ret, sizeof(ret), &actual_length, 100000);
                    uint16_t lux = ret[5] << 8   ret[6];
                    fprintf(stderr, "Charge: %d %%\nLight: %d lux\n", ret[4], lux);
                 
                libusb_release_interface(device_handle, iface_index);
                libusb_attach_kernel_driver(device_handle, iface_index);
             
         
     
    libusb_close(device_handle);
    libusb_exit(ctx);
 
</figure> What the program is doing is the following: This gives the following:
Found keyboard 0x0x24ec8e0
Charge: 64 %
Light: 21 lux
Challenge accomplished! To be continued Unfortunately, this approach has at least one major drawback. We have to disconnect the Logitech Unifying Receiver from the kernel. That means that while we're waiting for the packet, we're dropping packets corresponding to other events from every connected device (key presses, pointer motions ). In order to solve that, I sent a request for help on the linux-input mailing list. That way, I learned that Logitech is actually using the HID++ protocol to communicate with the devices using the Unifying Receiver. Lars-Dominik Braun managed to get the HID++ specifications from Logitech and published them with their authorization. This opens a whole new world. With that document, I may be able to understand the part I reverse engineered and convert this to a more useful and generic library using the hidraw interface (so we don't have to disconnect the devices from the kernel driver).

John Goerzen: Decisions on Listening to Music

A couple of days ago, I commented about my thoughts on finding a better way to listen to music, and asked for suggestions. I ve checked out the avenues suggested, and here are my thoughts so far. Streaming service: Spotify Of the streaming service offerings, Spotify seems the most compelling. It has both Linux and Android clients. The Linux client is unofficial and a bit buggy, but serviceable. The service is fairly nice, and the serendipity of listening to radio of tracks like any track or artist I may pick is an incredible feature. It will mean a different mindset; rather than carefully choosing what I buy, I have a vast catalog at my fingertips. I d have to give up some control, but might be better off for it. Spotify can play local files, but sadly lacks support for FLAC that most of my collection is in. A little work with the shell, and I had a quick multi-core FLAC to MP3 conversion done. It can also identify local files and match them up with its catalog, but it does a somewhat poor job of it, despite the fact that my files are tagged by Musicbrainz. There are also a fair number of albums in my collection (from Magnatune and from local artists) that aren t in Spotify s catalog and probably never will be. I may wind up using git-annex or something like it to sync them between PCs. Surprisingly, it doesn t have any kind of web-based player available. It has a lot of features, but many of them are under-documented. Subsonic / Supersonic The idea of being able to stream music anywhere certainly has appeal, and the idea of streaming that music from my own machine my own collection is quite the appealing one. Subsonic seems to be the standard that people recommend for this. It has a web-based player that works well, as well as mobile clients for various platforms. The Android mobile client is particularly well-regarded and is probably the best one here, aside from Spotify. The playlists work well, as expected, and generally it has the appearance of a well-rounded system. There is a lot of flexibility in bitrates; maximum bitrates can be set per-user and per-client. It, of course, transcodes on the fly. My chief gripe about Subsonic is that it has no true album/artist/genre browser. Its main browser is mostly a filesystem browser. I have used clients that show album/artist/genre panes or sortable lists, based on tags, for so long that my filesystem organization is rather different centered around bitrate of rip, origin, etc. It is not very easy to navigate my collection in that way. The search feature can search artist and album, and this can partially make up for it. But there is simply no way to see a list of all artists in the collection. The server is written in Java, and takes up 362MB at startup. That s, I guess, decent for a Java server, but doesn t make me pleased as I d be running it on lower-end hardware. Supersonic is a git-maintained actively synced subsonic fork which removes the license cost (the source is GPL3 anyhow so that wasn t a big deal). Ampache Often mentioned in the same breath as Subsonic, this is yet another frequent suggestion for streaming your own collection. Its primary player is web-based, like Subsonic. It has full artist/genre browsers, and adds tag cloud and various other options as well by default, it loaded the tag cloud with my genre tags, which was a nice touch. The search feature is a little less robust, as it simply searches anywhere and returns a list of all matching tracks, rather than listing matching albums and artists before going into the list of all tracks. But that is somewhat of a minor nit. Ampache s web-based player, liks Subsonic s, uses Flash, but the implementation is really quite awkward. You can t simply click a play button and have playback start. Rather, you must add things to the playlist (queue), a frame on the web interface. Then, you must click play in THAT frame, which launches a new window for the player and transfers its contents to it. You thus have two separate running playlists. And if you think you can just add to the playlist of the player window, think again; you can only replace it. It is simply not possible to add a track to the end of that list without losing your existing place. This makes the usefulness of the web-based player rather, well, low. Playlist functionality in Ampache is rather odd. You can have playlists, but if you want to add a track to a saved playlist, first you have to clear you current playlist, then add the track to it, then save it to the existing playlist. Frontends for Ampache exist in banshee and in amarok. Banshee s is broken, and simply silently fails to do anything. Unless you look in the console, in which case you ll see a .NET backtrace. Banshee has had these mysterious errors for me for years. The Amarok plugin is so basic that you can browse by artist but not by album, and is almost unusable. There is also a rhythmbox plugin in Debian, but it is apparently for an old version of Rhythmbox and no longer works. The best frontend is a dedicated Ampache frontend called Viridian. It has a nice interface, provides features such as taskbar integration, and generally works pretty well. The only thing it s bad at is playlists, and here it s even worse than the web interface. It doesn t support modifying the server-side playlists at all. And accessing one is an annoying load playlist operation, that replaces your current play queue (also called the playlist ) with its contents by default. Several Android clients exist for Ampache, though it seems only one has seen any updates since 2010. mpd This is an interesting program, as it is designed more to provide multiple points of control for a single player than it is to stream a single collection to multiple devices. Nevertheless, I tested its HTTP and icecast stream modules. (It can also stream with PulseAudio, but PulseAudio causes devious breakage on my workstation, so I refuse to bother.) This could be made to work reasonably well on my LAN using the gmpc client. There are several Android clients as well, though none of them integrate as well as Subsonic or Spotify. Most seem focused on controlling a remote player. The ones that do stream do so with some hiccups, and don t integrate controls with the lock screen. gmpc is a reasonable control program and, combined with mplayer, makes a decent player. It can browse the database every way I d want to. It doesn t let me right-click on tracks to add them to playlists, but I can use copy/paste to do so, or add them to the play queue and from there to a playlist. It s annoying, but not completely unworkable. But here s the real concern. mpd has no provision whatsoever for any sort of encryption such as SSL. If I want to be listening to my music collection from on the road, I don t really wish to open up a streaming music service for the entire Internet. Nor do I really want to mess with SSH tunnels and the like, which are cumbersome on Android (and can be in general). This is pretty much a fatal flaw to me. Google Music An interesting service, but ultimately one I don t see the point for in my situation where I already have my music on a PC that is always on. It would probably accommodate my entire collection free, but how many weeks would it take me to get it there? And how would it stay in sync? It simply doesn t offer me any compelling advantage over streaming direct from my PC, so I don t see much need to bother with it. Others I briefly considered Streeme (poor mobile support), Audiogalaxy (no Linux support), Jinzora (appears dead), and Sockso (no mobile support). I also briefly considered other streaming services such as Pandora, Slacker, rdio, etc., but based on reviews didn t think that they were worth bothering with. Conclusions I m left mainly considering Spotify and Subsonic. For right now, I m going to try Spotify and see where that leads me, but may wind up getting back to Subsonic if I can t find tracks I want with Spotify.

8 July 2012

Dirk Eddelbuettel: New package RcppCNPy with release 0.1.0 (and 0.0.1 earlier last week)

A few days ago I had blogged about getting NumPy data in R by using a simple converter script. That works fine, but it is a little annoying to have to write an entire file only to read from it again. So I kept looking around for a better solution---and soon found the cnpy library by Carl Rogers which provides simple C++ functions to read and write NumPy files. Bringing such a C++ library to R is done very easily via Rcpp modules. The resulting package contains a single R file with a single line: loadModule("cnpy", TRUE). And it relies on the following module declarations in the a C++ file:
RCPP_MODULE(cnpy) 
    using namespace Rcpp;
    function("npyLoad",                         // name of the identifier at the R level
             &npyLoad,                          // function pointer to helper function defined above
             List::create( Named("filename"),   // function arguments including default value
                           Named("type") = "numeric"),
             "read an npy file into a numeric or integer vector or matrix");
    function("npySave",                         // name of the identifier at the R level
             &npySave,                          // function pointer to helper function defined above
             List::create( Named("filename"),   // function arguments including default value
                           Named("object"), 
                           Named("mode") = "w"),
             "save an R object (vector or matrix of type integer or numeric) to an npy file");
 
which give us at the R prompt
R> library(RcppCNPy)
Loading required package: Rcpp
R> npyLoad
internal C++ function <0x243af70>
    docstring : read an npy file into a numeric or integer vector or matrix
    signature : Rcpp::RObject npyLoad(std::string, std::string)
R> npySave
internal C++ function <0x23033e0>
    docstring : save an R object (vector or matrix of type integer or numeric) to an npy file
    signature : void npySave(std::string, Rcpp::RObject, std::string)
R> 
these two functions (and their docstrings) defined above. That's all! Well there are about one hundred more lines dealing with whether we have integer or numeric data, and whether we use a vector or a matrix. But all in all pretty simple... So version 0.1.0 of this new package RcppCNPy completes the initial release 0.0.1 from earlier in the week by adding The NEWS entry for this release (as well as the initial one) follow:
News for Package RcppCNPy Changes in version 0.1.0 (2012-07-07)
  • Added automatic use of transpose to automagically account for Fortran-vs-C major storage defaults between Python and R.
  • Support for integer types in dependent on the int64_t type which is available only when the -std=c++0x switch is used at build-time (and CRAN still discourages use of it)
  • Added support for reading gzip'ed files ending in ".npy.gz"
  • Added regression tests in directory tests/
  • Added a vignette describing the package
  • Added a timing benchmark in demo/timings.R
Changes in version 0.0.1 (2012-07-04)
  • Initial version, as a straightforward Rcpp modules wrap around the cpny library by Carl Rogers (on github under a MIT license).
  • At present, npy files can be read and written for vectors and matrices of either numeric or integer type. Note however that matrices are currently transposed because of the default Fortran ordering done by numpy.
I will follow up with a little usage example later. CRANberries also provides a diffstat report for 0.1.0 relative to 0.0.1. As always, feedback is welcome and the rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

7 July 2012

Keith Packard: FEC

Forward Error Correction with the TI CC1120 Sub-GHz transceiver We re building a new, fancier flight computer and decided to give the new, fancier TI CC1120 chip a try. It has significantly more transmit power (+16dBm) and receive sensitivity (-109dBm) than the transciever built in to the TI CC1111 RF SoC that we ve built our other boards with. Given that we d already decided to use a fancier STM32L processor, using the nicest stand-alone radio chip we could find seemed to make good sense. One of the nice features found in the CC1111 is support for FEC using a 1/2 rate constraint length 4 convolutional code. Both the encoder and the soft-decision Viterbi decoder are built right into the chip; all we had to do was flip a couple of bits and we got error correction, interleaving, data whitening and CRC checking all for free. We assumed that, as the CC1120 was advertised as a better replacement for our CC1111 radio, that it would also include all of these fine features. After we got the chip all soldered down to our first prototype boards, we read through the reference manual looking for the right register values to flip for forward error correction. We found support for CRC computation and data whitening, but no mention of interleaving or forward error correction at all. Thanks, TI. Of course, without the FEC pieces, the other packet handling hardware is worthless you can t exactly do a CRC on the encoded data and expect it to be useful on the other end of the link, and data whitening happens after the CRC. Basic CC1120 driver I assumed I d eventually figure out something to do about the FEC pieces and started writing a basic CC1120 driver AltOS, the operating system used in all of our projects. I started with the transmit side, as that seemed easier as I could use the existing packet hardware by uploading the desired bit sequence using whatever fancy encoding desired and the hardware would happily transmit it. I first tested that using low data-rate bits (2kbps), transmitting an alternating sequence of zeros and ones to generate a 1kHz tone on a regular 70cm receiver. That seemed to work just fine, demonstrating that I had some grasp of the basic operation of the chip. Convolutional Encoding in Software Not to be daunted by a small challenge in software, I set out to replicate the encoding scheme used in the CC1111 chips for use in our CC1120-based design. We obviously want our CC1120 boards to interoperate with the CC1111 boards. Fortunately, TI fully documents the CRC, data whitening, convolutional encoding and interleaving done in the hardware. I created my own implementation of the encoder (licensed, as usual, under the GPLv2) which you can see in the file ao_fec_tx.c It s not quite as flexible as the hardware, it always whitens the data and always appends the CRC bytes to the end of the packet, along with the necessary trellis termination bytes. The core of that code is surprisingly simple:
static const uint8_t ao_fec_encode_table[16] =  
/* next 0  1      state */
    0, 3,   /* 000 */
    1, 2,   /* 001 */
    3, 0,   /* 010 */
    2, 1,   /* 011 */
    3, 0,   /* 100 */
    2, 1,   /* 101 */
    0, 3,   /* 110 */
    1, 2    /* 111 */
 ;
uint8_t
ao_fec_encode(const uint8_t *in, uint8_t len, uint8_t *out)
 
    uint8_t     extra[AO_FEC_PREPARE_EXTRA];
    uint8_t     extra_len;
    uint32_t    encode, interleave;
    uint8_t     pair, byte, bit;
    uint16_t    fec = 0;
    const uint8_t   *whiten = ao_fec_whiten_table;
    extra_len = ao_fec_prepare(in, len, extra);
    for (pair = 0; pair < len + extra_len; pair += 2)  
        encode = 0;
        for (byte = 0; byte < 2; byte++)  
            if (pair + byte == len)
                in = extra;
            fec  = *in++ ^ *whiten++;
            for (bit = 0; bit < 8; bit++)  
                encode = encode << 2   ao_fec_encode_table[fec >> 7];
                fec = (fec << 1) & 0x7ff;
             
         
        interleave = 0;
        for (bit = 0; bit < 4 * 4; bit++)  
            uint8_t byte_shift = (bit & 0x3) << 3;
            uint8_t bit_shift = (bit & 0xc) >> 1;
            interleave = (interleave << 2)   ((encode >> (byte_shift + bit_shift)) & 0x3);
         
        *out++ = interleave >> 24;
        *out++ = interleave >> 16;
        *out++ = interleave >> 8;
        *out++ = interleave >> 0;
     
    return (len + extra_len) * 2;
 
ao_fec_encode_table takes care of the convolutional piece, ao_fec_whiten_table (not shown) contains the data whitening codes while the little loop at the bottom does the interleaving. I didn t spend a lot of time optimizing this as it seemed to run plenty fast on the 32MHz processor. With the bits all nicely encoded, I passed them to my simple CC1120 driver and let it send them along to our CC1111 receiver. Much to my delight, with only a few bugs fixed, it worked just fine! I d managed to get one direction working, validating the hardware and making it clear that we d at least be able to use the board in flight for telemetry. Getting Soft-decision data out of the CC1120 In packet mode, the CC1120 receiver converts the incoming GFSK stream directly into bits, making its best guess as to whether the detected value was a one or zero. Of course, like any FM modulation, the receiver will be producing intermediate values, especially when the signal is weak or masked by interference. A Viterbi decoder can make good use of this information;these noisy intermediate values are less influential in the final resulting path cost which allows the strong 0/1 values to pull the solution to the right one. Soft decision decoding yields a theoretical 2dB gain over hard decision decoders, and given our desire for maximum packet reception over long distances, it seemed important to make this work. The CC1120 has a way to recover the soft decision values, but only one 8-bit value at a time. It does, however, helpfully provide a GPIO wire which can be programmed to interrupt the CPU when the soft decision data has arrived. So, I hooked up a GPIO line on the processor to the GPIO line on the CC1120 and let it interrupt once per bit time, or 38400 times per second, to read this valuable soft decision data out of the radio part. According to TI support, this is the only way to make this work, and that they generally recommend that developers use the packet mode stuff and throw away this valueable source of data. I was happily surprised to learn that the STM32L can in fact survive this interrupt rate, and by setting interrupt priorities appropriately, I manage to reliably capture an entire packets worth of data without dropping a single bit. I set up a simple packet dumping function to run on the prototype board and captured raw encoded bytes from our CC1111-based flight computer so that I d have some real data to work from. Soft-decision Viterbi Decoding The sample encoding implementation from TI made that piece fairly straightforward. Lacking a similar sample for the decoding piece meant that I d be starting from scratch. And, implementing something that seemed like magic to me when I started. I started by reading a few articles on Viterbi decoding: I found Chip Fleming s tutorial very useful as it included a bunch of worked examples using a convolutional encoders with constraint length (k) of 3 instead of 4; this makes all of the trellis diagrams half the size. I also sent my friend, Phil Karn, a query about this process, and he responded with a bunch of helpful suggestions, while also recommending that I just steal his DSP and FEC library implementation. Of course, I don t like using code that I can t understand, and his code only implements much longer codes than I needed, so I d have to understand it anyways to make it work for me. Phil s code is a marvel, but it s not exactly comprehensible to someone who doesn t even know what a Viterbi decoder is supposed to do. So, with Chip s tutorial on my screen, I wrote a primitive Viterbi decoder. This did only the Viterbi step, and split that into two phases. The first computed the cost for each state in the system for every received pair of symbols received. It also tracked the chain of states through the entire decode process:
    for (state = 0; state < 8; state++)  
        struct ao_soft_sym zero = ao_soft_sym(ao_fec_encode_table[state * 2 + 0]);
        struct ao_soft_sym one = ao_soft_sym(ao_fec_encode_table[state * 2 + 1]);
        uint8_t zero_state = ao_next_state(state, 0);
        uint8_t one_state = ao_next_state(state, 1);
        int zero_cost = ao_cost(s, zero);
        int one_cost = ao_cost(s, one);
        zero_cost += cost[b][state];
        one_cost += cost[b][state];
        if (zero_cost < cost[b+1][zero_state])  
            prev[b+1][zero_state] = state;
            cost[b+1][zero_state] = zero_cost;
         
        if (one_cost < cost[b+1][one_state])  
            prev[b+1][one_state] = state;
            cost[b+1][one_state] = one_cost;
         
     
At the end of the packet, it found the least costly path and walked that from back to front, generating output bits (yes, one bit per element of the bits array):
for (state = 1; state < 8; state++)  
    if (cost[b][state] < c)  
        c = cost[b][state];
        min_state = state;
     
 
for (b = len/2; b > 0; b--)  
    bits[b-1] = min_state & 1;
    min_state = prev[b][min_state];
 
Completely separate from this code were the interleaving and data whitening stages. Keeping those separate allowed them to be tested independently, which made debugging them far easier. None of this code ever ran on the STM32L processor, it was instead run in a test framework on my laptop. I first got this working with data generated by the software convolutional encoder, then I managed to get it to decode the raw packets that I had recorded directly from the receiver it self. This marked a significant milestone in my mind I d managed to replace, in software, the CC1111 FEC encoder and decoder. However, the decoder performance was not fast enough to manage back-to-back packets yet. Having something in hand that worked meant that further development would be directly comparable to something that did work, making optimization much easier. Yet another lesson in making it work, and then making it work fast. Optimizing the Viterbi Decoder The first step was to reduce memory usage within the decoder. While it s best to defer computing the path until the very last bit has been received, it s also true that the chances of a bit affecting old data reduces as the distance between the new and old data increases. With the rule of thumb being a distance of k 6 or k 7, I saved 32 output bits for each state and wrote the oldest of 8 each time the buffer filled up, making a distance of 24 bits between the newest saved data and the current decoding position. That meant a fixed amount of storage was needed for an arbitrarily long packet. I tested a variety of different history sizes from 8 to 56 bits (cooresponding to 16, 32 and 64 bits of saved data for each state). With 100000 random packets generated that had gaussian noise added to each bit, here are the error rates for the different history lengths:
History LengthTotal PacketsCorrectIncorrect
8 bits10000090520 (90.52%)9480 (9.48%)
24 bits10000093614 (93.61%)6386 (6.39%)
56 bits10000093620 (93.62%)6380 (6.38%)
The performance of the 8-bit history and 16-bit history versions turned out to be essentially identical (given the 32-bit nature of the STM32L, that shouldn t be too surprising). The performance of the 56-bit version, which manipulated uint64_t data types was sigificantly slower, and given the very modest benefit, deemed not worth it. If we move to a larger constraint length (k > 4), or started using a punctured convolutional encoding, we would want to re-measure this. The next step was to combine the de-interleaving, decoding, de-whitening and CRC computation into a single loop. This would eliminate a pile of data copying and extra memory usage. Nothing fancy here, but it made enough of a difference that I could decode most of the incoming packets now, dropping only about 25% of them. To get to being able to decode every packet, I needed to start decoding the packet while it was being received. Because of interleaving, I had to have 32 bits of data to be able to decode anything, so it was a simple matter of having my per-soft-value interrupt handler start the decoder going after each interleave block of data was received. With this in place, I was able to decode a 76-byte payload (160 transmitted bits) in 14.7 ms, or just about 50% of the time it took to transmit the packet. Unrolling one of the inner decoding loops eliminated a bunch of computation and sped this up even more, decoding packets in 9.5ms. Success at last, a solid 100% of packets received were being decoded, as long as the CPU was otherwise idle. Finally, I went and re-read Phil Karn s code and pondered over how it works. It was opaque until I wrote a message back to Phil asking how it worked and pointing out sections that were confusing. Fortunately, simply phrasing my confusion in words made me understand how the code works. Here s my original code for computing the per-state cost metric:
/* Metric for a zero bit */
uint32_t    bitcost = ((uint32_t) (s0 ^ ao_fec_decode_table[(state<<1)]) +
               (uint32_t) (s1 ^ ao_fec_decode_table[(state<<1)+1]));
 
    /* Total path metric and state for a zero bit */
    uint32_t    cost0 = cost[p][state] + bitcost;
    uint8_t     state0 = ao_next_state(state, 0);
    /* Compare the total path metric against all existing
     * metrics heading into the same new state, choosing
     * the least expensive and killing the others.  Record
     * the new lowest cost and output bit.
     */
    if (cost0 < cost[n][state0])  
        cost[n][state0] = cost0;
        bits[n][state0] = (bits[p][state] << 1)   (state & 1);
     
 
 
    /* Total path metric and state for a one bit.  Because
     * encoding a one is always exactly the opposite of
     * encoding a zero from any state, this cost is
     * 255*2 - bitcost
     */
    uint32_t    cost1 = cost[p][state] + 510 - bitcost;
    uint8_t     state1 = ao_next_state(state, 1);
    if (cost1 < cost[n][state1])  
        cost[n][state1] = cost1;
        bits[n][state1] = (bits[p][state] << 1)   (state & 1);
     
 
Before this loop is run, I have to initialize cost[n] to a large value so that the comparisons will be true for the first case to reach each state. Phil s code, in contrast, doesn t have to initialize cost[n] as we know which two states can reach any possible new state. His code looks like this:
/* Metric for a zero bit */
bitcost = ((uint32_t) (s0 ^ ao_fec_decode_table[(state<<1)]) +
       (uint32_t) (s1 ^ ao_fec_decode_table[(state<<1) 1]));
/* Only state and state+4 reach state<<1 with a zero bit */
/* Cost from state to state<<1 for a zero bit*/
m0 = cost[p][state] + bitcost;
/* Cost from state+4 to state<<1 for a zero bit */
m1 = cost[p][state+4] + (510 - bitcost);
/* Which source state is cheaper? */
bit = m0 > m1;
/* Record the cheaper cost and the add to the path of bits */
cost[n][state<<1] = bit ? m1 : m0;
bits[n][state<<1] = (bits[p][state + (bit<<2)] << 1)   (state&1);
/* State and state+4 reach (state<<1)+1 with a one bit */
/* Cost from state to (state<<1)+1 for a one bit */
m0 -= (bitcost+bitcost-510);
/* Cost from state+4 to (state<<1)+1 for a one bit */
m1 += (bitcost+bitcost-510);
/* Which source state is cheaper? */
bit = m0 > m1;
/* Record cheaper cost and add to the path of bits */
cost[n][(state<<1)+1] = bit ? m1 : m0;
bits[n][(state<<1)+1] = (bits[p][state + (bit<<2)] << 1)   (state&1);
This code does two states at a time, and so while it s slightly longer than the above, it only needs to run half as many times. The resulting code now decodes packets in 5.7ms. The final version of the code can be see in ao_fec_rx.c The whole AltOS operating system, with many drivers and support for CC1111, AVR and STM32L processirs is available, under the GPLv2 from git://git.gag.com/scm/git/fw/altos

6 July 2012

Gregor Herrmann: DebCamp impressions

I'm here in managua since sunday evening, & by now DebCamp is slowly getting to its end. So here are some of my impressions so far: thanks to all organizers & sponsors who make this experience happen!

Christian Perrier: DebConf running: take 4 (Masaya volcano)

Yet another Grand Plan today: run in Masaya Volcan National Park. Masaya Volcano is one of the most active volcanoes in Nicaragua. It is located only 20km south-east of Managua, so going there is as easy as renting a taxi. The original plan was to drive to the entrance of the National park (on the road to Masaya city), then run up to the volcan, so from 250m altitude up to about 600m. I therefore rented a taxi with the kind help of Norman, and we agreed to meet up at 08:30 at the hotel, along with No l and Daniel (who was planning to walk up, not running). After discussing here or there, Kurt and Gaudenz joined, both with the intent of running up. Others would also have liked to join, but I wasn't in the mood of organizing a full bus of Debconfers..:-). My plan was to enjoy pyself the hike...not to manage several taxis, etc. We thus packed in Denis blue taxi...a fancy and very visible customized blue car that looked like it had been the topic of on episode of the "Pimp My Ride" TV show..:-) Sitting 4 people at the back of a regular car without hitting some blue neon lights, or 7" TV screens is kinda tricky, but we made it. When arriving at the park's entrance, we however learned that it is forbidden to walk or hike up the road by foot, because of potential high concentrations of Carbon Dioxide. Sadly, we then had to pack again in the car and go up with it. After reaching the parking close to the volcano's active crater (that is said to have a small lava lake at its bottom, one of the very few in the world), we decided to head up for going round the Nindira crater, an inactive crater located slightly above. Unfortunately, again, the path around the Masaya crater is forbidden to walk on, because of landslides. We indeed still had great fun by running around this crater (which is about a 2.5 kilometers round trip, very very hilly and sometimes hard to run on such as a mountain path. After about 50 minutes and two laps (after all, we had a giant stadium!), we headed back to the parking, packed 5 sweating geeks in the car and went back to the hotel. Full GPS trace is here. I also put a few pictures on Facebook, supposedly visible by anybody : Masaya crater and Nindiri crater.

5 July 2012

Steve McIntyre: DebCamp

Yay, got to Nicaragua OK last night. It's the first time I've managed to get to DebCamp in quite a while, and I'm making the most of it. I've got slides ready for two of my BoF sessions next week (both ARM-related), and I'm preparing for the other four now. Why did I sign up to run six sessions? Maybe because I'm a bit silly, or maybe because they needed doing and other people hadn't registered them. Maybe both. :-) BTW: these are all BoF (Birds of a Feather) sessions, not lectures. While I have an interest in each topic, I'm neither wanting nor expecting to just talk at a room full of people - I'm wanting to provoke discussions in each case. If you're interested, please join in those discussions, either physically in Managua or virtually using the live video streaming and IRC back-channels: See you there...!

4 July 2012

Michal &#268;iha&#345;: Weblate 1.1

Pretty much on schedule, Weblate 1.1 has been released today. It comes with translation updates, bug fixes, improvements in working with Git repositories and brings support for offloading indexing. Full list of changes for 1.1: You can find more information about Weblate on it's website, the code is hosted on Github. If you are curious how it looks, you can try it out on demo server. You can login there with demo account using demo password or register your own user. Ready to run appliances can be found in SUSE Studio Gallery. Weblate is also being used https://l10n.cihar.com/ as official translating service for phpMyAdmin, Gammu, Weblate itself and others. If you are free software project which would like to use Weblate, I'm happy to help you with set up or even host Weblate for you (this will be decided case by case as my hosting space is limited). Update: Weblate appliace has now also been updated to 1.1.

Filed under: English Phpmyadmin Suse Weblate 0 comments Flattr this!

Dirk Eddelbuettel: RcppBDT 0.2.0

A new release of the RcppBDT package appeared on CRAN earlier today. RcppBDT uses Rcpp, and in particular the nifty Rcpp modules feature of wrapping C++ code for R just by declaring the (class or function) interfaces. It uses this to bring in some useful functions from Boost Date.Time to R so that one can do things like
R> library(RcppBDT)
R> sapply(2012:2016, function(year)
+         format(getNthDayOfWeek(first, Mon, Sep, year)))
[1] "2012-09-03" "2013-09-02" "2014-09-01" "2015-09-07" "2016-09-05"
R> 
to compute the next five Labor Day dates in the US, given the year and the first Monday of September requirement. More examples are e.g. on the earlier blog post announcing version 0.1.0. Changes are mostly internal. R 2.15.1 brough a better / easier way to load such 'modules' into R, and Rcpp 0.9.13 allows us to use this. So RcppBDT continues to be useful as an example package for Rcpp modules. I also streamlines the interface a little: identifiers are now directly accessible in the package's NAMESPACE rather than just via the an instantiated object. I also added a NEWS file, using the .Rd format so that we can import the marked-up text:
News for Package RcppBDT Changes in version 0.2.0 (2012-07-02)
  • The core module, which wraps what in C++ is boost::gregorian::date, is now exposed as an Rcpp module bdtDate. As all example and demos operated off a (package-global) variable 'bdt', no user visible change was needed outside of the code instantiating at package load.
  • Updated package instantiation to the new facilities offered by the current versions R 2.15.1 and Rcpp 0.9.13 which make Rcpp module initialization easier and more streamlined.
Changes in version 0.1.0 (2011-01-17)
  • First CRAN upload (and see ChangeLog for more granular details) bug fix in svm cross validation
Courtesy of CRANberries, there is also a diffstat report for 0.2.0 relative to 0.1.0. As always, feedback is welcome and the rcpp-devel mailing list off the R-Forge page for Rcpp is the best place to start a discussion.

2 July 2012

Matthew Garrett: Fun with firmware ABI confusion

I spent a bunch of the past week trying to figure out why my UEFI bootloader code was crashing in bizarre ways. It turns out to be a known issue that various people have hit in the past. After a lot of staring (and swearing) I finally got to the point where I could run a debugger against a running UEFI binary under qemu and found that I was entering a function with valid arguments but executing it with invalid ones. After spending a while crying on IRC people suggested that maybe the red zone was getting corrupted, and it turned out that this was the problem.

The red zone is part of the AMD64 System V ABI and consists of 128 bytes at the bottom of the stack frame. This region is guaranteed to be left untouched by any interrupt or signal handlers, so it's available for functions to use as scratch space. For the code in question, gcc was copying the registers onto the stack before executing the function. Something then trod on those registers. This also explained why I wasn't seeing the problem when I added debug print statements - signal and interrupt handlers won't touch it, but other functions might, so it tends to only be used by functions that don't call any other functions. Adding a print statement meant that the compiler left the red zone alone.

But why was it getting corrupted? UEFI uses the Microsoft AMD64 ABI rather than the System V one, and the Microsoft ABI doesn't define a red zone. UEFI executables built on Linux tend to use System V because that means we can link in static libraries built for Linux, rather than needing an entirely separate toolchain and libraries to build UEFI executables. The problem arises when we run one of these executables and there's a UEFI interrupt handler still running. Take an interrupt, Microsoft ABI interrupt handler gets called and the red zone gets overwritten.

There's a simple fix - we can just tell gcc not to assume that the red zone is there by passing -mno-red-zone. I did that and suddenly everything was stable. Frustrating, but at least it works now.

comment count unavailable comments

Mehdi Dogguy: Adding useful links on bugreports pages

I'm a heavy user of Debian's Bug Tracking System (especially during this period of the year ;)) and I quite like it. And as a Debian developer, I have to check multiple sources when looking at some source package. Namely, I mainly check the following sources:
All this is great since you can move from the first one to the second one (and vice-versa) just by clicking with your mouse. It becomes less handy when you are looking at any bug report. In fact, the BTS doesn't link back to the previously mentioned services [1]. So, you have to copy/paste the name of the package or start a new tab and write down the address of the desired service. All this is not very nice for a lazy person like me :)

A few days ago, I remembered that I was already using GreaseMonkey and thought : "Hey, it would be nice to hack quickly something using GM to add the missing links". After a few tests, I've come up with a (very simple, yet extremely useful to me) user script that I called Debian BTS extra links [2]. It adds what I've been missing for years. If you are lazy and don't want to install right now, you can check the following pages to see what it adds:
The userscript will add links to PTS, Buildd status pages and Popcon near every pkgreport.cgi?src= link. Simple and useful. I hope you'll enjoy it as much as I do.

Of course, I've submitted a bug (#679282) against the pseudo package bugs.debian.org kindly asking its maintainer to add a link to (for now) the PTS. But I know that they are busy so I'm not expecting a result soon. Fow now, my simple and dumb userscript WFM.

[1] This is partly true since it links back to the PTS on main pages (pages that list all bug reports of a given set of packages).

[2] Only tested with a recent Iceweasel (from experimental) and Chromium (from unstable).

P.S.: If you do wonder, #4300 is the least bug number known to the BTS. Well, or at least, the result of running ls -1 /srv/bugs.debian.org/spool/archive/00/ sort -h head -n1 on bugs.debian.org.

Michal &#268;iha&#345;: Surprising uses of Weblate

When I wrote Weblate I expected that major use cases will be free software projects, which will use it for public translating service. At least this was my intention for it. But I seem to be wrong, at least for now. Most of people who contact me regarding Weblate are doing private installations and I already know few places where it was massively used. Unfortunately most of them rejected to be publicly mentioned, so I can't tell you the names. But they do exist :-). Anyway I hope for some free software projects to adopt Weblate as well, otherwise I should have written it as commercial software :-)

Filed under: English Weblate 6 comments Flattr this!

Rapha&#235;l Hertzog: My Debian Activities in June 2012

This is my monthly summary of my Debian related activities. If you re among the people who made a donation to support my work (168.12 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Dpkg This month, I resumed my work on dpkg. I concentrated my efforts on some polishing of the 3.0 (quilt) format. With the latest version (1.16.6 which was uploaded to unstable shortly before the freeze), dpkg-source restores the source tree in a clean state after a failed patch application (#652970), doesn t overwrite the patch header from the pre-existing automatic patch, updates automatically debian/source/include-binaries during dpkg-source commit, and supports a new no-unapply-patches option for those who dislike the auto-unapplication at the end of the process when the patches were not applied at the start. I wanted to go further and offer a new feature that could insert the automatic patch at the bottom of the quilt series but I have been short on time to complete this feature. I just managed to factorize all the quilt handling in a dedicated Perl module (Dpkg::Source::Quilt) to have cleaner code in the module handling the source format (Dpkg::Source::Package::V3::quilt). For those who wonder, this feature is meant primarily for the X Strike Force team which maintains packages in Git and are doings lots of upstream cherry-picks (to fix regressions, etc.). But they also use quilt on top of that tree to keep some lasting Debian specific changes. With the 1.0 format, the automatic diff is a bit messy but at least it gets smaller automatically when a new upstream release gets out, there s nothing to clean out. I d like them to be able to use 3.0 (quilt) while keeping their workflow. I m leaning towards allowing --auto-commit=first:cherry-picks that would name the automatic patch cherry-picks and put it in the first position in the quilt series. (Opinions welcome on that feature, BTW) Packaging There s been quite some packaging in this last month before the freeze: While doing all this packaging work, I found 2 possible improvements that I filed as bug reports: Debian France Booth at Solutions Linux From June 19th to June 21th, I manned the Debian France booth at Solutions Linux together with Carl Chenet, Tanguy Ortolo and other members of the association. We answered lots of questions, sold all t-shirts and umbrellas that Carl imported from Germany and Switzerland (we really need to get our own merchandising stuff produced in France!), got people to join the association. We also presented a printed copy of the Debian Administrator s Handbook and of the corresponding French book. You can see Carl, me and Tanguy on this picture (click on it to see a bigger picture, thanks to S bastien Dubois of Evolix for this one!):
I know lots of people are preparing for Debconf but I decided to not attend this year, the price of the air plane ticket was a bit too hefty for me and it was also in partial conflict with our family vacations. I thought about attending the Libre Software Meeting instead but alas I won t go there either (but Roland Mas will be there!), I have too much work to complete before my own vacation in 2 weeks. Thanks See you next month for a new summary of my activities.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

Joey Hess: notes for a caretaker

I recently had the sort of weird experience of a recent blog post being on the top of Hacker News and there being a fair amount of interest in the details of my wacky living situation. I could write a detailed post explaining everything, but that'd be boring .. instead I'll tease the stalkers with more oblique references to it. So here's my notes for a caretaker who will be here while I'm away at DebConf.
Please make yourself at home!

1 July 2012

Gunnar Wolf: Arrived to Nicaragua. DebCamp has officially started!

Yesterday night, Regina and me arrived to Nicaragua. Ready to greet us, we found quite a good number of good friends. We had a nice pizza+beer dinner at Diana's house, and some of the foreigners among us were distributed among the houses of several locals. This morning, we woke up together with V ctor, Moray and Gaudenz in Norman's brother's beautiful house. We had breakfast with the family, were picked up to go to the hotel that will have the ho(n rr)or to host us all for the following two weeks, and walked to the Universidad Centroamericana (UCA) campus. Contrary to our usual practices... It seems everything is working fine! I mean, I'm sure we will stumble with some unforseen details and what not... But coming on the very first day to the university, to find that food is all sorted out, that we have food tickets (and they are all printed!), that network works (and it's by a fiber connection that was laid out expressly for us), that we have all the hardware I was worried about, that people are arriving and getting accepted at the hotel. I mean, things work! So, I'm quite optimistic this DebCamp will have everything ready to be a success And the DebConf following it as well, of course! If you have not yet arrived - See you soon!

Next.

Previous.