Search Results: "spa"

7 February 2013

Matthew Garrett: The Samsung laptop issue is not fixed

The recent Linux kernel commits avoid one mechanism by which Samsung laptops can be bricked, but the information we now have indicates that there are other ways of triggering this. It also seems likely that it's possible for a userspace application to cause the same problem under Windows. We're still trying to figure out the full details, but until then you're safest ensuring that you're using BIOS mode on Samsung laptops no matter which operating system you're running.

comment count unavailable comments

Russell Coker: Links February 2013

Aaron on Software wrote an interesting series of blog posts about psychology and personal development collectively Titled Raw Nerve , here s a link to part 2 [1]. The best sections IMHO are 2, 3, and 7. The Atlantic has an insightful article by Thomas E. Ricks about the failures in leadership in the US military that made the problems in Afghanistan and Iraq a lot worse than they needed to be [2] Kent Larson gave an interesting TED talk about how to fit more people in cities [3]. He covers issues of power use, transport, space use, and sharing. I particularly liked the apartments that transform and the design for autonomous vehicles that make eye contact with pedestrians. Andrew McAfee gave an interesting TED talk titled Are Droids Taking Our Jobs [4]. I don t think he adequately supported his conclusion that computers and robots are making things better for everyone (he also presented evidence that things are getting worse for many people), but it was an interesting talk anyway. I Psychopath is an interesting documentary about Sam Vaknin who is the world s most famous narcissist [5]. The entire documentary is available from Youtube and it s really worth watching. The movie Toy Story has been recreated in live action by a couple of teenagers [6]. That s a huge amount of work. Rory Stewart gave an interesting TED talk about how to rebuild democracy [7]. I think that his arguments against using the consequences to argue for democracy and freedom (he suggests not using the torture doesn t work and women s equality doubles the workforce arguments) are weak, but he made interesting points all through his talk. Ernesto Sirolli gave an interesting TED talk about aid work and development work which had a theme of Want to help someone? Shut up and listen! [8]. That made me think of Mary Gardiner s much quoted line from the comments section of her Wikimania talk which was also shut up and listen . Waterloo Labs has some really good engineering Youtube videos [9]. The real life Mario Kart game has just gone viral but there are lots of other good things like the iPhone controlled car and eye controlled Mario Brothers. Robin Chase of Zipcar gave an interesting TED talk about various car sharing systems (Zipcar among others), congestion taxes, the environmental damage that s caused by cars, mesh networks, and other things [10]. She has a vision of a future where most cars are shared and act as nodes in a giant mesh network. Madeleine Albright gave an interesting TED talk about being a female diplomat [11]. She s an amazing speaker. Ron Englash gave an interesting TED talk about the traditional African use of fractals [12]. Among the many interesting anecdotes concerning his research in Africa he was initiated as a priest after explaining Georg Cantor s set theories. Racialicious has an insightful article about the low expectations that members of marginalised groups have of members of the privileged groups [13]. Rick Falkvinge has a radical proposal for reforming copyrights with a declared value system [14]. I don t think that this will ever get legislative support, but if it did I think it would work well for books and songs. I think that some thought should be given to how this would work for Blogs and other sources of periodical content. Obviously filing for every blog post would be an unreasonable burden. Maybe aggregating a year of posts into one copyright assignment block would work. Scott Fraser gave an interesting TED talk about the problem with eyewitness testimony [15]. He gave a real-world example of what had to be done to get an innocent man acquitted, it s quite amazing. Sarah Kendzior wrote an interesting article for al Jazeera about the common practice in American universities to pay Adjunct Professors wages that are below the poverty line [16]. That s just crazy, when students pay record tuition fees there s more than enough money to pay academics decent wages, where does all the money go to anyway?
Related posts:
  1. Links January 2013 AreWomenHuman has an interesting article about ViolentAcrez and the wide...
  2. Links February 2012 Sociological Images has an interesting article about the attempts to...
  3. Links February 2009 Michael Anissimov writes about the theft of computers from the...

Ian Wienand: Shared libraries and execute permissions

In the few discussions you can find on the web about shared-libraries and execute-permissions, you can find a range of various opinions but not a lot about what goes when you execute a library. The first thing to consider is how the execute-permission interacts with the dynamic loader. When mapping a library, the dynamic-loader doesn't care about file-permissions; it cares about mapping specific internal parts of the .so. Specifically, it wants to map the PT_LOAD segments as-per the permissions specified by the program-header. A random example: 
 $ readelf --segments /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0
Elf file type is DYN (Shared object file)
Entry point 0x77c00
There are 7 program headers, starting at offset 64
Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x00000000001b5ea4 0x00000000001b5ea4  R E    200000
  LOAD           0x00000000001b6b60 0x00000000003b6b60 0x00000000003b6b60
                 0x0000000000028dc4 0x000000000002c998  RW     200000
The permissions to load the code and data segments are given the Flags output. Code has execute-permissions, and data has write-permissions. These flags are mapped into flags for the mmap call, which the loader then uses to map the various segments of the file into memory. So, do you actually need execute-permissions on the underlying file to mmap that segment as executable? No, because you can read it. If I can read it, then I can copy the segment to another area of memory I have already mapped with PROT_EXEC and execute it there anyway. Googling suggests that some systems do require execute-permissions on a file if you want to directly mmap pages from it with PROT_EXEC (and if you dig into the kernel source, there's an ancient system call uselib that looks like it comes from a.out days, given it talks about loading libraries at fixed addresses, that also wants this). This doesn't sound like a terrible hardening step; I wouldn't be surprised if some hardening patches require it. Maximum compatability and historical-features such as a.out also probably explains why gcc creates shared libraries with execute permissions by default. Thus, should you feel like it, you can run a shared-library. Something trivial will suffice: 
int function(void)  
      return 100;
 

$ gcc -fPIC -shared -o libfoo.so foo.c
$ ./libfoo.so
Segmentation fault
This is a little more interesting (to me anyway) to dig into. At a first pass, why does this even vaguely work? That's easy -- an ELF file is an ELF file, and the kernel is happy to map those PT_LOAD segments in and jump to the entry point for you: 
$ readelf --segments ./libfoo.so
Elf file type is DYN (Shared object file)
Entry point 0x570
There are 6 program headers, starting at offset 64
Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x000000000000070c 0x000000000000070c  R E    200000
  LOAD           0x0000000000000710 0x0000000000200710 0x0000000000200710
                 0x0000000000000238 0x0000000000000240  RW     200000
What's interesting here is that the shared-library has an entry point (e_entry) at all. ELF defines it as:
This member gives the virtual address to which the system first transfers control, thus starting the process. If the file has no associated entry point, this member holds zero.
First things first, where did that entry point come from? The ld manual tells us that the linker will set the entry point based upon the following hierarchy:
  • the -e entry command-line option;
  • the ENTRY(symbol) command in a linker script;
  • the value of a target specific symbol
  • the address of the first byte of the .text section, if present;
  • The address 0.
We know we're not specifying an entry point. The ENTRY command is interesting; we can check our default-link script to see if that is specified: 
$ ld --verbose   grep ENTRY
ENTRY(_start)
Interesting. Obviously we didn't specify a _start; so do we have one? A bit of digging leads to the crt files, for C Run-Time. These are little object files automatically linked in by gcc that actually do the support-work to get a program to the point that main is ready to run. So, if we go and check out the crt files, one can find a definition of _start in crt1.o 
$ nm /usr/lib/x86_64-linux-gnu/crt1.o
crt1.o:
0000000000000000 R _IO_stdin_used
0000000000000000 D __data_start
                 U __libc_csu_fini
                 U __libc_csu_init
                 U __libc_start_main
0000000000000000 T _start
0000000000000000 W data_start
                 U main
But do we have that for our little shared-library? We can get a feel for what gcc is linking in by examining the output of -dumpspecs. Remembering gcc is mostly just a driver that calls out to other things, a specs file is what gcc uses to determine which arguments pass around to various stages of a compile: 
$ gcc -dumpspecs
...
*startfile:
% !shared: % pg p profile:gcrt1.o%s;pie:Scrt1.o%s;:crt1.o%s 
 crti.o%s % static:crtbeginT.o%s;shared pie:crtbeginS.o%s;:crtbegin.o%s
The format isn't really important here (of course you can read about it); but the gist is that various flags, such as -static or -pie get passed different run-time initailisation helpers to link-in. But we can see that if we're creating a shared library we won't be getting crt1.o. We can double-confirm this by checking the output of gcc -v (cut down for clarity). 
$ gcc -v -fPIC -shared -o libfoo.so foo.c
Using built-in specs.
 ...
/usr/lib/gcc/x86_64-linux-gnu/4.4.5/collect2 -shared -o libfoo.so
/usr/lib/gcc/x86_64-linux-gnu/4.4.5/../../../../lib/crti.o
/usr/lib/gcc/x86_64-linux-gnu/4.4.5/crtbeginS.o
/tmp/ccRpsQU3.o
/usr/lib/gcc/x86_64-linux-gnu/4.4.5/crtendS.o
/usr/lib/gcc/x86_64-linux-gnu/4.4.5/../../../../lib/crtn.o
So this takes us further down ld's entry-point logic to pointing to the first bytes of .text, which is where the entry-point comes from. So that solves the riddle of the entry point. There's one more weird thing you notice when you run the library, which is the faulting address in kern.log: 
libfoo.so[8682]: segfault at 1 ip 0000000000000001 sp 00007fffcd63ec48 error 14 in libfoo.so[7f54c51fa000+1000]
The first thing is decoding error; 14 doesn't seem to have any relation to anything. Of course everyone has the Intel 64 Architecture Manual (or mm/fault.c that also mentions the flags) to decode this into 1110 which means "no page found for a user-mode write access with reserved-bits found to be set" (there's another post in all that someday!). So why did we segfault at 0x1, which is an odd address to turn up? Let's disassemble what actually happens when this starts. 
00000000000004a0 <call_gmon_start>:
 4a0:   48 83 ec 08             sub    $0x8,%rsp
 4a4:   48 8b 05 2d 03 20 00    mov    0x20032d(%rip),%rax        # 2007d8 <_DYNAMIC+0x190>
 4ab:   48 85 c0                test   %rax,%rax
 4ae:   74 02                   je     4b2 <call_gmon_start+0x12>
 4b0:   ff d0                   callq  *%rax
 4b2:   48 83 c4 08             add    $0x8,%rsp
 4b6:   c3                      retq
We're moving something in rax and testing it; if true we call that value, otherwise skip and retq. In this case, objdump is getting a bit confused telling us that 2007d8 is related to _DYNAMIC; in fact we can check the relocations to see it's really the value of __gmon_start__: 
$ readelf --relocs ./libfoo.so
Relocation section '.rela.dyn' at offset 0x3f0 contains 4 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000200810  000000000008 R_X86_64_RELATIVE                    0000000000200810
0000002007d8  000200000006 R_X86_64_GLOB_DAT 0000000000000000 __gmon_start__ + 0
0000002007e0  000300000006 R_X86_64_GLOB_DAT 0000000000000000 _Jv_RegisterClasses + 0
0000002007e8  000400000006 R_X86_64_GLOB_DAT 0000000000000000 __cxa_finalize + 0
Relocation section '.rela.plt' at offset 0x450 contains 1 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000200808  000400000007 R_X86_64_JUMP_SLO 0000000000000000 __cxa_finalize + 0
Thus call_gmon_start, rather unsurprisingly, checks the value of __gmon_start__ and calls it if it is set. Presumably this is set as part of profiling and called during library initialisation -- but it is clearly not an initialiser by itself. The retq ends up popping a value off the stack and jumping to it, which in this case just happens to be 0x1 -- which we can confirm with gdb by putting a breakpoint on the first text address and examining the stack-pointer: 
(gdb) x/2g $rsp
0x7fffffffe7d8:        0x0000000000000000      0x0000000000000001
So that gives us our ultimate failure. Of course, if you're clever, you can get around this and initalise yourself correctly and actually make your shared-library do something when executed. The canonical example of this is libc.so itself: 
$ /lib/x86_64-linux-gnu/libc-2.13.so
GNU C Library (Debian EGLIBC 2.13-37) stable release version 2.13, by Roland McGrath et al.
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
...
You can trace through how this actually does work in the same way as we traced through why the trivial example doesn't work. If you wondering my opinion on executable-bits for shared-libraries; I would not give them execute permissions. I can't see it does anything but open the door to confusion. However, understanding exactly why the library segfaults the way it does actually ends up being a fun little tour around various parts of the toolchain!

6 February 2013

Aigars Mahinovs: Pensijas prognoze 18.43

Update: I apologise for spamming all the nice people on Planet Debian with this unrelated post in foreign language it got mis-tagged and (although I tried classifying it correctly and removing it from the separate Debian-only RSS feed going to the Debian Planet) Planet software just keeps it around until admins get a spare moment to remove it manually. The basic idea of the post is that there is a e-government service by the Latvian government that provides any citizen a way to check what their pension would be, but it is kinda useless, because it calculates what you pension would be if you decided to retire right now. So in the post below I provide a few simple steps on how to estimate what the real pension could be (in todays currency values) if a person kept paying into the system at the same rate until the normal retirement age. Latvija.lv ir r ks savas n kotnes pensijas prognoz anai, ta u tas ir, maigi izsakoties, bezj dz gs tiem, kas nepl no iet pensij tuv k gada laik . tri apskatoties atteic gos likumus un noteikumus es nonacu pie lietder g kas metodes. Vispirms m s varam dro i ignor t pensiju indeks ciju un visas ar to saist t s pensiju kapit la p rr in anas vai iemaksu palielin anos ir skaidrs, ka gadiem ejot uz priek u b s kaut k ds infl cijas procents, kura d palielin sies algas, palielin sies iemaksas pensiju kapit l un tiks p rr in tas vis das izmaksas, ta u t pati infl cija tie i ietekm s ar to cik v rt gs b s katrs gala pensijas lats taj br d , kad pensija tiks sa emta, l dz ar to infl cijas efekts tiek neitraliz ts, ja m s v rt jam gala ciparu br a cenu kontekst t.i. ja gal san ks, ka pensija b s 400Ls, tad to vai tas ir daudz vai maz j dom br a cen s, nevis m inot izdom t k das b cenas p c 30 gadiem. To visu emot v r pensijas apr ins stipri vienk r ojas:
  1. Pa emam atskaiti no t Latvija.lv pakalpojuma. Tas mums dos uz o br di uzkr to pensijas kapit lu (U) un aptuvenu ciparu par to k ds ir kapit la ikgad jais piaugums (P)
  2. Apr inam cik v l gadi ir paliku i l dz pension an s vecumam (t) t.i. l dz 62 gadiem. Es pat r in tos ar to, ka ap to laiku m r a vecums jau b s 65.
  3. Sar inam to cik b s aptuvenais pensijas kapit ls pension anas vecuma, ja m s turpinam iemaksas k tagad K = U + ( P * t )
  4. Apr inam k da san k ikm ne a pensija no da pensijas kapit la Pensija = K / G / 12, kur G ir emts no pension an s vecumam atbilsto s rindi as MK noteikumos, piem ram 62 gadiem tas ir 18.43
Piez me: G ir statisti u apr in ta prognoze par to cik vid ji gadus Latvijas iedz vot ji nodz vo ja vi i ir sasniegu i attiec go pension an s vecumu, t.i. t ir prognoze par to cik gadus vid ji jums b s tas prieks baud t pensiju. Ja rezult ts san k par k mazs, tad j em v r , ka summa neiek auj dividendes no 2 un 3 pensijas l me a, kas p ris gadu desmitu laik var b t t ri iespaid gs. Da as prognozes ir t das, ka 2ais un 3ais pensijas l menis kop var tu dod v l aptuveni t du pa u pensiju kl t k 1ais l menis. Ta u tas ir stipri atkar gs no fondu tirgus ilglaic gas dinamikas. Man to visu apr inot san k ap 400 Ls pensijas prognoze. Ja piepild s paredz jumi par 2o un 3o pensijas l meni, tad, ja es iedom jos sevi tagad k pension ru ar 600-800 Ls pensiju zin nav ne vainas. :)

Biella Coleman: Edward Tufte was a phreak

It has been so very long since I have left a trace here. I guess moving to two new countries (Canada and Quebec), starting a new job, working on Anonymous, and finishing my first book was a bit much. I miss this space, not so much because what I write here is any good. But it a handy way for me to keep track of time and what I do and even think. My life feels like a blur at times and hopefully here I can see its rhythms and changes a little more clearly if I occasionally jot things down here. So I thought it would nice to start with something that I found surprising: famed information designer, Edward Tufte, a professor emeritus at Yale was a phone phreak (and there is a stellar new book on the topic by former phreak Phil Lapsley. He spoke about his technological exploration during a sad event, a memorial service in NYC which I attended for the hacker and activist Aaron Swartz. I had my wonderful RA transcribe the speech, so here it is [we may not have the right spelling for some of the individuals so please let us know of any mistakes]:
Edward Tufte s Speech From Aaron Swartz s Memorial
Speech starts 41:00 [video cuts out in beginning]
We would then meet over the years for a long talk every now and then, and my responsibility was to provide him with a reading list, a reading list for life and then about two years ago Quinn had Aaron come to Connecticut and he told me about the four and a half million downloads of scholarly articles and my first question is, Why isn t MIT celebrating this? .
[Video cuts out again]
Obviously helpful in my career there, he then became president of the Mellon foundation, he then retired from the Mellon foundation, but he was asked by the Mellon foundation to handle the problem of JSTOR and Aaron. So I wrote Bill Bullen(sp?) an email about it, I said first that Aaron was a treasure and then I told a personal story about how I had done some illegal hacking and been caught at it and what happened. In 1962, my housemate and I invented the first blue box, that s a device that allows for free, undetectable, unbillable long distance telephone calls. And we got this up and played around with it and the end of our research came when we concluded what was the longest long distance call ever made, which was from Palo Alto to New York time-of-day via Hawaii, well during our experimentation, AT&T, on the second day it turned out, had tapped our phone and uh but it wasn t until about 6 months later when I got a call from the gentleman, AJ Dodge, senior security person at AT&T and I said, I know what you re calling about. and so we met and he said You what you are doing is a crime that would , you know all that. But I knew it wasn t serious because he actually cared about the kind of engineering stuff and complained that the tone signals we were generating were not the standard because they record them and play them back in the network to see what numbers they we were that you were trying to reach, but they couldn t break though the noise of our signal. The upshot of it was that uh oh and he asked why we went off the air after about 3 months, because this was to make long distance telephone calls for free and I said this was because we regarded it as an engineering problem and we made the longest long distance call and so that was it. So the deal was, as I explained in my email to Bill Bullen, that we wouldn t try to sell this and we were told, I was told that crime significance would pay a great deal for this, we wouldn t do any more of it and that we would turn our equipment over to AT&T, and so they got a complete vacuum tube isolator kit for making long distance phone calls. But I was grateful for AJ Dodge and I must say, AT&T that they decided not to wreck my life. And so I told Bill Bullen that he had a great opportunity here, to not wreck somebody s life, course he thankfully did the right thing.
Aaron s unique quality was that he was marvelously and vigorously different. There is a scarcity of that. Perhaps we can be all a little more different too.
Thank you very much.

Michal &#268;iha&#345;: phpMyAdmin translations status

Next round of phpMyAdmin 4.0 translation status report is coming. So let's look at which translations are at 100% right now (no change this week): There are also several languages which need just few strings to be complete: As you can see, there is still lot of languages missing, this might be your opportunity to contribute to phpMyAdmin. Also you are welcome to translate phpMyAdmin 4.0 using translation server. If your language is already fully translated and you want to help as well, you can translate our documentation as well.

Filed under: English Phpmyadmin 0 comments Flattr this!

Joachim Breitner: Evaluation-State Assertions in Haskell

I have just uploaded a new version of ghc-heap-view to Hackage that provides Evaluation state assertions in the module GHC.AssertNF. Imagine you are writing a web application in Haskell that sports a global number-of-visitors counter in an IORef Int. For every request, you call modifyIORef (+1). Eventually, you notice your very popular web site to hog more and more memory. So you browse to the internal page that shows the counter, and you have to wait for a long time until you eventually see the result (or get a stack overflow). The reason: The applications of (+1) were not performed until you looked at the number; instead, a long chain of such computation first filled your heap and then your stack. So you have learned the hard way that you might want to avoid space leaks, and want calculations to be done during the request that caused them, and want the IORef to always contain fully evaluated data. So you stumble about modifyIORef' in Data.IORef and indeed, this fixes your problem. Later, you notice that you want to count POST and GET requests separately. You change the type to IORef (Int, Int) and call modifyIORef' (first (+1)) or modifyIORef' (second (+1)). And suddenly, the space leak is back (which you only notice after the next push to the real site, because your local tests never caused enough requests to make it noticeable). So you not only want to fix it, you also want to ensure that it does not break again. In other words, you want to ensure the policy that values stored in an IORef are always in normal form. You achieve this with the following alternative to modifyIORef': 
modifyIORef'Assert :: IORef a -> (a -> a) -> IO ()
modifyIORef'Assert ref f = do
    x <- readIORef ref
    let x' = f x
    x'  seq  return ()
    assertNF x'
    writeIORef ref x'
Using this instead of modifyIORef' will print this warning to standard error output right the first time you call modifyIORef'Assert (first (+1)): 
Parameter not in normal form: 2 thunks found:
let x1 = (S# 1,S# 1)
in _bh (_thunk x1 (_bco (S# 1)),_sel x1)
(Otherwise, the program runs as usual.) So obviously, you need to use a strict variant of first (or strict pairs): 
first' :: (a -> b) -> (a, c) -> (b, c)
first' f (x,y) = let   x' = f x; r = (x', y)   in x'  seq  r  seq  r
With this, the warning goes away. Whenever you now change the type of the IORef or modify it in a too-lazy-way, you can be sure that you ll be warned about it, before the space leak itself becomes noticeable. In the production code, you might want to disable the check. For that, simply put disableAssertNF somewhere in your main function. Why is this better than just calling deepseq in modifyIORef'Assert? Because this way, the code still creates unwanted thunks that are then evaluated before storing them in the IORef, whereas with assertNF you are told about the thunks and can prevent them from being created in the first place. Also, assertNF does not add a type class constraint. This is just one example application for assertNF (and its variants assertNFNamed, which includes a name in the warning to better spot the cause, and $assertNFHere, which uses Template Haskell to include the current source code position in the warning), and I hope that there are more. If you happen to make use of it, I d like to hear your story.

5 February 2013

Dirk Eddelbuettel: New Rcpp page on upcoming events -- including Master Class in New York

Lots of exciting things are happening with and around Rcpp. I just added a new page about Upcoming Events to the recently-created Rcpp site. This events page has lots to cover: an upcoming talk at Columbia on March 8 (details still TBD), a day-long workshop in New York on March 9, a possible participation at a CERN / ROOT conference in Switzerland on May 11-14, an upcoming talk in May in Milwaukee, and last but not least the tutorial by Romain and Hadley at UseR! 2013 in Spain. Phew! With that, a few quick words about the upcoming master class in New York. It will be a full day, covering an introduction and motivation, details about the core data types, tools for working with and and extending Rcpp and of course applications galore, including RcppArmadillo and RInside. I have done the same one day class format a few times before, most recently (with Revolution Analytics) in San Francisco in late 2011, and also as a two-part seminar at UseR! 2012. This time, we plan on providing cloud-hosted RStudio instances for participants. Better still, RStudio's own JJ Allaire will be on deck as well for RStudio --- and Rcpp Attributes --- questions. Details and registration information for the New York class are at this page.

Wouter Verhelst: Transcoding videos

Since some people asked for it, and since it's (unfortunately) fairly nontrivial, here is the script that I'm using currently to transcode a .dv file into webm: 
#!/bin/bash
set -e
newfile=$(basename $1 .dv).webm
wavfile=$(basename $1 .dv).wav
normalfile=$(basename $1 .dv)-normal.wav
normalfile=$(readlink -f $normalfile)
oldfile=$(readlink -f $1)
echo "Audio split"
gst-launch-0.10 uridecodebin uri=file://$oldfile ! progressreport ! audioconvert ! audiorate ! wavenc ! filesink location=$wavfile
echo "Audio normalize"
sox --norm $wavfile $normalfile
echo "Pass 1"
gst-launch-0.10 webmmux name=mux ! fakesink \
  uridecodebin uri=file://$oldfile name=demux \
  demux. ! ffmpegcolorspace ! deinterlace ! vp8enc multipass-cache-file=/tmp/vp8-multipass multipass-mode=1 threads=2 ! queue ! mux.video_0 \
  demux. ! progressreport ! audioconvert ! audiorate ! vorbisenc ! queue ! mux.audio_0
echo "Pass 2"
gst-launch-0.10 webmmux name=mux ! filesink location=$newfile \
  uridecodebin uri=file://$oldfile name=video \
  uridecodebin uri=file://$normalfile name=audio \
  video. ! ffmpegcolorspace ! deinterlace ! vp8enc multipass-cache-file=/tmp/vp8-multipass multipass-mode=2 threads=2 ! queue ! mux.video_0 \
  audio. ! progressreport ! audioconvert ! audiorate ! vorbisenc ! queue ! mux.audio_0
rm $wavfile $normalfile
It's fairly non-optimal, because many of these command-line A/V tools are either fairly badly documented, or have a horrific interface, or something similar. avconv is supposed to be able to do audio normalisation, but I haven't been able to figure out exactly how it's done. The options that are specified in the manpage seemingly have no effect. gstreamer has a 'ReplayGain' plugin which can do audio normalisation. Audio normalisation requires two passes; I could in theory just add the element in the gstreamer first-pass pipeline, add a '-t' to the gst-launch-0.10 invocation, and parse out the required gain value that I can then add to the second-pass pipeline. However, the elements that I can then pass that gain value to either have a different range for the gain (rgvolume's fallback-gain parameter) or expect a completely different unit of values with no obvious way to translate between the two (the 'volume' element's 'volume' parameter expects a multiplier, the replaygain plugin produces a dB value; a simple conversion seems wrong as it produces a value way out of range). The 'volume' element reportedly has an interface that takes a dB value, except you can't reach it from gst-launch-0.10. Bummer. So instead we split out the audio, do normalisation in sox, and mux that back in during the second pass of the video transcoding. Sox is good. Sox is easy. If all A/V command-line tools were like sox, I would be smiling now. What's missing from the above script is how to throw away the start or end of a file in case there's several minutes of uninterestingness there. This is easiest (in my experience) with avconv's -t and -ss options. I suspect gstreamer is capable of doing that, too; but I haven't figured out how, and this works. You may want to play with some of options of the vp8enc element. For instance, the threads= value can be increased (or decreased) depending on how many cores your transcoding machine has. You could probably also add a "target-bitrate" value if you find the quality is too low or the system uses too much diskspace. VP8 has several more options; the documentation lists them all. If you've done video recordings for a devroom at FOSDEM outside of the FOSDEM video team, note that FOSDEM is more than willing to host the videos. To do so, the easiest way is if the videos are online somewhere (this does not have to be a public system) that we can wget or scp them from; if you've done that, contact me with the location to the files and we'll put them on the FOSDEM video server.

Wouter Verhelst: Video at FOSDEM 2013

For the second year in a row, I've led the team that organized video at FOSDEM. This year, most things went pretty smoothly; more so than last year. We definitely learned from some of our mistakes, and although I can't say that everything went better than it did last year, after having done a first look at things I can certainly say that overall, the result is better now than it was last year. Our setup was very similar to what it was last year. However, there were some key differences: Unfortunately, that didn't mean all was well. The internal server would transcode the DV stream into WebM, which was then sent on to the Flumotion platform where it was going to be transcoded to Ogg Theora, before being streamed in both formats. Due to an issue which we could not track down (possibly something incorrect in my configuration), the transcoding into Ogg Theora did not work smoothly, and even had the unwanted effect of disrupting the WebM streaming. Since this made it difficult to debug without disrupting service, eventually we decided to give up on Ogg Theora and only provide WebM streams; I considered that to be a better solution than no streaming at all. In addition, apparently in the year since FOSDEM 2012, libav and gstreamer have changed significantly enough that the scripts that I used then had to be somewhat revisited in order to be able to transcode the existing recordings into WebM ready for video.fosdem.org. Oh well, I had to add in normalisation anyway; only it would've been nice if I didn't have to revisit it from scratch. Meh. At any rate, our streams seem to have been very popular. Some statistics:
Saturday

Sunday

Served traffic by country (over the whole weekend)
Served sessions by country (over the whole weekend)
As you can see from these graphs, watching the streams from Belgium seems to be happening fairly often; I suspect much of that is people at the conference watching a stream in a different room. Additionally, we peaked at 212 viewers, which is slightly more than the peak of 206 that we had last year. The final conclusion we can take home from these statistics is that the most popular talk on Saturday (at least according to people watching the streams) was the eudev one in the cross-distro room (with 68 viewers); on sunday, the talks in Janson were all fairly equally popular. The peak of 102 viewers was reached around 15:45. Overall, I'm quite happy with the result. Yesterday, Thomas Eug ne of NamurLUG (who used to do video for several years before I took over last year) helped me do the first review. What's left now is to do some transcoding and uploading; this will happen over the next few days and weeks.

Joey Hess: LCA2013 wrapup

After 27 hours of travel, I'm finally back from my first Linux.conf.au. This was a great conference! Some of my favorite highlights:
My git-annex talk went ok, despite technical problems with the video output, which the video team did a great job of working around. The first demo went well, and the Q&A was excellent. The demo of the git-annex assistant webapp was marred by, apparently, a bad build of the webapp -- which I ironically used rather than my usual development build because I assumed it'd be less likely to fail. At least I may have a way to reproduce that hang somewhat reliably now, so I can get on with debugging it. I will be redoing a demo of the webapp as a screencast.
Here are some of the best talks I attended. (I have quite a lot more queued up to watch still, and had to cut several due to space.) Click titles for videos, or browse all the videos.

4 February 2013

Rapha&#235;l Hertzog: My Free Software Activities in January 2013

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (84.25 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Debian Packaging In one of my customer projects, I had to use libwebsockets and since it was not packaged for Debian, I filed a Request For Package (RFP #697671). I discovered a fork of this library on github and decided to mail the original author and the author of the fork to learn a bit more about the reason of the fork. It turns out that they miscommunicated and that the original author was interested by most of the improvements. The fork still exists but the important fixes and most of the improvements have been merged (and he released a version 1.0 after that!). Furthermore the original author setup a bug tracker to better organize the project and so that the author of the fork can submit patches and be sure that they won t be forgotten (as it happened in the past). I spend quite some time discussing with both parties but at the end I m pleased to see that good progress has been made (although nobody stepped up to maintain this package in Debian). I packaged zim 0.59 (an important bugfix release) and wordpress 3.5.1 (with several security fixes). I updated the dpkg-dev squeeze backports to version 1.16.9~bpo60+1 on request of Daniel Schepler. This backport led me to file #698133 on kgb-client because the bot literally spammed the #debian-dpkg IRC channel for multiple hours by resending old commit notices that got merged in the squeeze-backports branch. BTW, they need help to get this issue fixed. I updated python-django-registration to fix a compatibility issue with python3-sphinx (see #697721 for details). Misc Debian Stuff Serious bug with salt. I filed a grave bug on salt (#697747 ) and prepared the upload to fix the issue on request of the maintainer. In the mean time, the maintainer orphaned the package. Franklin G. Mendoza already announced its willingness to take over but this package deserves multiple maintainers since this is a good piece of software that is getting more and more popular. net-retriever and alternate keyrings. I filed a wishlist bug (#698618) on net-retriever to request a way for derivatives to use another keyring package (i.e. not debian-archive-keyring-udeb) without having to fork net-retriever. Linux 3.7 on armel/armhf. I helped the kernel maintainers to fix the 3.7 kernel on armel/armhf by reporting on IRC the results of successive failing kernel rebuilds on those architectures (this kernel version is only in experimental). Carl9170 firmware. I also pinged the kernels maintainers about a missing firmware for the carl9170 driver (already reported in #635840) and Ben Hutchings took care of re-activating its inclusion in upstream s linux-firmware.git and then uploaded firmware-free 3.2 to Debian. Thanks Ben! New QA team member. And to finish with the miscellaneous stuff, I helped Holger Levsen to be added to the qa group so that he could integrate his awesome work on automated QA checks with Jenkins. Debian France Preparation for Solutions Linux. The people organizing the village of associations in the Solutions Linux conference have asked all organizations to apply for a booth if they wanted one. Last year Carl Chenet took care of organizing this and this time we had to find someone else. I made multiple call for volunteers (on the mailing list, on my blog) without much success but I finally managed to convince Tanguy Ortolo to take care of this. Thank you Tanguy! Get in touch with treasurer who disappeared. During the transition with the former Debian France officers, it has been said that Aur lien G r me another former treasurer of Debian France had entirely disappeared together with some papers that he never gave to his successor. I didn t want to give up on this without at least trying to get in touch by myself so after multiple tries (over IRC, phone, and snail mail), and some weeks without answers, he got back to me, explaining that he s currently in a foreign country and that he will take care of that next time that he comes in France. \o/ New website in preparation. Replacing the single-page website webpage with a more comprehensive website is an important goal. Alexandre Delano provided a basic ikiwiki setup inspired by dsa.debian.org. I cleaned it and integrated it in a git repository on our machine. There s thus a new test website on http://france.debian.net/test/. Tanguy Ortolo and Fernando Lagrange immeditaly made some small improvements but since then nobody stepped up to further complete the website. I ll try to do this in February and put the new website in production. Paypal and handling of members. We installed a paypal plugin in galette so that members can renew their membership online. I asked Christian Bayle to try it out and we found some issues that I reported upstream and that got fixed. But this is only the first step, we want to go much further and automate all the membership handling, from membership renewal mail reminders up to integration in the accounting system. To this end, I filed some new tickets in the Galette tracker and completed some that were already opened: #490, #368 and #394. We requested a quote for those tickets and Debian France is going to fund the work on those tickets so that we have a 100% free software solution for our needs. Thanks See you next month for a new summary of my activities.

No comment Liked this article? Click here. My blog is Flattr-enabled.

Antoine Beaupr : Live radio streaming with MPD, part 2: multicast RTP

The previous article was introducing basic streaming principles based on Icecast. The issue with this, of course, is lag and overhead of HTTP-based connexions. In this article we introduce RTP-based streaming system, (unfortunately) based on Pulseaudio and multicast. (Update: there was a rant here about Pulseaudio (PA) that I have moved out of this post because it's not what I was aiming to talk about. Those wanting to answer that troll are welcome to join the flamewar here.)

A word on multicast I took the opportunity of trying out Multicast (it's actually IP multicast, to be more precise), since I wanted to have only minimal configuration to perform on the client side. Plus, since the RTP frames are raw 44khz 16 bit audio, the traffic is actually quite heavy, 1.4mbps, to be more precise (punch in 44kHz * 16bit/Hz * 2 in the excellent Qalculate to see why), so I didn't want bandwidth to explode as more listeners joined in. As a comparison, also note that the Icecast Vorbis stream is around 64 or 128kbps depending on quality. Multicast allows packets to be sent only once for multiple listeners on the same network. It's actually quite cool, but doesn't work very well over the wider internet, for reasons that are still unclear to me. I presume that the fact there is virtual monopoly over traditional broadcasting systems (e.g. television and radio) means that multicast hasn't really kicked in yet... I hope this will make sense over the Montreal mesh, although the quality would probably need to be dialed down, or the audio streamed encoded instead of be sent as raw samples.

So why PA? VLC/pipe failures That aside, it seems that to stream through RTP, I have no choice. I have tried to use VLC as a pipe to stream data out of MPD as RTP, but somehow this config snippet just failed: 
audio_output  
   type            "pipe"
   name            "RTP"
   command         "cvlc -q - --sout '#rtp dst=239.72.72.144,port=5004 '"
   encoder         "vorbis"                # optional, vorbis or lame
   format          "44100:16:2"
I don't know why - the multicast port wouldn't open properly and MPD would even hang when restarting. It seems the pipe plugin is somewhat broken. Note that the vlc command above actually works fine to stream audio content out through multicast, which is a good way to test if multicast works in the first place...

Shoving PA down MPD's throat Instead, I had to go to the clunky PA-way (also know as "my way or the highway"). The first step was to configure the default.pa to enable RTP streaming. I copied over the default /etc/pulse/default.pa file into ~mpd/.pulse/default.pa and uncommented the following lines: 
# just to get the PA announced
load-module module-zeroconf-publish
# we need a null sink, no clue why, but without this glue, it fails
load-module module-null-sink sink_name=rtp format=s16be channels=2 rate=44100 sink_properties="device.description='RTP Multicast Sink'"
# port is specified otherwise it is random (?!)
# loop=1 is to broadcast locally too
# other useful setting: destination (defaults to 224.0.0.56)
load-module module-rtp-send source=rtp.monitor port=5004 loop=1
See the rtp-send module documentation for more information here. Then we need MPD to connect to that sink: 
audio_output  
    type            "pulse"
    name            "Pulsaudio RTP"
    sink            "rtp"
Also, now that we got into the PA kitchen sink, we are kind of forced to use it for the local sound card output too, so we'll add this as a sink to: 
audio_output  
    type            "pulse"
    name            "Pulseaudio sound card"
    sink            "alsa_output.pci-0000_00_1b.0.analog-stereo"
(Take a moment to admire the nice sink name. Yours may be different, have fun finding it in the output of pacmd list-sinks, but that works only once PA is started!) So now you need to restart MPD for those changes to take effect. Note that if the mpd user has already started PA, you're in luck: it won't restart it by itself, and you need to: 
service mpd stop
killall pulseaudio
service mpd start
... yep. Basically because pulseaudio just detaches completely from the parent process when spawned automatically. Lots of fun.

Testing and listening Of course, you can listen to the stream using... Pulseaudio. I assume this would magically show up on the network thanks to avahi, if you run Pulseaudio on (say) your laptop. Since I still want to stay away from it, I would rather use a straight commandline tool to listen to the stream. According to this page, there aren't that many clients that could listen to such a funky stream. I chose VLC, since its commandline options actually make sense. Here's the magic mantra to repeat every night before you go to bed: 
cvlc rtp://@224.0.0.56:5004
Yes, that's all. Drop the c if you want a GUI. With mplayer, it's more complicated - you need to specify a bitmask using a hexadecimal string (fun stuff): 
mplayer -demuxer rawaudio -rawaudio format=0x20776172 rtp://224.0.0.56:5004
Yes, good old 0x20776172...

Results! We want results! All this work reduced the lag from 15 seconds to around 1 second. It's still present, and confusing to listen to, but maybe more acceptable. It seems most of the lag is in the vlc client buffering, when I enable debugging, I see: 
[0x963c7bc] main input debug: Stream buffering done (1001 ms in 976 ms)
I was able to reduce that delay significantly by using the --rtp-caching option of vlc, as such: 
cvlc --rtp-caching=48  rtp://@224.0.0.56:5004
The value is in miliseconds. Anything below 48ms introduces audible artifacts (probably buffer underruns). With that setting, there's still a noticeable delay, especially with isolated sounds that are naturally more noticeable. With more harmonious music, it sounds more like an echo. I have also tried to fiddle with similar settings with mplayer and simply failed to get any results. --no-cache simply drops samples like crazy. --cache 64 is the lowest I could push it down to, and it still has noticeable delay, which I couldn't measure, but I would assume would be around 400ms at least. Also note that contrarily to the previous setup I had, based on Icecast, VLC will automatically resume streaming from the RTP stream as it is stopped or started, which is also very nice - as it allows me to disable remote RTP streams easily and harmlessly.

Troubled areas Speaking of insane nonsense, here are a few pitfalls to avoid. Take care to disable the ALSA output, if it's not yet commented out, because MPD will freak out with the following error message: 
Feb 03 20:25 : output: "Sound card" [alsa] failed to play: Resource temporarily unavailable
I have not been able to find a way to make PA let go of the audio sound card. I would have prefered MPD to open ALSA directly if PA wasn't needed (I need it only for RTP streaming), but it seems that PA assumes you'll want to play stuff locally. I have tried commenting out the following in default.pa: 
load-module module-udev-detect
Seems like it's not enough. Also note that PA is designed with "dumb users" in mind (read: you need a GUI to operate it), so in the context of MPD, it's really annoying to debug things unless you can actually fire up a X11 session as the MPD user (which strikes me as a crazy idea) to configure PA properly with paprefs and all the graphical gizmos. I just gave up and went with PA all the way, and just added the RTP sink through the commandlines. Other guides can show you screenshots to clikety your way through inane graphical interfaces to do the above, if you are into that kind of stuff. With PA, you will want to get familiar with the pacmd and pactl commands. Why there are two different commands to basically do the same thing is completely beyond me.

Reference and more reading I got a lot of information in the Pulseaudio MPD wiki page and the Pulseaudio user documentation. The VLC manual about RTP streaming (server) (client) was also useful for debugging. The Pulseaudio Network documentation has a set of recipes for using pulseaudio for everything. This Ask Ubuntu question explains how to clikety your way through paprefs to essentially do the above.

Antoine Beaupr : Live radio streaming with MPD, part 1: basic setup

I have been using MPD for a good while as my media player, in conjunction with gmpc as a graphical interface (since MPD is just a server). It's really cute, works reasonably well, and is one of the only software that can handle my huge music collection gracefully. The cool thing with the setup is that, because MPD is a network-transparent server, I can control (and listen to!) my music remotely.

Basic HTTP / Icecast streaming For a while, I have also been streaming my music through Icecast so that I can basically listen to my music from anywhere there is an internet connection with sufficient bandwidth. This configuration is simple enough: 
audio_output  
    type            "shout"
    name            "Currently listening to..."
    host            "localhost"
    port            "8000"
    mount           "/radio.ogg"
    password        "hackme"
    quality         "4.0"
    format          "44100:16:2"
    description     "My playlist of the day. High quality (4) OGG/Vorbis stream."
This will make a stream available at http://radio.example.com:8000/radio.ogg, provided Icecast is configured properly. That setup is a bit more complicated than necessary, because it needs an icecast server to be installed (which is just apt-get install icecast on Debian, but still). So an alternative, simpler setup, but which may not scale as well to many listeners, is: 
audio_output  
   type            "httpd"
   name            "My HTTP Stream"
   encoder         "vorbis"                # optional, vorbis or lame
   port            "8000"
   quality         "5.0"                   # do not define if bitrate is defined
   bitrate         "128"                   # do not define if quality is defined
   format          "44100:16:1"
I haven't played around too much with that plugin, so I don't know how well it works - I am using the Icecast plugin instead.

Always on radio Another cool thing is that I have actually two MPD daemons running at the same time - one is my regular player, but the other is the "shuffle" player. The basic idea is that when the regular player stops, the shuffle starts, and vice versa, so there is always something playing on the stream. The shuffle, by default, doesn't play on the local sound card so that even though the radio stream is up, it doesn't blast random music through my living room speakers all day (and night!). This trick is accomplished through a hook in the Icecast daemon: 
<mount>
    <mount-name>/radio.ogg</mount-name>
    <fallback-mount>/random.ogg</fallback-mount>
    <!-- when I listen to my radio, I turn off the random MPD stream to save CPU -->
    <on-connect>/home/anarcat/bin/mpd-stop-random</on-connect>
    <!-- ... and when I stop listening, the other stream just kicks in -->
    <on-disconnect>/home/anarcat/bin/mpd-start-random</on-disconnect>
    <!-- i put those here to try to display something on the stats page when there's no client, but it's not working -->
    <stream-name>Currently listening to...</stream-name>
    <stream-description>My playlist of the day. High quality (5) OGG/Vorbis stream. If I'm not listening to anything, you'll be redirected to the random playlist.</stream-description>
</mount>
<!-- final fallback: the random radio, which is started when the above playlist disconnects -->
<mount>
    <mount-name>/random.ogg</mount-name>
    <stream-name>Random!</stream-name>
    <stream-description>Random songs from a 33 days playlist</stream-description>
</mount>
The mpd-start-random script is rather idiotic: 
#!/bin/sh
export MPD_HOST=hackme@radio.anarcat.ath.cx
export MPD_PORT=6601
mpc play
You can guess what the mpd-stop-random script does (hint: the opposite of play is pause). So that's the old way of doing things. The one problem I found with this setup is when I want to setup another listener in the same house - the problem is that the Icecast stream has significant delay: a good 14 seconds here, with ogg123 as a client. Here's the special configuration for the mpd-slave daemon that runs the random playlist: 
# same as main player
music_directory         "/var/lib/mpd/music"
playlist_directory      "/var/lib/mpd/playlists"
# those need to be different
db_file                 "/var/lib/mpd-slave/tag_cache"
log_file                "/var/log/mpd/mpd-slave.log"
pid_file                "/var/run/mpd/slave.pid"
state_file         "/var/lib/mpd-slave/state"
# avoids having to update the database by hand all the time
auto_update    "yes"
# disabled by hand after daemon is started    
audio_output  
    type            "alsa"
    name            "Sound card"
 
# main icecast stream
audio_output  
    type            "shout"
    name            "Random!"
    host            "localhost"
    port            "8000"
    mount           "/random.ogg"
    password        "hackme"
    quality         "4.0"
    format          "44100:16:2"
    description     "Random songs from a 33 days playlist"
 
# another player for low bandwidth situations, disabled
audio_output  
    type            "shout"
    name            "Random! (lowfi)"
    host            "localhost"
    port            "8000"
    mount           "/random.ogg"
    password        "hackme"
    quality         "0.0"
    format          "44100:16:2"
    description     "Random songs from a 33 days playlist"
 
filesystem_charset "UTF-8"
user               "mpd"
default_permissions     "read"
port    "6601"
mixer_type                      "software"
volume_normalization            "yes"
The random playlist is done by simply adding the whole library to the playlist. MPD chokes a little the first time you do that, but it's surprisingly fast. I also went through the playlist to remove spoken words and other stuff I don't want to stumble upon, and after I shuffled the playlist.

Final notes Otherwise, this is a fairly standard MPD setup, although there are some special configuration that vary from the upstream default: 
# this needs to be commented out for the server to be accessible from everywhere
#bind_to_address                "localhost"
# setup a password for remote access
password                "hackme@read,add,control"
# make sure the mixer controls don't affect the local hardware mixer
mixer_type                      "software"
# we are beyond the ASCII world, but MPD doesn't know
filesystem_charset              "UTF-8"
id3v1_encoding                  "UTF-8"
This is the first part of a series of articles I intend to write about my MPD configuration, next ones detail the configuration of a dumb player and RTP streaming.

3 February 2013

Dirk Eddelbuettel: The Rcpp Gallery and my Seinfeld Streak

A good three weeks ago, we introduced the Rcpp Gallery. While this is a joint effort by several of us on the Rcpp team, the backend was conceived and implemented entirely by JJ who also bootstrapped it with same first content, drawing on posts by Hadley, Romain and myself. As the How to contribute page makes plain, this is all backed by GitHub and all logs are public anyway. So after it was up and working, JJ and I refined the look and feel, and I started to add more content so that would have something by the time the initial announcement came around. A few years I read about an (attributed) secret to Seinfeld's producitivity: "Don't break the chain". Just keep writing, and write every day. I made my goal of a post every day for just over a month, and created this sequences: <quote> (20 Dec) simulating-pi, (21 Dec) vector-minimum, (22 Dec) gsl-colnorm-example, (23 Dec) fibonacci-sequence, (24 Dec) random-number-generation, (25 Dec) armadillo-sparse-matrix, (26 Dec) timing-rngs, (27 Dec) stl-inner-product, (28 Dec) stl-transform, (29 Dec) stl-transform-for-subsetting, (30 Dec) stl-random-shuffle, (31 Dec) stl-random-sample, (01 Jan) stl-for-each, (02 Jan) armadillo-subsetting, (03 Jan) accessing-environments, (04 Jan) armadillo-eigenvalues, (05 Jan) r-function-from-c++, (06 Jan) using-the-rcpp-timer, (07 Jan) sugar-function-clamp, (08 Jan) using-rcout, (09 Jan) first-steps-with-C++11, (10 Jan) simple-lambda-func-c++11, (11 Jan) eigen-eigenvalues, (12 Jan) getting-attributes-for-xts-example, (13 Jan) intro-to-exceptions, (14 Jan) a-first-boost-example, (15 Jan) a-second-boost-example, (16 Jan) timing-normal-rngs, (17 Jan) creating-xts-from-c++, (18 Jan) gsl-for-eigenvalues, (19 Jan) accessing-xts-api, (20 Jan) custom-as-and-wrap-example, (21 Jan) passing-cpp-function-pointers, </quote> The Rcpp Gallery continues to grow, we now have 58 posts from 7 different authors. And it is open for business: new contributions are always welcome.

2 February 2013

Petter Reinholdtsen: Bitcoin GUI now available from Debian/unstable (and Ubuntu/raring)

My last bitcoin related blog post mentioned that the new bitcoin package for Debian was waiting in NEW. It was accepted by the Debian ftp-masters 2013-01-19, and have been available in unstable since then. It was automatically copied to Ubuntu, and is available in their Raring version too. But there is a strange problem with the build that block this new version from being available on the i386 and kfreebsd-i386 architectures. For some strange reason, the autobuilders in Debian for these architectures fail to run the test suite on these architectures (BTS #672524). We are so far unable to reproduce it when building it manually, and no-one have been able to propose a fix. If you got an idea what is failing, please let us know via the BTS. One feature that is annoying me with of the bitcoin client, because I often run low on disk space, is the fact that the client will exit if it run short on space (BTS #696715). So make sure you have enough disk space when you run it. :) As usual, if you use bitcoin and want to show your support of my activities, please send Bitcoin donations to my address 15oWEoG9dUPovwmUL9KWAnYRtNJEkP1u1b.

1 February 2013

Daniel Kahn Gillmor: proprietary software activation fail

i have a colleague who is forced by work situations to use Windows. Somehow, I'm the idiot^W^W^W^W^Wfriendly guy who gets tapped to fix it when things break. Well, this time, the power supply broke. As in, dead, no lights, no fan, no nothing. No problem, though, the disk is still good, and i've got a spare machine lying around; and the spare is actually superior hardware to the old machine so it'll be an upgrade in addition to a fix. Nice! So i transplant the disk and fire up the new chassis. But WinXP fails to boot with a lovely "0x0000007b" BSOD. The internet tells me that this might mean it can't find its own disk. OK, pop into the new chassis' BIOS, tell it to run the SATA ports in "legacy IDE" mode, and try again. Now we get a "0x0000007e" BSOD. Some digging on the 'net makes me think it's now complaining now about the graphics driver. Hmm. Well, i figure i can probably work around that by installing new drivers from Safe Mode. So i reboot into Safe Mode. Success! It boots to the login screen in Safe Mode. And, mirabile dictu, i happen to know the Administrator password. I put it in, and get a message that this Windows installation isn't "activated" yet -- presumably because the hardware has changed out from under it. And by the way, i'm not allowed to log in via safe mode until it's activated. So please reboot to "normal" Windows and activate it first. Except, of course, the whole reason i'm booting into safe mode was because normal Windows gives a BSOD. Grrrr. Who thought up this particular lovely catch-22? OK, change tactics. Scavenging the scrap bin turns up a machine with a failed mainboard, but a power supply with all the right leads. It's rated for about 80W less than the old machine's failed supply, but i figure if i rip out the DVD-burner and the floppy drive maybe it will hold. Oh, and the replacement power supply doesn't physically fit the old chassis, but it hangs halfway out the back and sort of rattles around a bit. I sacrifice the rest of the scrap machine, rip out its power supply, stuff the power supply into the old chassis, swap the original disk back in, and ... it boots successfully, finally. That was the shorter version of the story :P So now my colleague has a horrible mess of a frankencomputer which is more likely to fail again in the future, instead of a nice shiny upgrade. Why? Because Microsoft's need to control the flow of software takes priority over the needs of their users. This is what you get when you let Marketing and BizDev drive your technical decisions. Do i still need to explain why i prefer free software? Tags: griping, windows

Martin F. Krafft: A botnet for configuration management

Following my last rant about configuration management, I've had a closer look at Salt, both for my personal use, as well as for a client who would like to deploy something that is not Puppet. Salt has some very good ideas, for instance: But there are also a couple of downsides to Salt: Those are the big issues. There are many small issues two, but those won't be around for too long as the project is moving along quickly and the community is vibrant. This is surely an important point that speaks for Salt. However, the above issues seem to hint at design choices that might well turn out to stand in the way later. Following a day of frustration, I now feel the overpowering urge to write my own configuration management system, because of course I feel that I could do it better than everyone else. Does this sound familiar to you? Let's just say hypothetically that I would, then I'd want to reuse as much existing functionality as possible. For instance, I'd want the entire remote execution framework to be independent from any configuration management implemented on top. So what does this mean? What would such a remote execution framework need? Here are some thoughts: Doesn't this sound like a Unix botnet to you? ;) I could imagine whacking this up with a bit of Python, some shell glue, socat and SSH: the server would have an authorized_keys file with forced commands connecting the client to the server process via sockets. Or I could imagine using twisted for that. But I would prefer if something like this already existed. Anyone? Comments are broken on my blog, and I cannot be bothered to work on them. If you have any input, please write to me. I will (eventually) condense all feedback into a new article. NP: Mouse on Mars: Parastrophics

Russell Coker: Speaking Stacks

Brianna Laugher wrote a blog post about the speaking stack used in the free software activism BoF at LCA 2013 [1]. Occupy Wall St uses what they call a progressive speaking stack this means that white men step back in the queue and people from marginalised groups step forward [2]. During the free software activism BoF the speaking queue that was used was that people who hadn t already spoken had priority over those who had spoken before. This was a really good idea and could be used a lot more in LCA and other conferences. It is fairly common that a small number of delegates take up the vast majority of question time. I suggest that all white men watch the questions and observe how many are asked by white men and how many are asked by everyone else. Also note the way that questions are asked, who shouts a question, who wins when two delegates ask at the same time, and who waits until the end of the talk. The Reasons for a Speaking Stack In Occupy Wall St there is a real benefit in giving priority to members of minority groups. The political needs of white men are generally reasonably well publicised due to disparities in media coverage. As the aim of the occupy movement is not to replace one group of white men with another there is an obvious need to get opinions from members of minority groups. Bugs in software generally affect members of all groups equally (with the exception of bugs related to accessibility features). But even so I think it is important to encourage diversity among people who ask questions. When someone is in the audience sees that no-one who is in their minority group is asking questions they will get the impression that they are just watching someone else s conference. We should aim to have a conference for everyone. How to Implement it When taking questions for one of my talks I generally try to give priority to people who find it difficult to be heard. But doing that requires some concentration and I often don t have any to spare when giving a demanding technical talk. I think that this needs to be managed by the moderator/MC/microphone holder. Someone who doesn t need to think much about the content of the talk can concentrate on choosing the best people to ask questions. Also a significant issue is questions that are called out during a talk. Some speakers insist that questions are only asked at the end of their talk. But I prefer some degree of interaction with the audience so my talks often end up being more about having a conversation with the audience than reading from a script. The difficulty with an interactive talk is that it strongly favors those who are prepared to shout a question over those who wait their turn. I think I ll try to make a strict policy of having people raise their hand to ask a question in future to address this issue, but I will need assistance from someone who s not concentrating on the technical issues. For a conference I think it would make sense for the people who hold the microphones to keep a mental list of who s asked questions. If someone asks their share of questions on the first day of the conference then they would deserve a lower priority for questions on later days. This would also encourage delegates to consider whether their question is really worth asking during the lecture or whether they should save their question quota and talk to the speaker afterwards. Also we could ask delegates to exercise restraint. One suggestion I heard was that people should set themselves a quota of 3 questions per conference or 1 per day. In a conference with ~600 delegates and ~33 sessions per day if everyone asked a question each day that would be about 18 questions per session more than is typical. So it seems that anyone who asks a single question per day is still likely to be asking more than 1/600 of all questions. Exceptions There are occasions when multiple questions and comments make sense. One example is where a member of the audience has significant expertise in the topic in question. Another is when a speaker completes significantly before the end of their allotted time and some questions from the MC or an experienced member of the audience can help them spend all their time educating the audience. But I think there needs to be a compelling reason that has a clear benefit for the audience. General Benefits How many of the repeat questions are useful to the audience? It seems to me that there is a correlation between multiple questions and questions that are more about the person asking than about clarifying issues that are likely to matter to the audience. Would such limits improve the quality of the discussion even for people who don t care about diversity? Also I have asked a disproportionate number of questions in the past. I am reducing the number of questions that I ask although I think I asked more than 3 at this conference.
Related posts:
  1. Questions During Lectures An issue that causes some discussion and debate is the...
  2. Advice for speakers I am not an expert at public speaking. Attending Toastmasters...
  3. what is a BOF? BOF stands for Birds Of a Feather, it s an informal...

Russ Allbery: WebAuth 4.4.1

WebAuth is the site-wide web authentication software that we use at Stanford. We're about to start another round of intensive development on multifactor authentication support, so I wanted to get a minor release out with the other things that have accumulated in the past month and a half. The main feature change in this release is to add support for another WebLogin configuration callback. This one is run whenever WebLogin is attempting to establish the user's identity and can return any identity it wishes, so it's available as a generic callback to determine identity information from the environment. The most likely use will be to inspect the results of client-side certificate authentication (which Apache puts in various environment variables) and interpret them in a site-specific way. Also in this release I refactored the WebLogin scripts so that they use FastCGI more correctly. The application objects are now instantiated only once and then reused for the lifetime of the FastCGI script, rather than torn down and set up for each new request. This is primarily of interest to those using the new features that require a memcached connection, since that connection will no longer be constantly set up and torn down, but it should help with speed in other situations. This is the riskiest change, since if I didn't get it exactly right it would be possible for information to leak from one request to the next. I've audited it fairly carefully, though, and think I've gotten everything. Finally, besides some documentation updates, I've switched the Perl build to Module::Build from ExtUtils::MakeMaker. This should mostly be transparent, but it means some additional Perl modules from CPAN will be required to build the distribution with --enable-webkdc for versions of Perl older than 5.10. (5.10 is fairly old, so I doubt this will be a serious issue.) You can get the latest release from the official WebAuth distribution site or from my WebAuth distribution pages.

Next.

Previous.