Search Results: "zed"

28 October 2013

Keith Packard: Quaternions

Tracking Orientation with Quaternions I spent the flight back from china and the weekend adding orientation tracking to AltOS. I d done a bit of research over the last year or so working out the technique, but there s always a big step between reading about something and actually doing it. I know there are a pile of quaternion articles on the net, but I wanted to write down precisely what I did, mostly as a reminder to myself in the future when I need to go fix the code Quaternion Basics Quaternions were invented by Sir William Rowan Hamilton around 1843. It seems to have started off as a purely theoretical piece of math, extending complex numbers from two dimensions to four by introducing two more roots of -1 and defining them to follow:
i  = j  = k  = ijk = -1
Use these new roots to create numbers with four real components, three of which are multiplied by our three roots:
r + ix + jy + kz
With a bit of algebra, you can figure out how to add and multiply these composite values, using the above definition to reduce and combine terms so that you end up with a set which is closed under the usual operations. Then we add a few more definitions, like the conjugate:
q = (r + ix + jy + kz)
q* = (r - ix - jy - kz)
The norm:
  q   =  (qq*) =  (r  + x  + y  + z )
u is a unit quaternion if its norm is one:
  u   = 1
Quaternions and Rotation Ok, so we ve got a cute little 4-dimensional algebra. How does this help with our rotation problem? Let s figure out how to rotate a point in space by an arbitrary rotation, defined by an axis of rotation and an amount in radians. First, take a vector, v , and construct a quaternion, q as follows:
q = 0 + ivx + jvy + kvz
Now, take a unit quaternion u , which represents a vector in the above form along the axis of rotation, and a rotation amount, , and construct a quaternion r as follows:
r = cos  /2 + u sin  /2
With a pile of algebra, you can show that the rotation of q by r is:
q  = r q r*
In addition, if you have two rotations, s and r , then the composite rotation, t , a rotation by r followed by s can be computed with:
q  = s (r q r*) s*
    = (sr) q (r*s*)
    = (sr) q (sr)*
t   = s r
q  = t q t*
That s a whole lot simpler than carrying around a 3x3 matrix to do the rotation, which makes sense as a matrix representation of a rotation has a bunch of redundant information, and it avoids a pile of problems if you try to represent the motion as three separate axial rotations performed in sequence. Computing an initial rotation Ok, so the rocket is sitting on the pad, and it s tilted slightly. I need to compute the initial rotation quaternion based on the accelerometer readings which provide a vector, g pointing up. Essentially, I want to compute the rotation that would take g and make it point straight down. Construct a vector v , which does point straight up:
g = (0, ax, ay, az) / norm(0, ax, ay, az)
v = (0, 0, 0, 1)
G is normalized so that it is also a unit vector. The cross product between g and v will be a vector normal to both, which is the axis of rotation. As both g and v are unit vectors, the length of their cross product will be sin
a = g   v
  = u sin  
The angle between g and v is the dot product of the two vectors, divided by the length of both. As both g and v are unit vectors, the product of their lengths is one, so we have
cos   = g   v
For our quaternion, we need cos /2 and sin /2 which we can get from the half-angle formulae:
cos  /2 =  ((1 + cos  )/2)
sin  /2 =  ((1 - cos  )/2)
Now we construct our quaternion by factoring out sin from the a and:
q = cos  /2 + u sin   sin  /2 / sin  
Updating the rotation based on gyro readings The gyro sensor reports the rate of rotation along all three axes, to compute the change in rotation, we take the instantaneous sensor value and multiply it by the time since the last reading and divide by two (because we want half angles for our quaternions). With the three half angles, (x,y,z), we can compute a composite rotation quaternion:
   cos x cos y cos z + sin x sin y sin z +
i (sin x cos y cos z - cos x sin y sin z) +
j (cos x sin y cos z + sin x cos y sin z) +
k (cos x cos y sin z - sin x sin y cos z)
Now we combine this with the previous rotation to construct our current rotation. Doing this faster If we read our sensor fast enough that the angles were a small fraction of a radian, then we could take advantage of this approximation:
sin x   x
cos x   1
that simplifies the above computation considerably:
1 + xyz + i (x - yz) + j (y + xz) + k (z - xy)
And, as x, y, z 1, we can further simplify by dropping the quadratic and cubic elements as insignificant:
1 + ix + jy + kz
This works at our 100Hz sampling rate when the rotation rates are modest, but quick motions will introduce a bunch of error. Given that we ve got plenty of CPU for this task, there s no reason to use this simpler model. If we did crank up the sensor rate a bunch, we might reconsider. Computing the Current Orientation We have a rotation quaternion which maps the flight frame back to the ground frame. To compute the angle from vertical, we simply take a vector in flight frame along the path of flight (0, 0, 0, 1) and rotate that back to the ground frame:
g = r (0 0 0 1) r*
That will be a unit vector in ground frame pointing along the axis of the rocket. The arc-cosine of the Z element will be the angle from vertical. Results All of the above code is checked into the AltOS git repository I added a test mode to the firmware that just dumps out the current orientation over the USB link which lets you play with rotating the board to see how well the system tracks the current orientation. There s a bit of gyro drift, as you d expect, but overall, the system tracks the current orientation within less than a tenth of a degree per second. Even with all of this computation added, the whole flight software is consuming less than 7% of the STM32L CPU time.

27 October 2013

Petter Reinholdtsen: Teaching vmdebootstrap to create Raspberry Pi SD card images

The vmdebootstrap program is a a very nice system to create virtual machine images. It create a image file, add a partition table, mount it and run debootstrap in the mounted directory to create a Debian system on a stick. Yesterday, I decided to try to teach it how to make images for Raspberry Pi, as part of a plan to simplify the build system for the FreedomBox project. The FreedomBox project already uses vmdebootstrap for the virtualbox images, but its current build system made multistrap based system for Dreamplug images, and it is lacking support for Raspberry Pi. Armed with the knowledge on how to build "foreign" (aka non-native architecture) chroots for Raspberry Pi, I dived into the vmdebootstrap code and adjusted it to be able to build armel images on my amd64 Debian laptop. I ended up giving vmdebootstrap five new options, allowing me to replicate the image creation process I use to make Debian Jessie based mesh node images for the Raspberry Pi. First, the --foreign /path/to/binfm_handler option tell vmdebootstrap to call debootstrap with --foreign and to copy the handler into the generated chroot before running the second stage. This allow vmdebootstrap to create armel images on an amd64 host. Next I added two new options --bootsize size and --boottype fstype to teach it to create a separate /boot/ partition with the given file system type, allowing me to create an image with a vfat partition for the /boot/ stuff. I also added a --variant variant option to allow me to create smaller images without the Debian base system packages installed. Finally, I added an option --no-extlinux to tell vmdebootstrap to not install extlinux as a boot loader. It is not needed on the Raspberry Pi and probably most other non-x86 architectures. The changes were accepted by the upstream author of vmdebootstrap yesterday and today, and is now available from the upstream project page. To use it to build a Raspberry Pi image using Debian Jessie, first create a small script (the customize script) to add the non-free binary blob needed to boot the Raspberry Pi and the APT source list:
#!/bin/sh
set -e # Exit on first error
rootdir="$1"
cd "$rootdir"
cat <<EOF > etc/apt/sources.list
deb http://http.debian.net/debian/ jessie main contrib non-free
EOF
# Install non-free binary blob needed to boot Raspberry Pi.  This
# install a kernel somewhere too.
wget https://raw.github.com/Hexxeh/rpi-update/master/rpi-update \
    -O $rootdir/usr/bin/rpi-update
chmod a+x $rootdir/usr/bin/rpi-update
mkdir -p $rootdir/lib/modules
touch $rootdir/boot/start.elf
chroot $rootdir rpi-update
Next, fetch the latest vmdebootstrap script and call it like this to build the image:
sudo ./vmdebootstrap \
    --variant minbase \
    --arch armel \
    --distribution jessie \
    --mirror http://http.debian.net/debian \
    --image test.img \
    --size 600M \
    --bootsize 64M \
    --boottype vfat \
    --log-level debug \
    --verbose \
    --no-kernel \
    --no-extlinux \
    --root-password raspberry \
    --hostname raspberrypi \
    --foreign /usr/bin/qemu-arm-static \
    --customize  pwd /customize \
    --package netbase \
    --package git-core \
    --package binutils \
    --package ca-certificates \
    --package wget \
    --package kmod
The list of packages being installed are the ones needed by rpi-update to make the image bootable on the Raspberry Pi, with the exception of netbase, which is needed by debootstrap to find /etc/hosts with the minbase variant. I really wish there was a way to set up an Raspberry Pi using only packages in the Debian archive, but that is not possible as far as I know, because it boots from the GPU using a non-free binary blob. The build host need debootstrap, kpartx and qemu-user-static and probably a few others installed. I have not checked the complete build dependency list. The resulting image will not use the hardware floating point unit on the Raspberry PI, because the armel architecture in Debian is not optimized for that use. So the images created will be a bit slower than Raspbian based images.

25 October 2013

Jorge Salamero Sanz: Zentyal is hiring (C/C++ developers for Samba4/OpenChange)

Zentyal is seeking to hire a results driven and autonomous software C/C++ developers, motivated to work in a start-up environment and to become part of an open source company. You will be actively involved into Zentyal products -OpenChange and Samba projects-, digging into the internals, answering escalated customers requests and participating in pushing the full solution to enterprise level. Also, you will take part in discussions and proof of concept developments related to scalability, high availability and best efficiency scenarios. Check Zentyal careers website for more information, or ping us in IRC!

24 October 2013

Russ Allbery: Review: Applied Cryptography

Review: Applied Cryptography, by Bruce Schneier
Publisher: John Wiley & Sons
Copyright: 1996
ISBN: 0-471-11709-9
Format: Trade paperback
Pages: 621
This is the next entry in the series of Russ reading books that he bought years ago and never got around to reading. Thankfully, this time, the book has aged somewhat better. This review is for the second edition of Applied Cryptography, published in 1996. Given how important computer security has become, and how central cryptography is to computer security, one might think that the passage of 17 years would make a book effectively obsolete. This turns out not to be the case. Yes, Rijndael (the current AES standard and the most widely-used block cipher), Camellia (the up-and-comer in the block cipher world), and the SHA-2 hash postdate this book and aren't discussed. Yes, there have been some further developments in elliptic-curve public-key cryptography. And yes, much of the political information in this book, as well as the patent situation for public-key cryptosystems, is now mostly of historical interest. But a surprising amount of this book still applies directly. Partly that's because much of Applied Cryptography is focused on general principles, classes of algorithms, and cryptographic protocols, and those do not change quickly. A rather amazing number of techniques and protocols still in widespread use today originated in the 1970s. Block ciphers, stream ciphers, cipher modes, public-key systems, signature systems, hash functions, key exchange, secret splitting, key management, and other, similar topics have not changed substantially. And even in the area of specific algorithms, DES is still in use (unfortunately) and its analysis is valuable for understanding any block cipher. RC4, Triple DES, CAST, RSA, MD4, MD5, and SHA-1 are still part of the cryptography landscape, and Schneier was already warning against using MD5 in 1996. The source code included in this book has, thankfully, been made obsolete by widely-available, high-quality free implementations of all common ciphers, available for any modern operating system. But the math, the explanations, and much of the cryptanalysis is still quite relevant. While it contains a comprehensive introduction to cryptographic concepts, Applied Cryptography is structured more like a reference manual than a tutorial. The first two chapters provide some foundations. Then, a wide variety of protocols, from the common to the completely esoteric, are discussed across the subsequent four chapters. There are four chapters on the basics of ciphers, another chapter on the mathematics, and then the longest part of the book: twelve chapters that provide a tour of all major and several interesting minor protocols that existed in 1996. These are organized into block ciphers, stream ciphers (and pseudo-random number generators), hash functions, and public-key systems. An entire chapter is devoted to the history and analysis of DES, and I think that was my favorite chapter of the book. Not only is it worth understanding how DES works, but it also provides a comprehensive introduction to the complexities of block cipher design, the politics of interactions with the NSA, and a fascinating history of early computer cryptosystems. Finally, Applied Cryptography closes with chapters on real-world systems and on politics. The politics section is mostly a historical curiosity, as is much of the chapter on real-world systems. Schneier discusses some of the PKCS and X.509 protocols here but doesn't use SSL as one of his examples despite its specification predating this book (perhaps the most glaring omission of the book). But he does discuss Kerberos (which is of personal interest to me), and it was useful to see it analyzed in the broader context of this sort of book. Those who read Schneier's blog regularly, or who have read any of his other books, will know that he's concise, incisive, and very readable, even with difficult material. I think he does err on the side of compactness this is not a book that handholds the reader or explains something more than once but there's a lot of ground to cover, so I can't really blame him. Expect to have to read the more difficult parts several times, and (at least on a first read) expect to just skip past some of the most complex sections as probably not worth the effort. But I think Schneier does a great job structuring the book so that one comes away with a high-level impression and overview sufficient to make informed guesses about capabilities and relative difficulty even if one doesn't have the patience to work through the details. I suspect most readers will want to skim at least part of this book. Unless you're willing to do a lot of background reading to understand the math, or a lot of careful study to follow the detailed descriptions and flaws in often-obscure algorithms, the algorithm discussions start to blend together. I found myself skipping a lot of the math and focusing on the basic descriptions and the cryptanalysis, particularly for the obscure ciphers. Many of the algorithms are also highly opaque and of obscure benefit; I doubt I will ever care about blind signatures or oblivious transfer, and I struggled to wrap my mind around the details of bit commitment (although it's apparently important in zero knowledge proofs, which in turn are relevant to authentication). But this is not the sort of book where you have to read every word. It's very well-structured, provides clear resynchronization points for readers who have had enough of a particular topic, and marks the topics that are more obscure or of dubious general usefulness. The most useful parts of this book, in my opinion, are the basic conceptual introductions to each class of cryptographic algorithm or protocol, and those chapters are excellent. I've been working in applied computer security, at the application and deployment level, for years, and this is the first time that I've felt like I really understood the difference between a block cipher and a stream cipher, or the implications of block cipher mode selection. Applied Cryptography is also a fascinating exercise in adjusting one's intuition for where the complexity lies. It's a memorable experience to move from the chapters on block ciphers, full of S-boxes and complex mixing and magic numbers, into the far more straightforward mathematical analysis of stream ciphers. Or the mathematical simplicity of RSA (although quite a bit of complexity is lurking on the cryptanalysis side, much of which is only touched on here). Of more historical interest, but still quite impressive, is that Applied Cryptography doubles as a comprehensive literature review. Not only does it cover nearly every algorithm in use at the time, it discusses the cryptography literature on nearly every topic with brief summaries of results and lines of investigation. The references section is a stunning 66 pages of small print featuring 1,653 references, and those are mentioned and put in context throughout the book. I'm not personally interested in chasing those lines of research further, and of course 17 years means there are many new papers that are as important to read today, but it's obvious that this book would have been an amazing and invaluable map to the research territory when it was first published. There are now newer books on this topic that you should consider if you're purchasing a book today. Cryptography Engineering in particular is in my to-read pile, and I'm interested to see how much of this book it could replace, although I believe it's lighter on the mathematical analysis and details. But I think Applied Cryptography still has a place on the bookshelf of any computer security professional. Schneier is succinct, detailed, straightforward, and comprehensive, at least for 1996, and I know I'll be reaching for this book the next time I forget the details of CFB mode and then get hopelessly confused by the Wikipedia article on the topic. Rating: 8 out of 10

23 October 2013

Russ Allbery: Review: Lean In

Review: Lean In, by Sheryl Sandberg & Nell Scovell
Publisher: Alfred A. Knopf
Copyright: March 2013
ISBN: 0-385-34994-7
Format: Hardcover
Pages: 217
Lean In is normally listed as written only by Sheryl Sandberg, which is also what's on the dust jacket, but the inside cover says "with Nell Scovell." Given what that normally means for celebrity books, supported by comments in the acknowledgments, I'm listing it as co-written by Scovell. However, much of the book is written in the first person and is clearly meant to be Sandberg's voice, so I will mostly refer to Sandberg as the author for the rest of the review. I was unfair to this book. Lean In made quite a splash when it first came out, since the named author is the Chief Operating Officer of Facebook, a former vice president of Google, and one of the most prominent female executives in the business world. The subtitle is Women, Work, and the Will to Lead, and the normal Internet summary was that it focused on how women could change their behavior to be more effective and successful in the business world. Throw in some ridiculous review hyperbole (one New York Times writer compared it to The Feminine Mystique, which is a bit like the comparison of every extruded fantasy product with The Lord of the Rings) and I was ready to hate this book. It's work advice from someone in a very privileged position, and it seemed to have a distinct whiff of victim-blaming. It's obvious, however, from the first few pages that Sandberg was very aware of this trap and had no intent of falling into it. This is hard: giving practical advice to women on how to navigate a sexist culture runs a severe risk of victim-blaming even with the best of intentions, particularly since part of the sexism of the culture is a pervasive impression that women aren't doing enough. But I think Lean In handles that about as well as one could while still writing a book focused on practical advice rather than revolutionary change. Sandberg is clear that many things women can do to help their careers shouldn't be necessary, and she takes some time, in each section, to talk about the shape of any structural unfairness that she's seen. What advice she offers here is clearly in the spirit of how to navigate a system that is unfair and unequal, but which can be navigated and can be made more fair by more women participating. This is a very practical book, with more in common with business self-help books than with feminist critiques. It's also a narrowly targeted book, not that you'd guess this from the number of copies it's sold and the sort of publicity it got. Lean In is advice to female executives about how to navigate the top levels of US business culture, with a strong eye for issues of feminism, gender-based double standards, and potential traps of self-sabotage. It offers a blend of encouragement, practical tips, and some forthright analysis that tries to bring to the surface internalized sexism and perceived barriers and limitations for female executives so that they can be addressed and hopefully overcome. It's not in any way a revolutionary book, either with respect to business culture or with respect to gender roles; rather, it's an open acknowledgment of a whole bunch of problems combined with practical advice for how to work around those problems until they can be solved. For the most part, it doesn't attempt to address how to solve the problems. There is nothing wrong with that as a focus for a book. Lean In has attracted a fair bit of criticism in some circles, but, after reading the book, I think a lot of that criticism is predicated on assuming different goals than this book actually has. Contrary to various glowing reviews, Lean In is not a book that's going to change the world; rather, it's a book that might help a narrow band of the population in a specific job and career path navigate problems of sexism and gender expectations and be more effective. Read in that light, it struck me as honest, open, considered, and well aware of its own limitations. Sandberg says repeatedly that many of the options she discusses simply aren't available to most women, and that women may not be in a position to be able to make the decisions she made. She's quite forthright about the limitations of her own experience and the limitations of the advice that she offers, and I think deserves full points for honesty. Similarly, when it comes to business culture, Lean In is not about changing that culture for the better. It's about navigating that culture as it exists. Here, I think Sandberg is less aware of the limitations of this perspective; there is far less acknowledgment in Lean In that executive culture may be inherently toxic as there is of the limitations that women face in business. But while Sandberg has an obvious blind spot, this is not a paean to overwork. It has some excellent advice about work-life balance, including the need to put strict limits on the amount of time one spends in the office and the necessity for someone to pick up substantial burdens in the home if one of the members of a household is going to pursue a business executive job. I was worried, going in, that Lean In would excuse, justify, or idealize the culture of exploitation so common with salaried jobs in the United States, but it really doesn't. As with feminism, it doesn't pick any deep fights, and those with a "kill it with fire" gut reaction towards US executive culture (such as myself) will find the bland acceptance of it irritating, but this is more of an issue with reader expectations than with the book. One should not expect a highway map to point out that trains are more efficient. So, I'm willing to stand up for Lean In to a degree. This is not a bad book. It's very readable, parts are quite interesting, and some of Sandberg's observations about the intersection of gender expectations and executive culture were worth hearing. I learned some things from it. But I think one should read it in full awareness of its limitations. This is a book of practical advice that will only be of direct use to 10% of the population, and which could be generalized with some effort to maybe 25%. It has little to say to women who are not in white-collar, professional jobs, and its applicability to women who aren't aiming for executive careers is a bit hazy. It's the kind of book that's full of anecdotes about how Sandberg got parking spots reserved for pregnant women at Google by talking to the founders about it, or about how to negotiate salary when offered an executive position, which are vaguely interesting but of dubious practical benefit to people who aren't offered jobs where salary and benefit negotiation is available. This isn't the fault of the book, but at times it is rather stunning how different (and how much more tractable) the world looks to someone with the wealth and personal connections that Sandberg has. (To her credit, Sandberg does seem to be aware of this and does acknowledge it.) If you are a female executive, or a male executive interested in developing your empathy and understanding of how institutional sexism affects your colleagues, I suspect Lean In would be quite helpful. If you're not, I think it's mostly a curiosity. But it's also well-written, well-organized, probably available in most libraries in the United States, and provides anecdotal data to help build a more complete picture of the concerns and experiences of female executives. Unless you're in the specific target audience, I'm not sure I'd recommend going out of your way to find it, but it's probably worth reading if it falls into your lap. Rating: 7 out of 10

14 October 2013

Bits from Debian: Ada Lovelace Day: meet some of the "women behind Debian"!

Today is Ada Lovelace Day:
"Ada Lovelace Day is about sharing stories of women - whether engineers, scientist, technologists or mathematicians - who have inspired you to become who you are today. The aim is to create new role models for girls and women in these male-dominated fields by raising the profile of other women in STEM". source
To celebrate, we asked to some of the "women behind Debian" to share their stories with us. Enjoy! Alt Ada Lovelace portrait Ana Guerrero Lopez (ana) Who are you? I'm a 30-something years old geek. I'm from Andalusia, Spain but live in France. What do you do in Debian? I work mostly on my packages, in sponsoring new people's packages and in this very blog you're reading now. I also maintain an unofficial Google+ page about Debian. At $PAID job, I work in an internal Debian distribution so from time to time, I get the opportunity to contribute back some of the stuff we do there. How and why did you start contributing to Debian? I started using Debian around 2003 switching from Mandrake. I was a happy Debian user when the Debian Women project started in the summer 2004. When I saw the project announced, I asked myself why I wasn't contributing to Debian and the rest is history... in a couple of weeks it'll be my 7 years DD-versary! If for some reason you want a longer reply to this question, read here. Beatrice Who are you? I am a PhD student with a degree in Biology. I am a computer fan since my first C64 and I am a self-taught computer geek wanna-be. And I am a bug fan - not software bugs, real bugs :) What do you do in Debian? I work on translations - doing the translation work itself, but also reviewing other translators' work and helping in coordinating the translation effort. How and why did you start contributing to Debian? I started using Linux because I liked the idea of an open source operative system based on collaboration and I began reviewing open source software translations. Since my first Linux system was Debian Potato and I sticked to Debian ever since, it only seemed natural to focus my translation work on Debian. Christine Caulfield Who are you? My name is Christine Caulfield. My day job is principal software engineer at Red Hat working on the cluster infrastructure components corosync & pacemaker. Outside computing I'm a musician and sound engineer. I play violin with lots of technology attached, and love avant garde music. What do you do in Debian? I'm not that active on Debian any more due to pressure of time, and maturity of the packages I work on. I currently maintain the, little-used, DECnet userspace packages and the, even less used I suspect, mopd bootloader. I used to maintain lvm2 for a while but dropped that a few years ago. How and why did you start contributing to Debian? My initial reasons for joining Debian were slightly selfish, to find a home for the DECnet project that I was heavily involved in at the time. I was a keep Debian user and people wanted a distribution where the software was easy to set up. DECnet is quite complicated for users to configure, being a totally independant networking stack to IP and so OS support is needed. Debian seemed like the logical place to make this happen. As mentioned above I got quite involved for a time and maintained other packages too. I picked up lvm2 because I was on the lvm2 dev team at work in Red Hat and as it was a new package at that time I seemed a logical choice. Elena Grandi (valhalla) Who are you? I'm a 30-something years old geek and Free Software enthusiast from Italy. What do you do in Debian? I'm currently maintaining a few packages (2 python modules and a python program) as a sponsored uploader; I'm also slowly looking around for other things to do (by preference technical, but not limited to packaging), with the aim to spend more time contributing to Debian. How and why did you start contributing to Debian? For a while I had being distro-hopping between "fun" distributions (the ones that break now and then) on the desktop while using Debian stable on the home server and in chroots. I was already doing marginal contributions to those distributions, where finding stuff that was missing was easy, but my perception as a stable user was that Debian was already working fine and probably didn't really need any help. Then I started to socialize on IRC with some DDs and DMs, and realized that my perception was superficial and that in reality there were dark holes in the depths of the archive where Evil festered and prospered and... ok, sorry, I got carried away :) Anyway, since I was actually using Debian more and more I decided to start contributing: I read documentation, I attended the useful IRC sessions on #debian-women and decided that it was probably best not to add new stuff, but look for things that I used and that needed help. Then nothing happened for a while, because finding stuff that doesn't work is hard (at least on my mostly textual systems). Then one day I was trying to write a python script that needed to verify gpg signed messages; it had to run on my Debian server, so I was trying to use python-pyme and its documentation was painful to use, while I remembered an earlier attempt using python-gnupg that was much more pythonic, but not available in Debian. In a fit of anger I decided to forgo all of my good intentions and actually add a new package: I checked the sources for problems, packaged, sent it to mentors@d-o, got reviews, fixed problems, resent and finally got sponsored and well, everything started. Francesca Ciceri (madamezou) Who are you? I'm Francesca, a 30-something Italian graduated in Social Sciences. What do you do in Debian? I'm a (non uploading) Debian Developer since 2011 and have been DPN editor, press officer, webmaster for www.debian.org and translator for the Italian l10n team. Recently, due to time constraints, I had to reduce my involvement and now only work on two things: writing/editing articles for bits.debian.org together with Ana Guerrero, and creating subtitles for the DebConf talks, in the DebConf Subs team. How and why did you start contributing to Debian? Basically thanks to the sudden abundance of free time - due to an health problem - and the desire to give something back to this wonderful operating system. After that, I found out that Debian is not only a great OS but also a very special community. Today, some of my dearest friends are people I met through Debian. :) Laura Arjona Who are you? I live in Madrid (Spain), and work as IT Assistant in the Technical University of Madrid (UPM). I'm married and I have a 4-years-old son. What do you do in Debian? In 2012 I started to clean spam and to translate Debian web pages into Spanish. I also follow the work of the web and publicity team, I hope I'll get more involved there too. And of course, I'm in Debian Women :) How and why did you start contributing to Debian? I'm using Debian at work since 2007 (servers), and in my desktops since 2010. I like very much that it is a community distro and I wanted to participate. I was already doing translations in other (small) free software projects, so I began here too. The Debian-Women list, the planet, and people in identi.ca helped me to learn a lot and feel part of the community even when I was not contributing yet. M nica Ram rez Arceda (monica) Who are you? My name is M nica Ram rez Arceda and I am an enthusiast of free software and sharing knowledge cultures: for me it's a kind of philosophy of life. I studied Maths a long time ago but ended up working as a developer for some years. Now I'm working as an IT teacher. What do you do in Debian? Debian is a huge project, so you can help in various scopes. Mainly, I work on packaging, fixing wnpp bug inconsistencies in BTS and helping in spam cleaning of the mailing lists. But I also enjoy doing some non-technical work from time to time: the project I am just now involved is organizing, with the rest of Debian Catalan community, a local team to propose Barcelona as the venue for a minidebconf where all the speakers will be women. How and why did you start contributing to Debian? In 2000 I discovered Free Software world and I fell deeply in love with its philosophy. Since then, I've been trying to do my best in different activities, like spreading the word, giving free courses, helping collectives and friends in technical stuff (from installing Debian to developing some helping apps for them)... but two years ago I was looking forward to join a free software project and I decided to try Debian, since it has been my first and only distro in my day-to-day life for about ten years. So, I wanted to give back Debian all what it had offered to me, but.... I thought I couldn't (hey, Debian is for real hackers, not for you little ant!), but I started to adopt some orphaned packages, do some QA uploads, fix some RC bugs, talk with some Debian Developers that helped me and encouraged me more than I expected, I traveled to my first Debconf... And one thing takes you to the other, and on March 2012 I became a DD. Now, I'm glad to see that everything that frightened me is not so scary :-)

13 October 2013

Francesca Ciceri: MissRepresentation: discussing gender stereotypes in Moldova

Recently, I've been attending a training course in Moldova about the representation of gender in the media and the influence on society of gender stereotypes.
The course was organized by Inesa Lupu, of Invento and financed by the UE as part of the Youth in Action program. It was the first time, for me, in this kind of international project and it was, in a word, fantastic.
Among the participants, there were people from Azerbaijan, Georgia, Italy, Latvia, Moldova and Slovenia.
That meant a great mix of cultures, languages and experiences.
And that was the key, in my opinion, of the success: coming from different countries and different social and cultural contexts meant that we were able to have a very broad and diverse point of view on the problem of gender stereotypes. So, here's some highlights on what we discussed. The media representation of gender (aka: gender issues are not only women issues) Tough Guise by Jackson Katz is a interesting documentary about the way the modern society - particularly in the US - defines masculinity as intrinsically tied to violence, aggressivity and "toughness" and on the consequences of this on the society as a whole. The author's theory is "that male violence, misogyny, and homophobia are inextricably linked to how we define manhood as a culture. The film gives special attention to how American media have glamorized increasingly regressive and violence masculine ideals in the face of mounting social and economic threats to traditional white male heterosexual authority". It then results clear that we cannot avoid to think about gender issues as women and men issues, because after all is the cultural definition of both femininity and masculinity that heavily influences the behaviour of men and women and their interaction.
In particular, I liked Katz's analysis of media's use of victim blaming speech while reporting of violences against women which tend to shift the focus from the violence perpetrated by men as part of definition of masculinity. Another documentary we watched, more focused on women representation, is MissRepresentation by Jennifer Siebel Newsom.
The film analyses the under representation of women in position of power, and the issue of women and leadership, especially in politics, in the US. Nothing too new, if you are interested in gender issues, as it is a widely discussed and studied topic, but still worth watching. After watching them, we formed little groups of three-four persons from different countries to discuss them, and it was even more interesting to listen the opinion of the guys (we had four male participants). The guys talking about Tough Guise Photography and Phototherapy As a workshop on media representation of gender, photography and advertisement were two of the main argument we spoke about. We focused on it for two days: our trainer, Lietta Granato - photographer and phototherapist - first gave us a lesson on photography, making us experimenting a bit with exposure times and diaphragm's aperture. a lesson on photography Then she presented the evolution in the last 50 year or so, of the depiction of women in the ads.
And that, to be honest, was a bit upsetting.
I've already had seen one of the most important and widely known films on this theme, the Killing Us Softly series by Jean Kilbourne, but some of the ads Lietta showed to us were really really horrible. Actually, most of them were horrible. Commenting and deconstructing these ads would take a separate blogpost, probably, and there are many people more competent than me on the topic, so I've decided to simply share some of the worst ads in a collage, without much commenting. These are vintage, from the fifties to the seventies. a collage of vintage sexist ads Do you think we got better? Pff. a collage of recent sexist ads They go from promoting violence and rape to heavily objectifying women. Special mention for the Compaq ad: as a geek woman I feel particularly insulted by the idea that I'm supposed to use a "pocket pc" as "pocket mirror". Because I wouldn't be able, obviously, to use it as a computer. Lietta also introduced us to photographic therapy and particularly the work of Jo Spence and Rosy Martin. If you browse Jo Spence's site, please be aware that there is a series of pics documenting all the stages of her illness (she died of breast cancer) and they are pretty intense. "In 1984, alongside Rosy Martin, Spence developed Photo-Therapy , adopting techniques from co-counselling. The considerable achievement of Photo-Therapy was to invert the traditional relationship between the photographer and the subject. If historically the subject had little control over their own representation, Photo-Therapy shifts this dynamic. The subject was able to act out personal narratives and claim agency for their own biography." (source: http://www.jospence.org/biography.html)
I love this concept and I really really love Spence's photos. To close on a happier note, I really suggest you to allocate 20 minutes of your time to watch this great talk by Nigerian novelist Chimamanda Ngozi Adichie, about the role of "single story" in shaping stereotypes and misunderstandings.

11 October 2013

Petter Reinholdtsen: Oslo community mesh network - with NUUG and Hackeriet at Hausmania

Wireless mesh networks are self organising and self healing networks that can be used to connect computers across small and large areas, depending on the radio technology used. Normal wifi equipment can be used to create home made radio networks, and there are several successful examples like Freifunk and Athens Wireless Metropolitan Network (see wikipedia for a large list) around the globe. To give you an idea how it work, check out the nice overview of the Kiel Freifunk community which can be seen from their dynamically updated node graph and map, where one can see how the mesh nodes automatically handle routing and recover from nodes disappearing. There is also a small community mesh network group in Oslo, Norway, and that is the main topic of this blog post. I've wanted to check out mesh networks for a while now, and hoped to do it as part of my involvement with the NUUG member organisation community, and my recent involvement in the Freedombox project finally lead me to give mesh networks some priority, as I suspect a Freedombox should use mesh networks to connect neighbours and family when possible, given that most communication between people are between those nearby (as shown for example by research on Facebook communication patterns). It also allow people to communicate without any central hub to tap into for those that want to listen in on the private communication of citizens, which have become more and more important over the years. So far I have only been able to find one group of people in Oslo working on community mesh networks, over at the hack space Hackeriet at Husmania. They seem to have started with some Freifunk based effort using OLSR, called the Oslo Freifunk project, but that effort is now dead and the people behind it have moved on to a batman-adv based system called meshfx. Unfortunately the wiki site for the Oslo Freifunk project is no longer possible to update to reflect this fact, so the old project page can't be updated to point to the new project. A while back, the people at Hackeriet invited people from the Freifunk community to Oslo to talk about mesh networks. I came across this video where Hans J rgen Lysglimt interview the speakers about this talk (from youtube): I mentioned OLSR and batman-adv, which are mesh routing protocols. There are heaps of different protocols, and I am still struggling to figure out which one would be "best" for some definitions of best, but given that the community mesh group in Oslo is so small, I believe it is best to hook up with the existing one instead of trying to create a completely different setup, and thus I have decided to focus on batman-adv for now. It sure help me to know that the very cool Serval project in Australia is using batman-adv as their meshing technology when it create a self organizing and self healing telephony system for disaster areas and less industrialized communities. Check out this cool video presenting that project (from youtube): According to the wikipedia page on Wireless mesh network there are around 70 competing schemes for routing packets across mesh networks, and OLSR, B.A.T.M.A.N. and B.A.T.M.A.N. advanced are protocols used by several free software based community mesh networks. The batman-adv protocol is a bit special, as it provide layer 2 (as in ethernet ) routing, allowing ipv4 and ipv6 to work on the same network. One way to think about it is that it provide a mesh based vlan you can bridge to or handle like any other vlan connected to your computer. The required drivers are already in the Linux kernel at least since Debian Wheezy, and it is fairly easy to set up. A good introduction is available from the Open Mesh project. These are the key settings needed to join the Oslo meshfx network:
SettingValue
Protocol / kernel modulebatman-adv
ESSIDmeshfx@hackeriet
Channel / Frequency11 / 2462
Cell ID02:BA:00:00:00:01
The reason for setting ad-hoc wifi Cell ID is to work around bugs in firmware used in wifi card and wifi drivers. (See a nice post from VillageTelco about "Information about cell-id splitting, stuck beacons, and failed IBSS merges! for details.) When these settings are activated and you have some other mesh node nearby, your computer will be connected to the mesh network and can communicate with any mesh node that is connected to any of the nodes in your network of nodes. :) My initial plan was to reuse my old Linksys WRT54GL as a mesh node, but that seem to be very hard, as I have not been able to locate a firmware supporting batman-adv. If anyone know how to use that old wifi access point with batman-adv these days, please let me know. If you find this project interesting and want to join, please join us on IRC, either channel #oslohackerspace or #nuug on irc.freenode.net. While investigating mesh networks in Oslo, I came across an old research paper from the university of Stavanger and Telenor Research and Innovation called The reliability of wireless backhaul mesh networks and elsewhere learned that Telenor have been experimenting with mesh networks at Gr nerl kka in Oslo. So mesh networks are also interesting for commercial companies, even though Telenor discovered that it was hard to figure out a good business plan for mesh networking and as far as I know have closed down the experiment. Perhaps Telenor or others would be interested in a cooperation? Update 2013-10-12: I was just told by the Serval project developers that they no longer use batman-adv (but are compatible with it), but their own crypto based mesh system.

10 October 2013

Russ Allbery: Review: 2312

Review: 2312, by Kim Stanley Robinson
Publisher: Orbit
Copyright: 2012
ISBN: 0-316-19280-5
Format: Kindle
Pages: 563
Swan Er Hong is a native of Mercury, granddaughter of one of the most influential women in the solar system. Alex, known as the Lion of Mercury, has just died as the story opens, and Swan is mourning. But she's also discovering that something else has been going on, something that Alex had not confided in her. Something that involves an inspector from the asteroids, a scientist from Titan, and a message Alex left in the event of her death, asking Swan to convey something personally to another scientist on Io. And then Mercury is attacked and the situation becomes much more complicated. 2312 is, like most Robinson novels, built on top of monumental world-building. The date is given away by the title; the setting is a solar system full of terraforming, habitats, and explosive human colonization. There are human settlements on every world that could possibly sustain them, including a terraformed Mars and a vast in-progress terraforming project on Venus. Mercury, Song's home, supports the city known as Terminator, a shielded habitat that travels around the planet on rails, pushed forward by the thermal expansion of metal at sunrise. Humans have scattered across the solar system and changed everything they've touched, even the asteroids. Many of those have been repurposed as long-haul shuttles: self-contained habitats on carefully-designed routes on which people live for months or years until they get to their destinations. Parts of this book are gorgeous. One of Robinson's strengths is creating images of the vastness of space, science, and human endeavor that feel plausible and inspiring at the same time. The setting of 2312 frees him to go wild with mega construction projects, and he takes full advantage. Parts of the book inspire the same sense of awe as the Hoover Dam or the Great Wall of China, transplanted into space and backed with neat scientific ideas to ponder. (Even if the more skeptical will note certain human habitation hazards, such as prolonged weightlessness and radiation, are mostly hand-waved away.) Unusually for a Robinson novel, 2312 also features an interesting and vibrant protagonist. Usually I can barely remember Robinson's protagonists five minutes after I finish the book, but Swan is so impulsive, erratic, and emotionally intense that I doubt I'll forget her. Her past is stuffed with outrageous risks and intense creativity: having songbird neurons transplanted into her brain, ingesting extraterrestrial bacteria, designing habitats in the asteroid belt, and creating art on the plains of Mercury. As the story gets underway, she meets the oversized and mellow Wahram; at first, she finds him boring, but then they're thrown together by one of the best set pieces of the book, and both they and the reader discover they work well together. Wahram serves as a straight man and as a stand-in for the reader's opinion, showing Swan all the more clearly in contrast. All this sounds quite promising. Unfortunately, what makes 2312 an ambitious failure rather than an excellent story is the story part. Robinson has built an amazing world, but he doesn't seem to know what to do with it. His economy, at least outside of Earth (and more on that in a moment), is essentially post-scarcity. Few of the characters we meet seem to have any concept of fiscal or practical limitation. They head across the solar system on a moment's notice, take vacations in habitats, or head to favorite spots for the view. It felt like Robinson was trying to write a Culture story. Unlike Banks, however, he struggles with generating a plot. 2312 is, in structure, a murder mystery. The characters witness a crime, survive it, and then intend to investigate it, and the crime appears to grow and link to fault lines in their society. This could have worked, but it has two problems: their society is only barely coherent enough for the reader to perceive the potential fault lines, and Robinson seems to have no idea how to construct a mystery plot. There is no sense of coherence to Swan and Wahram's approach, no sense of a gathering of clues, and very little sense of drama. All of the plot revelations are either dropped in Swan's lap by another character at a convenient moment or generated by Swan refusing to take the plot of the book seriously. The characters do essentially no meaningful investigation on-camera and show almost no investment in the outcome of the plot. When the climax comes, it's all weirdly forgettable. There are also large sections of the book that appear to have nothing to do with the rest of the plot, and that's where the unfortunate interval on Earth comes in. Robinson takes advantage of the scenes on Earth to do a bit of alienation and shows how foreign and strange and stifling Earth feels to someone who grew up outside of its atmosphere. Parts of this work, but he puts the plot on hold to do it. And parts of it do not work at all. The most glaring example is Swan and Wahram's bizarre bit of attempted charity in Africa, which comes across as stunningly high-handed and arrogant. This could be in character, particularly for Swan (who is not long on empathy), but, if so, the book doesn't signal to the reader that it should be read that way. Instead, there are some side (or snide) comments that seem to indicate Robinson knows nothing about the economic arc of Africa from the past twenty years. And when their absurd, botched, condescending charity plan fails for all the obvious reasons, the characters, and apparently the novel, throw up their hands and write Earth off as a stagnant lost cause that can't accept the imposition of a good idea and go back to the plot, never apparently caring about Earth again. Almost as frustrating is the way that these interludes are tied back into the story, which is usually through Swan getting ridiculously lucky on her random encounter rolls. It felt like whenever Robinson needed to make progress in the plot, Swan would just accidentally run into exactly the right person or situation to bring up the next plot point or to have some investigation make sense. (Not that Swan usually figured this out. Normally, the inspector explains it to her.) The author's finger was planted so firmly on the scales that it destroyed my suspension of disbelief and made a mockery of the idea that the characters were actually investigating anything. 2312 is built around a skeleton of a plot, but the lack of engagement with it, the lack of tension and emotion, the way the next developments are generally narrated to the protagonists and the reader, and the repeated use of random encounters to steer it left me without much reason to care. Robinson tries a few twists, but since the story never felt committed to its plot anyway, those twists feel less like planned complications and more like another random veer in the road. It didn't help that the final outcome was more prosaic and forgettable than the book had been implying it would be. There is a lot to like here. I'm very pleased to see Robinson finally write a memorable protagonist, and he's very good at both world-building and set pieces. But it needed a plot, and it needed a more coherent and complete cultural backdrop for its characters. Without both, it fell back into the typical Robinson trap: gorgeous moments separated by a whole lot of boring, and an overall impression of a construction tour rather than a story. There are bits of it I loved and still remember, particularly Song and Wahram in the tunnels of Mercury (which is possibly the best extended characterization Robinson has ever written). But the book as a whole is a mess, and I can't recommend wading through it for the good parts. Rating: 5 out of 10

7 October 2013

Gregor Herrmann: using lintian profiles for fun & profit

since lintian gained support for vendor profiles, or at least since it has a nice documentation about writing checks & even a tutorial (the latter only local/in the package), I thought about writing some checks in a specific 'pkg-perl' profile. after talking to niels at debconf13, I tried in late august, & realized that at least simple checks are fun to write & really not difficult; luckily niels also reviewed my first attempts. & axel soon joined the fun & added some more checks. so here we are: the new pkg-perl-tools package also ships a lintian vendor profile 'pkg-perl' with a couple of simple but still useful checks that help to enforce our group policy. if you want to use them, install the package and check /usr/share/doc/pkg-perl-tools/README.lintian, if you want to look at the checks, you can also clone the git repo or inspect it in gitweb (directory lintian).

Michal &#268;iha&#345;: First day of Hackweek

First day of hackweek did not really go as planned. I had to spend too much time on tasks which I did not now I will have to do in advance. I've started with releasing Weblate 1.7. As a part of that I update SUSE Studio images with Weblate. Unfortunately this turned out to be more challenging than usual as with new version the build script always failed but without any obvious error. After some time, I've realized that it just does not show whole output, so the last (and most important) lines are missing. Then it was easy to spot that the problem is in Whoosh version and it's just matter of providing newer version. After fixing the image, I could finally focus on Weblate's issue tracker and resolved few issues and questions. I've also improved some of the in application documentation and links to documentation or other resources. Meanwhile I could also setup translation hosting for monkeysign, happy translating!

Filed under: English SUSE Weblate 0 comments Flattr this!

3 October 2013

Lior Kaplan: freedom 0

The license says you can use free and open source software for any reason, but the FBI wouldn t agree with your questions Silk Road operator identified by asking a PHP related question on stackoverflow. See The Bonehead Mistake That Brought Down an Online Drug-Dealing Empire on Slate.com
Filed under: Uncategorized

2 October 2013

Joey Hess: insured

Here in the US, the Affordable Care Act is finally going into effect, with accompanying drama. I managed to get signed up today at healthcare.gov. After not having health insurance since 2000, I will finally be covered starting January 1 2014. Since my income is mosty publically known anyway, I thought it might be helpful to blog about some details. I was uninsured for 14 years due to a combination of three factors:
  1. Initially, youthful stupidity and/or a perfectly resonable cost/benefit analysis. (Take your pick.)
  2. Due to the US health insurance system being obviously broken, and my preference to avoid things that are broken. Especially when the breakage involved insurers refusing to cover me at any sane level due to a minor and easily controlled pre-existing condition.
  3. Since I'm not much motivated by income levels, and am very motivated to have time to work on things that are important to me, my income has been on average pretty low, and perhaps more importantly, I have intentionally avoided being a full-time employee of anyone at any point in the past 14 years (have rejected job offers), and so was not eligible for any employee plans which were the only reasonable way to be covered in the US. (See point #2.)
So, if you're stuck waiting in line on healthcare.gov (is this an entirely new online experience brought to us by the US government?), or have seen any of the dozen or so failure modes that I saw just trying to register for a login to the site, yeah, it's massively overloaded right now, and it's also quite broken on a number of technical levels. But you can eventually get though it. Based on some of the bugs I saw, it may help to have an large number of email addresses and use a different one for each application attempt. It also wouldn't hurt to write some programs to automate the attempts, because otherwise you may have to fill out the same form dozens of times. And no, you can't use "you+foo@bar.com" for your email; despite funding the development of RFC-822 in the 80's, the US government is clueless about what consititutes a valid email address. But my "favorite" misfeature of the site is that it refuses to let you enter any accented characters, or characters not in the latin alphabet when signing up. Even if they're, you know, part of your name. (Welcome back to Ellis Island..) I want to check the git repository to see if I can find the backstory for these and other interesting technical decisions, but they have forgotten to push anything to it for over 3 months. The good news is that once you get past the initial signup process, and assuming you get the confirmation mail before the really short expiration period of apparently less than 1 hour (another interesting technical choice, given things like greylisting), the actual exchange is not badly overloaded, and nor is it very buggy (comparatively). I was able to complete an application in about an hour. The irony is that after all that, I was only able to choose from one health insurer covering my area on the so-called "exchange". (Blue Cross/Blue Shield) I also signed up for dental insurance (it was a welcome surprise that the site offers this at all) and had a choice of two insurers for that. The application process was made more uncertian for me since I have no idea what I'll end up doing for money once my current crowdsourced year of work is done. The site wants you to know how much income you'll have in 2014, and my guess is anywhere between $6000 (from a rental property) and about what I made this year (approx $25000 before taxes). Or up, if I say, answered the Google pings. The best choice seemed to be to answer what I made this year, which is also close to what I made last year. So, I'll be paying around $200/month for a combination of not very good health insurance, and not very good dental insurance. There is around $750/year of financial aid to people at my guesstimated 2014 income level, which would drop that to $140/month, but I will let them refund me whatever that turns out to be in a lump sum later instead. For comparison, I am lucky to spend rather less renting a three bedroom house situated in 25 acres of woods.. It's strange to think that all of this is an improvement to the system here in the US, especially given all the better options that could have been passed instead, but it seems that it probably is. Especially when I consider the many people around me who are less fortunate than myself. If you'd like a broader perspective on this, see Tobias Buckell's "American healthcare was already socialized by Reagan, we re just fighting about how to pay for it".

1 October 2013

Gunnar Wolf: Piracy and culture circulation: #encirc13

This week's lesson on the Arte y cultura en circulaci n: crear y compartir en tiempos digitales course talks about piracy and the circulation of culture, a topic that over time has been debated over and over. And a topic, yes, that can always lead to interesting discussions. This time, we are requested to choose one among ten ideas among the media groups' discourse on what piracy is and means for the "cultural industry". There are tons of material written already on several of those ten lines (i.e. piracy disincentivates creativity, or two that can be seen as two faces of the same argument, If a consumer can have free access to cultural products, he will stop spending his money on them and Every time a consumer has access to an illegal copy, the industry loses a sale), and some are quite obvious (i.e. Piracy makes job positions be lost... Just look at the amount of people the unauthorized distribution industry feeds! Or possibly, Piracy is a prosper industry that gives money to people distributing illegal products Of course, that is true. The problem is, what causes said products to be illegal to begin with? Should they be so?). Some other ideas talk about harsher penalties and ways to punish illegal copying in order to drive actors out of that sector (and into the... void?) So, I chose item #4: Cultural products have a high cost because their production is complex (and a tag could be made, linking complex with expensive). I think this item can lead to a long discussion as to what does this complexity and cost mean. Some cultural products do require quite a bit of investment, yes. Others don't. How do content producers make the jump to produce expensive works? If I am a new programmer/artist/writer/screenplayer/whatever, most likely, my products will be not very complex or expensive. I will start small. And if I excel at my work, somebody will look at me and, in some way, become my patron, my sponsor. Being a sponsor might mean that, based on the results of my good work, I could get hired as a software developer at a large company, or an editorial company would buy the patrimonial rights of my book/music (be it for a fixed fee or for a percentage of sales), or whatever. But the leap is not made quantically A newcomer to the cultural scene will at first, most likely, have a hard time selling his products. At first, it takes convincing just getting people to take a shot at looking at your work ( Hey, please take a look at my program and tell me what you think about it! , Would you be interested in listening to my latest song? And those two are by far ahead of the first attempts where the interactions would more likely be Turn off that $#^#!^ computer, it's well past bedtime or stop murdering that guitar, I'm having a headache ). Maybe the toughest part is to get people to agree to read/hear your work. And there, you start into a continuum Selling your CDs while performing on the street, then getting to play to a bar, then getting somebody to want to produce (maybe even "discover"!) you. Publish some short stories in your school magazine, then in a "From our audience" section in a larger magazine, then a collective book, your self-published book, yet-unwritten books by contract... The same story over and over again, in each different field. Ok, yes, but... This logic succession still leaves space for the Important Producers with the Mighty Big Pockets for the most wanted/largest productions, right? And were unauthorized distribution (piracy) to be the norm (as it currently is, dare I say), wouldn't they stop producing an important portion of cultural works? I'd be tempted to say so. However, a different actor comes into play. When Mighty Big Pockets comes into play, they no longer worry only about getting money from each cultural creation, but from all derived uses of it. And the cultural creation industry (when seen as an industry) goes very much hand in hand with the advertising, marketing industries They end up blending with each other. So, the biggest best sellers will most likely have a hit from illegal copiers. Books are still a great business, but hey An even better business is (usually) movie making. And when you make a movie out of a great story, you will surely link some advertising into it (or at the very least, push advertising/product pushing campaigns to go after it). And there, illegal distribution actually helps the money circle to grow stronger. In the early 1990s, the link between dinosaurs and carbonated drinks was a top seller (because Pepsi was a Jurassic Park sponsor). Although I have always loathed the madness around the World Cups (and basically anything that involves football of any kind), I can perfectly remember several of the theme songs for most of the world cups played during my lifetime. So, in short... No. Illegal distribution does marginally little harm to the money income to the cultural business, at any level. And where it does get some direct harm, it increases the money flux given the auxiliary channels.

29 September 2013

Joachim Breitner: Heidelberg Laureates Forum 2013

During the last week I was attending the first Heidelberg Laureates Forum as one of the lucky 200 accepted young scientists. The HFL is a (from now on hopefully yearly) event that brings together Fields Medalists, Abel Prize laureates and Turing Award winners with young scientists (undergraduates, Ph.D. students and postdocs) from both fields in the city of Heidelberg. The extremely well organized week consisted of lectures from the laureates, some workshops held by postdocs, excursions and plenty of good food. Videos of the lectures are available (but don t work on Linux, at least not for me ), and I have shot a few pictures of the event as well. I believe that my favourite lectures where Michael Atiyah s Advice to a Young Mathematician , Vladimir Voevodsky s Univalent Foundations of Mathematics , William Morton Kahan s Desperately Needed Remedies for the Undebuggability of Large-Scale Floating-Point Computations in Science and Engineering and Alan Kay s Putting Turing to Work . Where are all the functional programmers? During that event, one gets to talk to many other math and computer scientists researchers; sometimes just Where are you from? and What do you do? , sometimes long discussions. Unfortunately, I hardly found one who is into functional programming language research is that only because the event was parallel to ICFP (which I really would have liked to attend as well), or is functional programming really just about 1% of all computer science? What is a proof? My other research interest lies in interactive theorem proving, especially using Isabelle. Of course that is a topic that one can discuss with almost everyone at that event, including the mathematicians. The reactions were rather mixed: On the one end of the spectrum, some mathematicians seriously doubt that they would ever trust a computer to check proofs and that it would ever be efficient enough to use. Others would not mind having a button that tells whether their paper written in LaTeX is correct , but were not keen to invest time or thought into making the proof readable by the computer. And then there were some (but very few!) who had not heard of theorem proving before and were very excited by the prospect of being able to obtain certainty about their proofs immediately and without having to bother other scientists with it. During the mathematician s panel discussions, where I posed the question Do you see value in or even a need for machine-checked proofs in mathematics. , Efim Zelmanov (Fields Medal 1994) said a proof is what other mathematicians see as a proof . I found this attitude a bit surprising for me, a proof has always been a rigorous derivation within a formal system (say, ZFC set theory), and what we write in papers is a (less formal) description of the actual proof, whose existence we believe in. Therefore I was very happy to see Vladimir Voevodsky give a very committed talk about Univalent Foundations and how using that as the language for mathematics will allow more of mathematics to be cast in a formal, machine-checked form. I got the chance to discuss this with him in person, as I wanted to hear his option on Isabelle, and especially on the usefulness of the style of structured proof that Isar provides, and which is closer to the style of proofs that mathematicians use in papers. He said that he enjoyed writing his proof in the style required in Type Theory and in Coq, and that maybe mathematicians should and will adjust to the language of the system, while I believe that a structured proof languages like Isar, independent of the underlying logic (HOL in this case; which is insufficient to form a base for all of abstract mathematics), is a very useful feature and that proof assistants should adjust to the mathematicians.
We also briefly discussed the idea of mine to work with theorem provers already with motivated students in high schools, e.g. in math clubs, and found that simple proofs about arithmetic of natural numbers could be feasible here, without being too trivial. All in all a very rewarding and special week, and I can only recommend to try to attend one of the next forums, if possible.

28 September 2013

Jelmer Vernooij: Book Review: Bazaar Version Control

Packt recently published a book on Version Control using Bazaar written by Janos Gyerik. I was curious what the book was like, and they kindly provided me with a digital copy. The book is split into roughly five sections: an introduction to version control using Bazaar's main commands, an overview of the available workflows, some chapters on the available extensions and integration, some more advanced topics and finally, a quick introduction to programming using bzrlib. It is assumed the reader has no pre-existing knowledge about version control systems. The first chapters introduce the reader to the concept of revision history, branching and merging and finally collaboration. All concepts are first discussed in theory, and then demonstrated using the Bazaar command-line UI and the bzr-explorer tool. The book follows roughly the same track as the official documentation, but it is more extensive and has more fancy drawings of revision graphs. The middle section of the book discusses the modes in which Bazaar can be used - centralized or decentralized - as well as the various ways in which code can be landed in the main branch ("workflows"). The selection of workflows in the book is roughly the same as those in the official Bazaar documentation. The author briefly touches on a number of other software engineering topics such as code reviews, code formatting and automated testing, though not sufficiently to make it useful for people who are unfamiliar with these techniques. Both the official documentation and the book complicate things unnecessarily by listing every possible option. The next chapter is a basic howto on the use of Bazaar with various hosting solutions, such as Launchpad, Redmine and Trac. The Advanced Features chapter covers a wide range of obscure and less obscure features in Bazaar: uncommit, shelves, re-using working trees, lightweight checkouts, stacked branches, signing revisions and using e-mail hooks. The chapter on foreign version control system integration is a more extensive version of the public docs. It has some factual inaccuracies; in particular, it recommends the installation of a 2 year old buggy version of bzr-git. The last chapter provides quite a good introduction to the Bazaar APIs and plugin writing. It is a fair bit better than what is available publically. Overall, it's not a bad book but also not a huge step forward from the official documentation. I might recommend it to people who are interested in learning Bazaar and who do not have any experience with version control yet. Those who are already familiar with Bazaar or another version control system will not find much new. The book misses an opportunity by following the official documentation so closely. It has the same omissions and the same overemphasis on describing every possible feature. I had hoped to read more about Bazaar's data model, its file format and some of the common problems, such as parallel imports, format hell and slowness.

27 September 2013

Erich Schubert: Big Data madness and reality

"Big Data" has been hyped a lot, and due to this now is burning down to get some reality checking. It's been already overdue, but it is now happening.
I have seen a lot of people be very excited about NoSQL databases; and these databases do have their use cases. However, you just cannot put everything into NoSQL mode and expect it to just work. That is why we have recently seen NewSQL databases, query languages for NoSQL databases etc. - and in fact, they all move towards relational database management again.
Sound new database systems seem to be mostly made up of three aspects: in-memory optimization instead of optimizing for on-disk operation (memory just has become a lot cheaper the last 10 years), the execution of queries on the servers that store the actual data (you may want to call this "stored procedures" if you like SQL, or "map-reduce" if you like buzzwords), and optimized memory layouts (many of the benefits of "colum store" databases come from having a single, often primitive, datatype for the full column to scan, instead of alternating datatypes in records.

However, here is one point you need to consider:
is your data actually this "big"? Big as in: Google scale.
I see people use big data and Hadoop a lot when they just shouldn't. I see a lot of people run Hadoop in a VM on their laptop. Ouch.
The big data technologies are not a one-size-fits-all solution. They are the supersize-me solution, and supersize just does not fit every task.
When you look at the cases where Hadoop is really successful, it is mostly in keeping the original raw data, and enabling people to re-scan the full data again when e.g. their requirements changed. This is where Hadoop is really good at: managing 100 TB of data, and allowing you to quickly filter out the few GB that you really need for your current task.
For the actual analysis - or when you don't have 100 TB, and a large cluster anyway - then just don't try to hadoopify everything.
Here is a raw number from a current experiment. I have a student work on indexing image data; he is implementing some state of the art techniques. For these, a large number of image features are extracted, and then clustering is used to find "typical" features to improve image search.
The benchmarking image set is 56 GB (I have others with 2 TB to use next). The subset the student is currently processing is 13 GB. Extracting 2.3 million 128 dimensional feature vectors reduces this to about 1.4 GB. As you can see, the numbers drop quickly.
State of the art seems to be to load the data into Hadoop, and run clustering (actually, this is more of a vector quantization than clustering) into 1000 groups. Mahout is the obvious candidate to run this on Hadoop.
However, as I've put a lot of effort into the data mining software ELKI, I considered also to try processing it in ELKI.
By cutting the data into 10 MB blocks, Mahout/Hadoop can run the clustering in 52x parallel mappers. k-Means is an iterative algorithm, so it needs multiple processes. I have fixed the number of iterations to 10, which should produce a good enough approximation for my use cases.
K-means is embarrassingly parallel, so one would expect the cluster to really shine at this task. Well, here are some numbers:

So what is happening? Why is ELKI beating Mahout by a factor of 10x?
It's (as always) a mixture of a number of things:
Let me emphasize this: I'm not saying, Hadoop/Mahout is bad. I'm saying: this data set is not big enough to make Mahout beneficial.

Conclusions: As long as your data fits into your main memory and takes just a few hours to compute, don't go for Hadoop.
It will likely be faster on a single node by avoiding the overhead associated with (current) distributed implementations.
Sometimes, it may also be a solution to use the cluster only for preparing the data, then get it to a powerful workstation, and analyze it there. We did do this with the images: for extracting the image features, we used a distributed implementation (not on Hadoop, but on a dozen PCs).
I'm not saying it will stay that way. I have plans for starting "Project HELKI", aka "ELKI on Hadoop". Because I do sometimes hit the barrier of what I can compute on my 8 core CPU in an "reasonable" amount of time. And of course, Mahout will improve over time, and hopefully lose some of its "Java boxing" weight.
But before trying to run everything on a cluster, always try to run it on a single node first. You can still figure out how to scale up later, once you really know what you need.
And last but not least, consider whether scaling up really makes sense. K-means results don't really get better with a larger data set. They are averages, and adding more data will likely only change the last few digits. Now if you have an implementation that doesn't pay attention to numerical issues, you might end up with more numerical error than you gained from that extra data.
In fact, k-means can effectively be accelerated by processing a sample first, and only refining this on the full data set then. And: sampling is the most important "big data" approach. In particular when using statistical methods, consider whether more data can really be expected to improve your quality.
Better algorithms: K-means is a very crude heuristic. It optimizes the sum of squared deviations, which may be not too meaningful for your problem. It is not very noise tolerant, either. And there are thousands of variants. For example bisecting k-means, which no longer is embarrassingly parallel (i.e. it is not as easy to implement on MapReduce), but took just 7 minutes doing 20 iterations for each split. The algorithm can be summarized as starting with k=2, then always splitting the largest cluster in two until you have the desired k. For many use cases, this result will be just as good as the full k-means result.
Don't get me wrong. There are true big data problems. Web scale search, for example. Or product and friend recommendations at Facebook scale. But chances are that you don't have this amount of data. Google probably doesn't employ k-means at that scale either. (actually, Google runs k-means on 5 mio keypoints for vector quantization; which, judging my experience here, can still be done one a single host; in particular with hierarchical approaches such as bisecting k-means)
Don't choose the wrong tool for the wrong problem!

25 September 2013

Jon Dowland: Married

Jon and Sarah
I got married! Sarah and I tied the knot on the 10th August this year. We married in a registry office where we live and had an evening reception on Tynemouth beach. A fantastic time was had by all. We honeymooned in Sorrento, Italy, which was fantastic. We spent one day visiting Pompeii (my fourth visit: about 6 hours on our feet this time, and we walked from one side right to the other); another day in Herculaneum (my second visit); one day visiting the village near where we were staying, where we visited a local food festival and bought the nicest Marmelade I've ever tasted; a few day and evenings in Sorrento town itself; but most of the time we just relaxed, swam, snoozed, read.

23 September 2013

Daniel Pocock: PostBooks packages available

A common theme that comes up in free software evangelism is the question of how to tap into the huge expenditure that small to medium-sized businesses make for off-the-shelf licensed software. It is well known that many small companies are paying over the odds compared to the site-license rates given to big business, so why is it that small businesses are not rushing to adopt free and open source solutions? The answer is not likely to be found by fighting a battle on pricing (even though it is a battle that free software is inherently well prepared for). Rather, the most compelling strategy involves helping businesses find better ways to solve their own problems. A problem that every business encounters in every industry is the maintenance of proper financial records. Up until now, packaged solutions like GnuCash and HomeBank, available in many Linux distributions, have only catered for the smallest businesses and household accounting. At the other end of the spectrum, solutions like Compiere/Adempiere and OpenERP require a level of effort to install and maintain that makes them unsuitable for small users and businesses that don't need their full range of features. OpenERP briefly appeared as a Debian package but subsequently dropped out due to difficulties supporting the package. There is another choice, PostBooks. PostBooks has recently been packaged for Debian and this experience leads me to believe it will be supportable in Fedora and other distributions too. PostBooks, which started off as a commercial product, was re-licensed as an open source solution in 2007. In March 2013, PostBooks was project of the month on Sourceforge. Of particular note,
  • PostBooks has a Qt-based GUI where users of equivalent commercial solutions are likely to feel comfortable very quickly
  • PostBooks does not depend on any dedicated server processes other than a PostgreSQL database
  • Using the quickstart database, it can be set up in less than half a day, making it suitable for many consultants and freelancers who may otherwise use a product having a setup wizard (such as QuickBooks).
  • However, with features like inventory, invoicing, a sophisticated data import GUI (known as CSVImp), a true SQL backend, multi-currency and multiple concurrent users, it scales beyond the use cases that Quickbooks and Sage Accounting are capable of servicing.
  • The company behind PostBooks, xTuple, is well established and offers related services such as training and an annual user conference (which takes place in just a few weeks). Many business users are likely to find this style of support very reassuring and familiar, even if they don't plan to immediately take advantage of these things.
Limitations Two particular limitations caught my attention early when evaluating PostBooks: The developers have acknowledged these limitations and offered to complete these solutions with a crowd-funding campaign dubbed Feature Mob 2.

While everybody likes a free lunch, I personally feel that developers are perfectly entitled to request payment for their work. Having already taken a great leap of faith in open-sourcing a product that had previously been developed in-house, companies like xTuple need to look at new ways to generate revenue. Having looked at some of the prices of equivalent solutions, I felt quite comfortable pledging a similar amount based on four things:
  • it is based on the Kickstarter model, they only keep the money if their funding target is reached and feature implementation goes ahead
  • they are releasing the work as open source
  • the desire to have the option of using a highly capable free software solution that will be around well into the future
  • the probability that this is one piece of software that is likely to create more commercial opportunities for free software developers
Next steps People trying the packages and people who potentially want to package PostBooks for other distributions are very welcome to come and ask questions in the xTuple Debian maintainers packaging list or using the xTuple web forums. Please also feel free to report bugs through the Debian bug tracker.

17 September 2013

Julien Danjou: Python 3.4 single dispatch, a step into generic functions

I love to say that Python is a nice subset of Lisp, and I discover that it's getting even more true as time passes. Recently, I've stumbled upon the PEP 443 that describes a way to dispatch generic functions, in a way that looks like what CLOS, the Common Lisp Object System, provides. What are generic functions If you come from the Lisp world, this won't be something new to you. The Lisp object system provides a really good way to define and handle method dispatching. It's a base of the Common Lisp object system. For my own pleasure to see Lisp code in a Python post, I'll show you how generic methods work in Lisp first. To begin, let's define a few very simple classes.
(defclass snare-drum ()
())

(defclass cymbal ()
())

(defclass stick ()
())

(defclass brushes ()
())

This defines a few classes: snare-drum, symbal, stick and brushes, without any parent class nor attribute. These classes compose a drum kit, and we can combine them to play sound. So we define a play method that takes two arguments, and returns a sound (as a string).
(defgeneric play (instrument accessory)
(:documentation "Play sound with instrument and accessory."))

This only defines a generic method: it has no body, and cannot be called with any instance yet. At this stage, we only inform the object system that the method is generic and can be then implemented with various type of arguments. We'll start by implementing versions of this method that knows how to play with the snare-drum.
(defmethod play ((instrument snare-drum) (accessory stick))
"POC!")

(defmethod play ((instrument snare-drum) (accessory brushes))
"SHHHH!")

Now we just defined concrete methods with code. They also takes two arguments: instrument which is an instance of snare-drum and accessory that is an instance of stick or brushes. At this stage, you should note the first difference with object system as built into language like Python: the method isn't tied to any class in particular. The methods are generic, and any class can implement them, or not. Let's try it.
* (play (make-instance 'snare-drum) (make-instance 'stick))
"POC!"

* (play (make-instance 'snare-drum) (make-instance 'brushes))
"SHHHH!"

* (play (make-instance 'cymbal) (make-instance 'stick))
debugger invoked on a SIMPLE-ERROR in thread
#<THREAD "main thread" RUNNING 1002ADAF23 >:
There is no applicable method for the generic function
#<STANDARD-GENERIC-FUNCTION PLAY (2)>
when called with arguments
(#<CYMBAL 1002B801D3 > #<STICK 1002B82763 >).

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
0: [RETRY] Retry calling the generic function.
1: [ABORT] Exit debugger, returning to top level.

((:METHOD NO-APPLICABLE-METHOD (T)) #<STANDARD-GENERIC-FUNCTION PLAY (2)> #<CYMBAL 1002B801D3 > #<STICK 1002B82763 >) [fast-method]

As you see, the function called depends on the class of the arguments. The object systems dispatch the function calls to the right function for us, depending on the arguments classes. If we call play with instances that are not know to the object system, an error will be thrown. Inheritance is also supported and the equivalent (but more powerful and less error prone) equivalent of Python's super() is available via (call-next-method).
(defclass snare-drum () ())
(defclass cymbal () ())

(defclass accessory () ())
(defclass stick (accessory) ())
(defclass brushes (accessory) ())

(defmethod play ((c cymbal) (a accessory))
"BIIING!")

(defmethod play ((c cymbal) (b brushes))
(concatenate 'string "SSHHHH!" (call-next-method)))

In this example, we define the stick and brushes classes as subclass of the accessory class. The play method defined will return the sound BIIING! regardless of the accessory instance that is used to play the cymbal. Except in the case where it's a brushes instance; only the most precise method is always called. The (call-next-method) function is used to call the closest parent method, in this case that would be the method returning _"BIIING!".
* (play (make-instance 'cymbal) (make-instance 'stick))
"BIIING!"

* (play (make-instance 'cymbal) (make-instance 'brushes))
"SSHHHH!BIIING!"

Note that CLOS is also able to dispatch on object instances themself by using the eql specializer. But if you're really curious about all features CLOS provides, I suggest you read the brief guide to CLOS by Jeff Dalton as a starter. Python implementation Python implements a simpler equivalence of this workflow with the singledispatch function. It will be provided with Python 3.4 as part of the functools module. Here's a rough equivalence of the above Lisp program.
import functools

class SnareDrum(object): pass
class Cymbal(object): pass
class Stick(object): pass
class Brushes(object): pass

@functools.singledispatch
def play(instrument, accessory):
raise NotImplementedError("Cannot play these")

@play.register(SnareDrum)
def _(instrument, accessory):
if isinstance(accessory, Stick):
return "POC!"
if isinstance(accessory, Brushes):
return "SHHHH!"
raise NotImplementedError("Cannot play these")

We define our four classes, and a base play function that raises NotImplementedError, indicating that by default we don't know what to do. We can then write specialized version of this function with a first instrument, the SnareDrum. We then check for the accessory type that we get, and return the appropriate sound or raise NotImplementedError again if we don't know what to do with it. If we run it, it works as expected:
>>> play(SnareDrum(), Stick())
'POC!'
>>> play(SnareDrum(), Brushes())
'SHHHH!'
>>> play(Cymbal(), Brushes())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/jd/Source/cpython/Lib/functools.py", line 562, in wrapper
return dispatch(args[0].__class__)(*args, **kw)
File "/home/jd/sd.py", line 10, in play
raise NotImplementedError("Cannot play these")
NotImplementedError: Cannot play these
>>> play(SnareDrum(), Cymbal())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/jd/Source/cpython/Lib/functools.py", line 562, in wrapper
return dispatch(args[0].__class__)(*args, **kw)
File "/home/jd/sd.py", line 18, in _
raise NotImplementedError("Cannot play these")
NotImplementedError: Cannot play these

The singledispatch module looks through the classes of the first argument passed to the play function, and calls the right version of it. The first defined version of the play function is always run for the object class, so if our instrument is a class that we did not register for, this base function will be called. For whose eager to try and use it, the singledispatch function is provided Python 2.6 to 3.3 through the Python Package Index. Limitations First, as you noticed in the Lisp version, CLOS provides a multiple dispatcher that can dispatch on the type of any of the argument defined in the method prototype, not only the first one. Unfortunately, Python dispatcher is named singledispatch for this good reason: it only knows to dispatch on the first argument. Guido van Rossum wrote a short article about the subject that he called multimethod a few years ago. Then, there's no way to call the parent function directly. There's no equivalent of the (call-next-method) from Lisp nor the super() function that allows to do that in Python class system. This means you will have to use various trick to bypass this limitation. So while I am really glad that Python is going toward that direction, as it's a really powerful way to enhance an object system, it really lacks a lot of more advanced features that CLOS provides out of the box. Though, improving this could be an interesting challenge. Especially to bring more CLOS power to Hy. :-)
The Hacker Guide to Python
A book I'm writing that will be launched soon, talking about designing Python applications, state of the art, and various Python tips. If you want to be the first to hear about the launch, subscribe now.

Next.

Previous.