Search Results: "nate"

1 May 2013

Raphaël Hertzog: My Free Software Activities in April 2013

This is my monthly summary of my free software related activities. If you re among the people who made a donation to support my work (102.70 , thanks everybody!), then you can learn how I spent your money. Otherwise it s just an interesting status update on my various projects. Debian France Work on Galette. I spent quite some time on Debian France s galette installation (the web application handling its member database), first converting its Postgres database to UTF-8, then upgrading to 0.7.4 while working-around many known problems. I also created Debian packages of three Galette plugins that we have been using (galette-plugin-paypal, galette-plugin-admintools, galette-plugin-fullcard). But every time I use galette, I tend to find something to report. This month I filed 5 tickets: I tested quite some fixes prepared by the upstream author (3 of the above bugs are already fixed) and this lead to the 0.7.4.1 bugfix release. Preliminary work on new bylaws. I have setup a git repository to make it easier to collaborate on new versions of our bylaws and internal rules. The goal is to make Debian France a trusted organization of Debian and to update everything to be compliant with the association 1901 law (we currently have a special statute reserved to associations from Alsace/Moselle). Kali Linux Improve accessibility support in Debian Wheezy. Offensive Security wanted Kali Linux to be fully accessible to disabled people. Since Wheezy was suffering from some serious regressions in that area, we hired Emilio Pozuelo Monfort to fix #680636 and #689559 in gdm3. On my side, I updated debian-installer s finish-install to correctly pre-configure the system when you make an install with speech synthesis (patch submitted in #705599).
Thanks to accommodating release managers, this work has already been integrated in Wheezy and won t have to wait the first point release. Fix bugs in Debian s live desktop installer. We also wanted to enable the desktop installer in the Kali live DVD. While our first tries a few months ago failed, this time it worked almost out of the box (thanks to Ben Armstrong who fixed it). I still identified a few issues that I fixed in debian-installer-launcher s git repository. Packaging and misc Debian work I also spent a significant number of hours to answer questions of students who want to participate in Google s summer of code and who are interested by the rewrite of the Package Tracking System with Python and Django. Some of the discussions happened on debian-qa@lists.debian.org. Thanks See you next month for a new summary of my activities.

One comment Liked this article? Click here. My blog is Flattr-enabled.

Antoine Beaupr : New Debian release: my contributions to the #newinwheezy game

So while I haven't work exceptionally hard to help with the Debian Wheezy release (I merely help with a few RC bugs and organised a bug squashing party), I still have a few significant contributions to the release. So here's my claim to fame to the #newinwheezy game:

26 April 2013

Keith Packard: Shared Memory Fences

Shared Memory Fences In our last adventure, dri3k first steps, one of the future work items was to deal with synchronization between the direct rendering application and the X server. DRI2 handles this by performing a round trip each time the application starts using a buffer that was being used by the X server. As DRI3 manages buffer allocation within the application, there s really no reason to talk to the server, so this implicit serialization point just isn t available to us. As I mentioned last time, James Jones and Aaron Plattner added an explicit GPU serialization system to the Sync extension. These SyncFences serializing rendering between two X clients, but within the server there are hooks provided for the driver to use hardware-specific serialization primitives. The existing Linux DRM interfaces queue rendering to the GPU in the order requests are made to the kernel, so we don t need the ability to serialize within the GPU, we just need to serialize requests to the kernel. Simple CPU-based serialization gating access to the GPU will suffice here, at least for the current set of drivers. GPU access which is not mediated by the kernel will presumably require serialization that involves the GPU itself. We ll leave that for a future adventure though; the goal today is to build something that works with the current Linux DRM interfaces. SyncFence Semantics The semantics required by SyncFences is for multiple clients to block on a fence which a single client then triggers. All of the blocked clients start executing requests immediately after the trigger fires. There are four basic operations on SyncFences: SyncFences are the same as Events as provided by Python and other systems. Of course all of the names have been changed to keep things interesting. I ll call them Fences here, to be consistent with the current X usage. Using Pthread Primitives One fact about pthreads that I recently learned is that the synchronization primitives (mutexes, barriers and semaphores) are actually supposed to work across process boundaries, if those objects are in shared memory mapped by each process. That seemed like a great simplification for this project; allocate a page of shared memory, map into the X server and direct rendering application and use the existing pthreads APIs. Alas, the pthread objects are architecture specific. I m pretty sure that when that spec was written, no-one ever thought of running multiple architectures within the same memory space. I went and looked at the code to check, and found that each of these objects has a different size and structure on x86 and x86_64 architectures. That makes it pretty hard to use this API within X as we often have both 32- and 64- bit applications talking to the same (presumably 64-bit) X server. As a last resort, I read through a bunch of articles on using futexes directly within applications and decided that it was probably possible to implement what I needed in an architecture-independent fashion. Futexes Linux Futexes live in this strange limbo of being a not-quite-public kernel interface. Glibc uses them internally to implement locking primitives, but it doesn t export any direct interface to the system call. Certainly they re easy to use incorrectly, but it s unusual in the Linux space to have our fundamental tools locked away for our own safety . Fortunately, we can still get at futexes by creating our own syscall wrappers.
static inline long sys_futex(void *addr1, int op, int val1,
                 struct timespec *timeout, void *addr2, int val3)
 
    return syscall(SYS_futex, addr1, op, val1, timeout, addr2, val3);
 
For this little exercise, I created two simple wrappers, one to block on a futex:
static inline int futex_wait(int32_t *addr, int32_t value)  
    return sys_futex(addr, FUTEX_WAIT, value, NULL, NULL, 0);
 
and one to wake up all futex waiters:
static inline int futex_wake(int32_t *addr)  
    return sys_futex(addr, FUTEX_WAKE, MAXINT, NULL, NULL, 0);
 
Atomic Memory Operations I need atomic memory operations to keep separate cores from seeing different values of the fence value, GCC defines a few such primitives and I picked _syncboolcompareandswap and _syncvalcompareandswap. I also need fetch and store operations that the compiler won t shuffle around:
#define barrier() __asm__ __volatile__("": : :"memory")
static inline void atomic_store(int32_t *f, int32_t v)
 
    barrier();
    *f = v;
    barrier();
 
static inline int32_t atomic_fetch(int32_t *a)
 
    int32_t v;
    barrier();
    v = *a;
    barrier();
    return v;
 
If your machine doesn t make these two operations atomic, then you would redefine these as needed. Futex-based Fences These wake-all semantics of Fences greatly simplify reasoning about the operation as there s no need to ensure that only a single thread runs past Await, the only requirement is that no threads pass the Await operation until the fence is triggered. A Fence is defined by a single 32-bit integer which can take one of three values: With those, I built the fence operations as follows. Here s Await:
int fence_await(int32_t *f)
 
    while (__sync_val_compare_and_swap(f, 0, -1) != 1)  
        if (futex_wait(f, -1))  
            if (errno != EWOULDBLOCK)
                return -1;
         
     
    return 0;
 
The basic requirement that the thread not run until the fence is triggered is met by fetching the current value of the fence and comparing it with 1. Until it is signaled, that comparison will return false. The compareandswap operation makes sure the fence is -1 before the thread calls futex_wait, either it was already -1 in the case where there were other waiters, or it was 0 before and is now -1 in the case where there were no waiters before. This needs to be an atomic operation so that the fence value will be seen as -1 by the trigger operation if there are any threads in the syscall. The futex_wait call will return once the value is no longer -1, it also ensures that the thread won t block if the trigger occurs between the swap and the syscall. Here s the Trigger function:
int fence_trigger(int32_t *f)
 
    if (__sync_val_compare_and_swap(f, 0, 1) == -1)  
        atomic_store(f, 1);
        if (futex_wake(f) < 0)
            return -1;
     
    return 0;
 
The atomic compareandswap operation will make sure that no Await thread swaps the 0 for a -1 while the trigger is changing the value from 0 to 1; either the Await switches from 0 to -1 or the Trigger switches from 0 to 1. If the value before the compareandswap was -1, then there may be threads waiting on the Fence. An atomic store, constructed with two memory barriers and a regular store operation, to mark the Fence triggered is followed by the futex_wake call to unblock all Awaiting threads. The Query function is just an atomic fetch:
int fence_query(int32_t *f)
 
    return atomic_fetch(f) == 1;
 
Reset requires a compareandswap so that it doesn t disturb things if the fence has already been reset and there are threads waiting on it:
void fence_reset(int32_t *f)
 
    __sync_bool_compare_and_swap(f, 1, 0);
 
A Request for Review Ok, so we ve all tried to create synchronization primitives only to find that our obvious implementations were full of holes. I d love to hear from you if you ve identified any problems in the above code, or if you can figure out how to use the existing glibc primitives for this operation.

Vincent Sanders: When you make something, cleaning it out of structural debris is one of the most vital things you do.

Collabora recently had a problem with a project's ARM build farm. In a nice change of pace it was not that the kernel was crashing, nor indeed any of the software or hardware.
The ProblemInstead our problem was our build farm could best be described as "a pile of stuff" and we wanted to add more systems to it and have switched power control for automated testing.

Which is kinda where the Christopher Alexander quote comes into this. I suggested that I might be able to come up with a better, or at least cleaner, solution.
The IdeaA subrack with sub modulesPrevious experience had exposed me to the idea of using 19 inch subracks for mounting circuits inside submodules.

I originally envisaged the dev boards individually mounted inside these boxes. However preliminary investigation revealed that the enclosures were both expensive and used a lot of space which would greatly increase the rack space required to house these systems.

imx53 QSB eurocard carrier
I decided to instead look at eurocard type subracks with carriers for the systems. Using my 3D printer I came up with a carrier design for the imx53 QSB and printed it. I used the basic eurocard size of 100mm x 160mm which would allow the cards to be used within a 3U subrack.

Once assembled it became apparent that each carrier would be able to share resources like power supply, ethernet port and serial console via USB just as the existing setup did and that these would need to be housed within the subrack.
The Prototype
The carrier prototype was enough to get enough interest to allow me to move on to the next phase of the project. I purchased a Schroff 24563-194 subrack kit and three packs of guide rails from Farnell and assembled it.

Initially I had envisaged acquiring additional horizontal rails from Schroff which would enable constructing an area suitable for mounting the shared components behind the card area.

Rear profile for Schroff subrackUnfortunately Schroff have no suitable horizontal profiles in their catalog and are another of those companies who seem to not want to actually sell products to end users but rather deal with wholesalers who do not have their entire product range!

Printed rear profile for Schroff subrack
Undaunted by this I created my own horizontal rail profile and 3D printed some lengths. The profile is designed to allow a 3mm thick rear cover sheet attached with M2.5 mounting bolts and fit rack sides in the same way the other profiles do.

At this point I should introduce some information on how these subracks are dimensioned. A standard 19 inch rack (as defined in IEC 60297) has a width of 17.75 inches(450.85mm) between the vertical posts. The height is measured in U (1.75 inches)

A subrack must obviously fit in the horizontal gap while providing as much internal space as possible. A subrack is generally either 3 or 6 U high. The width within a subrack is defined in units called HP (Horizontal Pitch) which are 0.2 inches(5.08 mm) and subracks like the Schroff generally list 84 usable HP.

However we must be careful (or actually just learn from me stuffing this up ;-) as the usable HP is not the same thing as the actual length of the horizontal rails! The enclosures actually leave and additional 0.1 inch at either end giving a total internal width of 85HP (17 inches, 431.8 mm) which leaves 0.75 inches for the subrack sides and some clearance.

The Schroff subrack allows eurocards to be slotted into rails where the card centre line is on HP boundaries, hence we describe the width of a card in the slot in terms of HP

I cannot manufacture aluminium extrusions (I know it is a personal failing) nor produce more than 100 mm long length of the plastic profile on my printer.

Even if full lengths are purchased from a commercial service (120 euros for a pair including tax and shipping) the plastic does not have sufficient mechanical strength.

The solution I came up with was somewhat innovative, as an alternative a M5 bolt into a thread in the aluminium extrusion I used a 444mm long length of 4mm threaded rod with nuts at either end. This arrangement puts the extrusion under compression and gives it a great deal of additional mechanical strength as the steel threaded rod is very strong.

Additionally to avoid having to print enough extrusion for the entire length I used some 6mm aluminium tube as a spacer between 6HP(30.48mm) wide sections of the printed extrusion.

It was intended to use a standard modular PC power supply which is 150mm wide which is pretty close to 30HP (6 inches) so it was decided to have a 6HP section of rail at that point to allow a rear mounting plate for the PSU to be attached.

This gives 6HP of profile, 21HP(106.68mm) of tube spacer, 6HP of profile, 46HP(233.68 mm) of tube spacer and a final 6HP profile summing to our total of 85HP. Of course this would be unnecessary if a full continuous 85HP rail had been purchased, but 6 of 6 HP long profile is only 51 euro a saving of 70 euro.

To provide a flat area on which to mount the power switching, Ethernet switch and USB hubs I ordered a 170 x 431 mm sheet of 3mm thick aluminium from inspiredsteel who, while being an ebay company, were fast, cheap and the cutting was accurate.

Do be sure to mention you would prefer it if any error made the sheet smaller rather than larger or it might not fit, for me though they were accurate to the tenth of a mm! If you would prefer the rear section of the rack to be enclosed when you are finished, buy a second sheet for the top. For my prototype I only purchased a 170 x 280mm sheet as I was unsure if I wanted a surface under the PSU (you do, buy the longer sheet)

PC power supply mounted to back plateMounting the PSU was a simple case of constructing a 3 mm thick plate with the correct cutouts and mounting holes for an ATX supply. Although the images show the PSU mounted on the left hand side of the rack this was later reversed to improve cable management.

The subrack needed to provide Ethernet switch ports to all the systems. A TP-Link TL-SF1016DS 16-Port 10/100Mbps Switch was acquired and the switch board removed from its enclosure. The switch selected has an easily removed board and is powered by a single 3.3V input which is readily available from the ATX PSU.

Attention now returned to the eurocard carriers for the systems, the boards to be housed were iMX53 QSB and iMX6 SABRE Lite and a Raspberry Pi control system to act as USB serial console etc.

The carriers for both main boards needed to be 8HP wide, comprised of:
Although only 38 mm this is 7.5HP and fractions of an HP are not possible with the selected subrack.

With 8HP wide modules this would allow for ten slots, within the 84 usable HP, and an eleventh 4HP wide in which the Raspberry Pi system fits.

iMX6 SABRE Lite eurocard carrierCarrier designs for both the i.MX53 QSB and the i.MX6 SABRE Lite boards were created and fabricated at a professional 3D print shop which gave a high quality finish product and removed the perceived risk of relying on a personal 3D printer for a quantity of parts.

This resulted in changes in the design to remove as much material as possible as commercial 3D services charge by the cubic cm. This Design For Manufacture (DFM) step removed almost 50% from the price of the initial design.

i.MX53 QSB carriers with wiring loom
The i.MX6 design underwent a second iteration to allow for the heatsink to be mounted and not mechanically interfere with the hard drive (although the prototype carrier has been used successfully for a system that does not require a hard drive). The lesson learned here is to be aware that an design iteration or two is likely and that it is not without cost.

The initial installation was to have six i.MX53 and two i.MX6 this later changed to a five/four split, however the carrier solution allows for almost any combination, the only caveat (discovered later) is the imx53 carriers should be to the right hand side with the small 4HP gap at that end as they have a JTAG connector underneath the board which otherwise foul the hard drive of the next carrier.

Racked cards showing unwanted cable tails
A wiring loom was constructed for each board giving them a connector tail long enough to allow them to be removed. This was the wrong approach! if you implement this design (or when I do it again) the connector tails on the wiring loom should present all the connections to the rear at the same depth as the Ethernet connection.

The rack cables themselves should be long enough to allow the slides to be removed but importantly it is not desirable to have the trailing cable on the cards. I guess the original eurocard designers figured this out as they designed the cards around the standard fixed DIN connectors at the back of the card slots.

USB relay board with wiring loom attached
We will now briefly examine a misjudgement that caused the initially deployed solution to be reimplemented. As the design was going to use USB serial converters to access the serial console a USB connected relay board was selected to switch the power to each slot. I had previously used serial controlled relay boards with a USB serial convertor however these were no longer available.

Initial deployment with USB controlled relay board
All the available USB relay boards were HID controlled, this did not initially seem to be an issue and Linux software was written to provide a reasonable interface. However it soon became apparent that the firmware on the purchased board was very buggy and crashed the host computer's USB stack multiple times.

Deployed solutionOnce it became apparent that the USB controlled power board was not viable a new design was conceived. As the Ethernet switch had ports available Ethernet controlled relay boards were acquired.

Evolution of 3mm PCB pillars
It did prove necessary to design and print some PCB support posts with M3 nut traps to allow the relay boards to be easily mounted using double sided adhesive pads.

By stacking the relay boards face to face and the Ethernet switch on top separated using nylon spacers it was possible to reduce the cable clutter and provide adequate cable routing space.

A busbar for Ground (black) and unswitched 12V (yellow) was constructed from two lengths of 5A chock block.

An issue with power supply stability was noted so a load resistor was added to the 12V supply and an adhesive thermal pad used to attach it to the aluminium base plate.

Completed redesign
It was most fortunate that the ethernet switch mounting holes lined up very well with the relay board mounting holes allowing for a neat stack.

This second edition is the one currently in use, it has proved reliable in operation and has been successfully updated with additional carriers.

The outstanding issues are mainly centered around the Raspberry Pi control board:
  • Needs its carrier fitting. It is currently just stuck to the subrack end plate.
  • Needs its Ethernet cable replacing. The existing one has developed a fault post installation.
  • Needs the USB hub supply separating from the device cable. The current arrangement lets the hub power the Pi which means you cannot power cycle it.
  • Connect its switched supply separately to the USB hub/devices.
Shopping listThe final bill of materials (excluding labour and workshop costs) which might be useful to anyone hoping to build their own version.

Prices are in GBP currency converted where appropriate and include tax at 20% and delivery to Cambridge UK and were correct as of April 2013.

The purchasing was not optimised and for example around 20GBP could be saved just by ordering all the shapeways parts in one order.
Base subrack
ItemSupplierQuantityLine Price
Schroff 24563-194 subrack kitFarnell141.28
Schroff 24560-351 guide railsFarnell313.65
Schroff rack rear horizontal railShapeways2100.00
1000mm length of 4mm threaded rodB and Q11.48
170mm x 431mm x 3mm Aluminium sheetinspired steel240.00
PSU mounting plateShapeways135.42
PCB standoffShapeways422.30
160mm Deep Modular PC supplyCCL155.76
TP-Link TL-SF1016DS 16-Port 10/100Mbps-SwitchCCL123.77
8 Channel 16A Relay Board Controlled Via EthernetRapid2126.00
Raspberry PiFarnell126.48
USB Serial convertersCCL1037.40
10 port strip style USB HUBEbay17.00
Parts for custom Ethernet cablesRS1326.00
Parts for custom molex power cables (salvaged from scrap ATX PSU)Workshop1111.00
33R 10W wirewound resistor for dummy loadRS11.26
24pin ATX female connector pre-wiredMaplin12.99
Akasa double sided thermal padMaplin15.00
Small cable tie basesMaplin16.49
Miscellaneous cable, connectors, nylon standoffs, solder, heatshrink, zip ties, nuts, washers etc. Workshop120.00
Total for subrack603.28

The carriers are similarly not optimally priced as over five GBP each can be saved by combining shipping on orders alone. Also the SSD drive selection was made some time ago and a newer model may be more suitable.
i.MX53 QSB carrier
ItemSupplierQuantityLine Price
i.MX53 QSBFarnell1105.52
Intel 320 SSD 80GCCL1111.83
Carrier boardShapeways130.00
combined sata data and power (15 to 20cm version)EBay15.00
Low profile right angle 5.5mm x 2.1mm barrel jackEBay10.25
Parts for 9pin serial cable extensionRS15.00
Miscellaneous solder, heatshrink, nylon nuts, bolts and washersWorkshop15.00
Total for carrier262.60

i.MX6 SABRE Lite carrier
ItemSupplierQuantityLine Price
i.MX6 SABRE LiteFarnell1128.06
Intel 320 SSD 80GCCL1111.83
Carrier boardShapeways135.00
combined sata data and power (15 to 20cm version)EBay15.00
Low profile right angle 5.5mm x 2.1mm barrel jackEBay10.25
Parts for 9pin serial cable modificationRS12.00
Miscellaneous solder, heatshrink, nylon nuts, bolts and washersWorkshop15.00
Total for carrier287.14
ConclusionThe solution works and in a 3U high 355mm deep subrack ten ARM development boards can be racked complete with local ethernet switching, power control and serial consoles.

Deployed system in situ configured as a build and test farm
The solution is neat and provides flexibility, density and reproducibility the "pile of stuff" solution failed to do.

For current prototype with nine filled slots the total cost was around 3000GBP or around 330GBP per slot which indicates a 100GBP per slot overhead over the "pile of stuff" solution. These figures omit the costs of the engineer and workshop time, which are estimated at an additional 1500GBP. Therefore a completed rack, fully filled with i.MX6 carriers costs around 5000GBP

Density could be increased if boards with lower height requirements were used however above twelve units there issues with Ethernet switch, power switch and USB port availability become a factor. For Example the 16 port Ethernet switch requires a port for uplink, one for each relay board and one for the console server which leaves only 12 ports for systems.

Addressing the outstanding issues would result in a much more user friendly solution. As the existing unit is in full time use and downtime is not easily scheduled for all ten systems, the issues are not likely to be fixed on the prototype and would have to be solved on a new build.

The solution is probably not suitable for turning into a product but that was not really the original aim. A commercial ARM blade server using this format would almost certainly use standard DIN connectors and a custom PCB design rather than adapting existing boards.

Steve Kemp: Modern console mail clients?

I've recently started staging upgrades from Squeeze to Wheezy. One unpleasant surprise was that the mutt-patched package available to Debian doesn't contain the "sidebar-new-only" patch. This means I need to maintain it myself again, which I'd rather avoid. Over time I've been slowly moving to standard Debian systems, trying to not carry too many local perversions around. Unfortunately if you've kept all your mail since 1994 you have many mailboxes. having mutt-patched available at all, with the sidebar patch, is a great timesaver. But I don't want to see mailboxes I'm never going to touch; just mailboxes with new mail in them. Also I find the idea of having to explicitly define mailboxes a pain. Just run inotify on ~/Maildir and discover the damn things yourself. Please computer, compute! If you divide up "mail client" into distinct steps it doesn't seem so hard: Obviously there is more to it than that. Sending mail? exec( sendmail ). Filtering mail? procmail/sieve/etc. Editing mail? exec(vim). I'm sure if I were to start a core of a program, suitable for myself, would be simple enough. Maybe with lua, maybe with javascript, but with a real language at the core. Anyway I've thought this before, and working with quilt and some ropy patches has always seemed like the way to go. Maybe it still is, but I can dream. (PS. Sup + Notmuch both crash on my archives. I do not wish to examine them further. Still some interesting ideas. It should be possible to say "maildirs are tags; view "~/Maildir/.livejournal.2003" and ~/Maildir/.livejournal.2007 at the same time. Why just a single directory in the "index-view? So 1994.) Disjointed posts R Us. Obquote: "How hard could it be?" -- Patrick.

Bits from Debian: Last days for the DebConf13 matching fund!

As part of the DebConf13 fundraising efforts, Brandorr Group is funding a matching initiative for DebConf13, which will be in place for 4 more days (through April 30th). You can donate here! Please consider donating $100, or even $5 or any amount in between, as we can use all the help we can get to reach our fundraising target. The rules are simple: This generous offer will only stay in place through the end of April 30th. Please act quickly, and help spread the world!

25 April 2013

Jeff Licquia: Linux Is Hard, Except When It Isn t

Online tech news site Ars Technica (which I recommend, by the way) recently reviewed the Dell XPS 13 Developer Edition. Its unique feature: it ships with Ubuntu Linux as the default operating system. This preload deal had a few unique properties: That last point deserves some explanation. A few vendors have grabbed a Windows computer they sell and allowed the option to preload Linux on it, but without support; you re on your own if it doesn t work in some way, which is likely. Essentially, they save you the time of wiping Windows off the box and doing a fresh install, but not much more. But this laptop comes out of Dell s Project Sputnik, a project to put out Linux machines for developers with a DevOps flavor, and they felt the machine had to work as well as their regular products. So they actually put effort and testing into getting the laptop to run Ubuntu well, with all the drivers configured properly and tweaked to support the machine s quirks, just like they do for Windows. And so, the review is surprised to learn that Ubuntu on the XPS 13, well, just works! It s even in the title of the review. Here s reviewer Lee Hutchinson s observations:
I ve struggled before with using Linux as my full-time operating environment both at work and at home. I did it for years at work, but it was never quite as easy as I wanted it to be on an older Dell laptop, keeping dual monitor support working correctly across updates required endless fiddling with xorg.conf, and whether or not it was Nvidia s fault was totally irrelevant to swearing, cursing Past Lee, trying desperately to get his monitors to display images so he could make his 10am conference call without having to resort to running the meeting on the small laptop screen.
And thence comes the astonishment: on this Linux laptop, everything just works. Most of the review is spent on the kinds of hardware features that distinguish this from other laptops: the keyboard is like this, the screen is that resolution, it has this CPU and this much RAM and so on. Some space is devoted to impressions of the default Ubuntu 12.04 install, and some space is given to the special DevOps software, which helps the developer reproduce the software environment on the laptop when deploying apps. But before all that, Hutchinson has to put in a dig:
It s an impressive achievement, and it s also a sad comment on the overall viability of Linux as a consumer-facing operating system for normal people. I don t think anyone is arguing that Linux hasn t earned its place in the data center it most certainly has but there s no way I d feel comfy installing even newbie-friendly Ubuntu or Mint on my parents computers. The XPS 13 DE shows the level of functionality and polish possible with extra effort, and that effort and polish together means this kind of Linux integration is something we won t see very often outside of boutique OEMs.
Of course, Windows is actually worse than Linux on the hardware front when you don t get it pre-installed. Imagine if more vendors put as much effort into preinstalled Linux as they did into preinstalled Windows. In that alternate reality, I imagine people would react more like this:
Isn t that what you re looking for in a mainstream product? Rick chided. In 1996 it was: Wow look at this, I got Linux running on xxxxxxxx. Even in 2006 that was at times an accomplishment When was the last time you turned on an Apple or Windows machine and marveled that it just worked? It should be boring.
Which was, of course, the reaction Hutchinson got when discussing the review with a Linux-using friend. With Microsoft being less of a friend to the hardware vendors every day, here s a case study more of them should be paying attention to.

24 April 2013

Joachim Breitner: The carbondioxide footprint of Debian's Haskell packages

By now, Debian ships quite a lot of Haskell packages (~600). Because of GHC's ABI volatility, whenever we upload a new version of a library, we have to rebuild all libraries that depend on that. In particular, if we upload a new version of the compiler itself, we have to rebuild all Haskell library packages. So we have to rebuild stuff a lot. Luckily, Debian has a decent autobuilding setup so that I just need to tell it what to rebuild, and the rest happens automatically (including figuring out the actual order to build things).
I was curious how much we use the buildd system compared to other packages, and also how long the builders are busy building Haskell packages. All the data is in a postgresql database on buildd.debian.org, so with some python and javascript code, I can visualize this. The graphs show the number of all uploads by autobuilder on the amd64 architecture, with haskell uploads specially marked, and the second graph does the same for the build time. You can select time ranges and get aggregate statistics for that time span. During the last four days a complete rebuild was happening, due to the upload of GHC 7.6.3. During these 2 days and 18 hours building 537 packages took 48 hours of build time and produced 15kg of CO2. That is 94% of all uploads and 91% the total build time. The numbers are lower for the whole of last year: 52% of uploads, 31% of build time and 57kg of CO2. (The CO2 numbers are very rough estimates.)
Note that amd64 is a bit special, as most packages are uploaded on this architecture by the developers, so no automatic builds are happening. On other architectures have, every upload of a (arch:any) package is built, so the share of Haskell packages will be lower. Unfortunately, at the moment the database does not provide me with a table across all architectures (and I was too lazy to make it configurable yet).

23 April 2013

Ulrich Dangel: Visualizing DPL votes

Axel XTaran Beckert recently asked in #debian-devel for a visualization of the Debian Project Leader Election results. Unfortunately there seems to be none, except the table in the mail. As I am currently trying to use ggplot2 for more things, I thought I would give it a try and convert the data into a csv file and process the data via R. For converting the original data i used vim to do a little text processing and created a csv file. By running the following code we can create a nice looking simple graphic:
library("ggplot2")
votes <- read.csv("dpl-2013.csv")
votes2 <- melt(subset(votes, select=c("Year", "DDs", "unique")), id="Year")
p <- ggplot(votes2, aes(Year, value, group=variable, fill=variable)) + ylab('DD #')
p <- p + geom_point() + geom_line() + geom_area(position='identity')
p <- p + scale_fill_manual("Values", values=c(alpha('#d70a53', 0.5),
alpha("blue", 0.5)), breaks=c("DDs", "unique"), labels=c("Developer count", "Voters"))
print(p)
DPL Votes 2013 I also created a little script for automatically processing the csv file and creating a similar plot. Feel free to fork/clone extend this script.

19 April 2013

Daniel Pocock: Switzerland Schilthorn with 007 (final leg)

Following up on the second and third legs of the cable car journey, we now have the final leg, rising up to the summit of the Schilthorn: <video controls="" height="340" poster="http://upload.wikimedia.org/wikipedia/commons/thumb/0/03/Schilthorn_with_Bernese_Alps%2C_2012_August.jpg/800px-Schilthorn_with_Bernese_Alps%2C_2012_August.jpg" width="560">
<source src="http://video.danielpocock.com/VID_20121025_123951.m4v" type="video/mp4"></source>
<source src="http://video.danielpocock.com/VID_20121025_123951.webm" type="video/webm"></source>
Video not appearing? Click here to access the page directly
</video> Unfortunately, the gangsters, guns and girls from On Her Majesty's Secret Service won't be waiting to greet you at the top, but there is a spectacular view. On a clear day, you can see all the way across to the venue of DebConf13 in Vaumarcus, Switzerland and also the Black Forest in Germany.

18 April 2013

Daniel Pocock: Getting to Switzerland, travel costs

With so many people contemplating the cost of travel to Switzerland this summer, I thought I would share some eye-opening insights into how to get a better deal. Let's look at some costs for a hypothetical DebConf13 visitor coming from New York. Flight prices taken today from a comparison site for 8 - 18 August, car hire prices from Europcar and rail prices from SBB (regular tickets) and Swiss Travel System (tourist rail passes):
Departure airport Arrival airport Flight cost Bus to Swiss border Rail pass Total (Flight+bus/train) Airport car hire, 10 days Total (Flight+car)
New York Geneva $1,153 n/a CHF 82
Return ticket
$1,235 CHF 668 $1,823
New York Milan $990 $20 CHF 315
Flexi pass
$1,350 EUR 413
CHF 496
$1,490
Warning: car rental prices based on advance booking - if you just arrive at a Swiss airport and hire a car on the spot, it is likely to be much more expensive, maybe CHF 200 per day For a single traveler, it is slightly cheaper to fly into Geneva (closest airport) and buy a return train ticket, as long as no other travel in Switzerland is desired and sleeping on-site at Vaumarcus, not using a bus every day. If traveling in Switzerland to see the mountains, then it works out cheaper to fly into Milan, Italy, which is very close to the Swiss border. Most of the journey from Milan to Vaumarcus can be covered using one of the Swiss railway travel passes, with stops for sight-seeing on the way, and the other days on the pass used for sight-seeing. Now imagine people arriving in a group: the rail ticket prices quickly add up. For a family of four, 4 * 315 = CHF 1,260. Car rental may be a better option. Hiring a car in Switzerland is expensive, hiring in Italy is much cheaper, the plane tickets to Italy are cheaper too, so there is a lot of money to be saved. But who wants the hassle of travelling to Switzerland via Italy? Of course that is the silliest rhetorical question you are going to see this week: visiting Italy is not a hassle at all. Even the Swiss enjoy going there so much that they just built the world's longest rail tunnel under the Alps to bring Como and Milan that little bit closer, the Gotthard Base Tunnel. The Italian connection could involve a few days in Como or Brunate. If designer fashion labels are appealing to you (or as a gift), Vertemate has one well known outlet for a major brand and the prices really are much cheaper than any of the cities. If you want a laugh, it's worth comparing the hotel prices in Lugano on the Swiss side of the border against prices in nearby Como. Traveling north from Como, taking a train to Luzern (and a connection to Vaumarcus), it is possible to stop in Goschenen and go up into the mountains to experience the Glacier Express featured in my earlier video or get off the train at Arth-Goldau to cross Rigi on the Rigi Bahn (from my other video), take a boat across the lake to Luzern and then continue to Vaumarcus. The Italian entry point isn't just an option for those coming from the US. Even a traveler from London will find that the budget airlines have largely snubbed the Swiss airports with their high operating costs. Easyjet offers a limited number of flights to Geneva and Zurich, but the prices are always higher than their other destinations. Ryanair offers flight's to Milan's Bergamo airport and Easyjet flies to Milan Malpensa.

17 April 2013

Joey Hess: Template Haskell on impossible architectures

Imagine you had an excellent successful Kickstarter campaign, and during it a lot of people asked for an Android port to be made of the software. Which is written in Haskell. No problem, you'd think -- the user interface can be written as a local webapp, which will be nicely platform agnostic and so make it easy to port. Also, it's easy to promise a lot of stuff during a Kickstarter campaign. Keeps the graph going up. What could go wrong? So, rather later you realize there is no Haskell compiler for Android. At all. But surely there will be eventually. And so you go off and build the webapp. Since Yesod seems to be the pinnacle of type-safe Haskell web frameworks, you use it. Hmm, there's this Template Haskell stuff that it uses a lot, but it only makes compiles a little slow, and the result is cool, so why not. Then, about half-way through the project, it seems time to get around to this Android port. And, amazingly, a Haskell compiler for Android has appeared in the meantime. Like the Haskell community has your back. (Which they generally seem to.) It's early days and rough, lots of libraries need to be hacked to work, but it only takes around 2 weeks to get a port of your program that basically works. But, no webapp. Cause nobody seems to know how to make a cross-compiling Haskell compiler do Template Haskell. (Even building a fully native compiler on Android doesn't do the trick. Perhaps you missed something though.) At this point you can give up and write a separate Android UI (perhaps using these new Android JNI bindings for Haskell that have also appeared in the meantime). Or you can procrastinate for a while, and mull it over; consider rewriting the webapp to not use Yesod but some other framework that doesn't need Template Haskell. Eventually you might think this: If I run ghc -ddump-splices when I'm building my Yesod code, I can see all the thousands of lines of delicious machine generated Haskell code. I just have to paste that in, in place of the Template Haskell that generated it, and I'll get a program I can build on Android! What could go wrong? And you even try it, and yeah, it seems to work. For small amounts of code that you paste in and carefully modify and get working. Not a whole big, constantly improving webapp where every single line of html gets converted to types and lambdas that are somehow screamingly fast. So then, let's automate this pasting. And so the EvilSplicer is born! That's a fairly general-purpose Template Haskell splicer. First do a native build with -ddump-splices output redirected to a log file. Run the EvilSplicer to fix up the code. Then run an Android cross-compile. But oh, the caveats. There are so many ways this can go wrong.. Anyway, if you struggle with it, or pay me vast quantities of money, your program will, eventually, link. And that's all I can promise for now.
PS, I hope nobody will ever find this blog post useful in their work. PPS, Also, if you let this put you off Haskell in any way .. well, don't. You just might want to wait a year or so before doing Haskell on Android.

12 April 2013

Keith Packard: dri3k first steps

DRI3K First Steps Here s an update on DRI3000. I ll start by describing what I ve managed to get working and then summarize discussions that happened on the xorg-devel mailing list. Private Back Buffers One of the big goals for DRI3000 is to finish the job of moving buffer management out of the X server and into applications. The only thing still allocated by DRI2 in the X server are back buffers; everything else moved to the client side. Yes, I know, this breaks the GLX requirement for sharing buffers between applications, but we just don t care anymore. As a quick hack, I figured out how to do this with DRI2 today allocate our back buffers separately by creating X pixmaps for them, and then using the existing DRI2GetBuffersWithFormat request to get a GEM handle for them. Of course, now that all I ve got is a pixmap, I can t use the existing DRI2 swap buffer support, so for now I m just using CopyArea to get stuff on the screen. But, that works fine, as long as you don t care about synchronization. Handling Window Resize The biggest pain in DRI2 has been dealing with window resize. When the window resizes in the X server, a new back buffer is allocated and the old one discarded. An event is delivered to invalidate the old back buffer, but anything done between the time the back buffer is discarded and when the application responds to the event is lost. You can easily see this with any GL application today resize the window and you ll see occasional black frames. By allocating the back buffer in the application, the application handles the resize within GL; at some point in the rendering process the resize is discovered, and GL creates a new buffer, copies the existing data over, and continues rendering. So, the rendered data are never lost, and every frame gets displayed on the screen (although, perhaps at the wrong size). The puzzle here was how to tell that the window was resized. Ideally, we d have the application tell us when it received the X configure notify event and was drawing the frame at the new size. We thought of a cute hack that might do this; track GL calls to change the viewport and make sure the back buffer could hold the viewport contents. In theory, the application would receive the X configure notify event, change the viewport and render at the new size. Tracking the viewport settings for an entire frame and constructing their bounding box should describe the size of the window; at least it should describe the intended size of the window. There s at least one serious problem with this plan applications may well call glClear before calling glViewport, and as glClear does not use the current viewport, instead clearing the whole window, we couldn t use the viewport as an indication of the current window size. However, what this exercise did lead us to realize was that we don t care what size the window actually is, we only care what size the application thinks it is. More accurately, the GL library just needs to be aware of any window configuration changes before the application, so that it will construct a buffer that is not older than the application knowledge of the window size. I came up with two possible mechanisms here; the first was to construct a shared memory block between application and X server where the X server would store window configuration changes and signal the application by incrementing a sequence number in the shared page; the GL library would simply look at the sequence number and reallocate buffers when it changed. The problem with the shared memory plan was that it wouldn t work across the network, and we have a future project in mind to replace GLX indirect rendering with local direct rendering and PutImage which still needs accurate window size tracking. More about that project in a future post though X Events to the Rescue So, I decided to just have the X server send me events when the window size changed. I could simply use the existing X configure notify events, but that would require a huge infrastructure change in the application so that my GL library could get those events and have the application also see them. Not knowing what the application is up to, we d have to track every ChangeWindowAttributes call and make sure the event_mask included the right bits. Ick. Fortunately, there s another reason to use a new event we need more information than is provided in the ConfigureNotify event; as you know, the Swap extension wants to have applications draw their content within a larger buffer that can have the window decorations placed around it to avoid a copy from back buffer to window buffer. So, our new ConfigureNotify event would also contain that information. Making sure that ConfigureNotify event is delivered before the core ConfigureNotify event ensures that the GL library should always be able to know about window size changes before the application. Splitting the XCB Event Stream Ok, so I ve got these new events coming from the X server. I don t want the application to have to receive them and hand them down to the GL library; that would mean changing every application on the planet, something which doesn t seem very likely at all. Xlib does this kind of thing by allowing applications to stick themselves into the middle of the event processing code with a callback to filter out the events they re interested in before they hit the main event queue. That s how DRI2 captures Invalidate events, and it works , but using callbacks from the middle of the X event processing code creates all kinds of locking nightmares. As discussed above, I don t care when GL sees the configure events, as long as it gets them before the application finds about about the window size change. So, we don t need to synchronously handle these events, we just need to be able to know they ve arrived and then handle them on the next call to a GL drawing function. What I ve created as a prototype is the ability to identify specific events and place them in a separate event queue, and when events are placed in that event queue, to bump a sequence number so that the application can quickly identify that there s something to process. Making the Event Mask Per-API Instead of Per-Client The problem described above about using the core ConfigureNotify events made me think about how to manage multiple APIs all wanting to track window configuration. For core events, the selection of which events to receive is all based on the client; each client has a single event mask, and each client receives one copy of each event. Monolithic applications work fine with this model; there s one place in the application selecting for events and one place processing them. However, modern applications end up using different APIs for 3D, 2D and media. Getting those libraries to cooperate and use a common API for event management seems pretty intractable. Making the X server treat each API as a separate entity seemed a whole lot easier; if two APIs want events, just have them register separately and deliver two events flagged for the separate APIs. So, the new DRI3 configure notify events are created with their own XID to identify the client-side owner of the event. Within the X server, this required a tiny change; we already needed to allocate an XID for each event selection so that it could be automatically cleaned up when the client exited, so the only change was to use the one provided by the client instead of allocating one in the server. On the wire, the event includes this new XID so that the library can use it to sort out which event queue to stick the event in using the new XCB event stream splitting code. Current Status The above section describes the work that I ve got running; with it, I can run GL applications and have them correctly track window size changes without losing a frame. It s all available on the dri3 branches of my various repositories for xcb proto, libxcb, dri3proto and the X server. Future Directions The first obvious change needed is to move the configuration events from the DRI3 extension to the as-yet-unspecified new Swap extension (which I may rename as Present , as in please present this pixmap in this window ). That s because they aren t related to direct rendering, but rather to tracking window sizes for off-screen rendering, either direct, indirect or even with the CPU to memory. DRI3 and Fences Right now, I m not synchronizing the direct rendering with the CopyArea call; that means the X server will end up with essentially random contents as the application may be mid-way through the next frame before it processes the CopyArea. A simple XSync call would suffice to fix that, but I want a more efficient way of doing this. With the current Linux DRI kernel APIs, it is sufficient to serialize calls that post rendering requests to the kernel to ensure that the rendering requests are themselves serialized. So, all I need to do is have the application wait until the X server has sent the CopyArea request down to the kernel. I could do that by having the X server send me an X event, but I think there s a better way that will extend to systems that don t offer the kernel serialization guarantee. James Jones and Aaron Plattner put together a proposal to add Fences to the X Sync extension. In the X world, those offer a method to serialize rendering between two X applications, but of course the real goal is to expose those fences to GL applications through the various GL sync extensions (including GLARBsync and GLNVfence). With the current Linux DRI implementation, I think it would be pretty easy to implement these fences using pthread semaphores in a block of memory shared between the server and application. That would be DRI-specific; other direct rendering interfaces would use alternate means to share the fences between X server and application. Swap/Present The Second Extension By simply using CopyArea for my application presentation step, I think I ve neatly split this problem into manageable pieces. Once I ve got the DRI3 piece working, I ll move on to fixing the presentation issue. By making that depend solely on existing core Pixmap objects as the source of data to present, I can develop that without any reference to DRI. This will make the extension useful to existing X applications that currently have only CopyArea for this operation. Presentation of application contents occurs in two phases; the first is to identify which objects are involved in the presentation. The second is to perform the presentation operation, either using CopyArea, or by swapping pages or the entire frame buffer. For offscreen objects, these can occur at the same time. For onscreen, the presentation will likely be synchronized with the scanout engine. The second form will mean that the Fences that mark when the presentation has occurred will need to signaled only once the operation completes. A CopyArea operation means that the source pixmap is ready immediately after the Copy has completed. Doing the presentation by using the source pixmap as the new front buffer means that the source pixmap doesn t become ready until after the next swap completes. What I don t know now is whether we ll need to report up-front whether the presentation will involve a copy or a swap. At this point, I don t think so the application will need two back buffers in all cases to avoid blocking between the presentation request and the presentation execution. Yes, it could use a fence for this, but that still sticks a bubble in the 3D hardware where it s blocked waiting for vblank instead of starting on the next frame immediately. Plan of Attack Right now, I m working on finishing up the DRI3 piece: The first three seem fairly straight forward. The fencing stuff will involve working with James and Aaron to integrate their XSync changes into the server. After that, I ll start working on the presentation piece. Foremost there is figuring out the right name for this new extension; I started with the name Swap as that s the GL call it implements. However, Swap is quite misleading as to the actual functionality; a name more like Present might provide a better indication of what it actually does. Of course, Present is both a verb and a noun, with very different connotations. Suggestions on this most complicated part of the project are welcome!

Michal &#268;iha&#345;: Hackweek 9 is over

Hackweek 9 is over and it's time to share what I've done on Weblate during that. I think everything went quite well and Weblate is now ready for 1.5 release. I'm slowly deploying it on my installations (unfortunately this release migration will need some noticeable downtime for bigger installations) and everything seems to work fine so far. I believe this is possible thanks to massive test coverage - all important code is covered by testcases. So what you can expect in 1.5 release? The most visible change is probably new machine translation support, providing support for way more backends and allow you to plug in own services as well. The other changes include word counting (what might give you more idea how much work is remaining) or fancy progress bars in all places (they used to be available for translations only). From the other side, Weblate can now run custom scripts to pre-process translations before commit, what can be used for various things from generating byte compiled files to sorting or cleaning up the translation files. Also Weblate should be now much faster - there were dozen of optimizations done, leading to much lower press on database server. If you want to see more detailed work progress, check Hackweek project page or Weblate changelog. PS: In case no problems appear, Weblate 1.5 should be released on Sunday.

Filed under: English Phpmyadmin Suse Weblate 0 comments Flattr this!

11 April 2013

Matthew Palmer: RSpec the easy way

Anyone that has a fondness for good ol RSpec knows that there s a fair number of matchers and predicates and whatnot involved. Life isn t helped by the recent (as of 2.11) decision to switch to using expect everywhere instead of should (apparently should will be going away entirely at some point in the future). There is a good looking RSpec cheatsheet out on the net, but it dates from 2006, and things have changed since then. We re using RSpec at work a lot at the moment, though, so our tech writer kindly updated it for the new-style syntax, gave it a nice blue tint, and put it out there for the world at large to use. Here is our updated RSpec cheatsheet for anyone who is interested. I can tell you for certain that a double-sided, laminated version of this sucker looks very nice, and is a handy addition to the desk-of-many-things.

Russ Allbery: Review: Familiar

Review: Familiar, by J. Robert Lennon
Publisher: Greywolf
Copyright: 2012
ISBN: 1-55597-535-6
Format: Hardcover
Pages: 205
This is the first book of an experiment. I'm fairly well-read in science fiction and fantasy and increasingly well-read in non-fiction of interest (although there's always far more of that than I'll get to in a lifetime), but woefully unfamiliar with what's called "mainstream" literature. Under the principal that things people are excited about are probably exciting, I've wanted to read more and understand the appeal. Powell's, which I like to support anyway, has a very nice (albeit somewhat expensive) book club called Indiespensable, which sends its subscribers very nice editions of new works that Powell's thinks are interesting, with a special focus on independent publishers. So I signed up and hope to stick with it for at least a year. (The trick will be fitting these books in amongst my regular reading.) Familiar is the first feature selection I received. Elisa Brown is a mother with a dead son and a living one, a failing marriage, an affair, and a life that is, in short, falling apart. Then, while driving back from her annual pilgrimage from her son's grave, the world seems to twist and change. She finds herself dressed for business, wearing a nametag and apparently coming back from a work-related convention, driving a car that's entirely unfamiliar to her. When she gets home, everything else has changed too: her marriage seems to be on firmer ground, but based on rules she doesn't understand. She has a different job, different clothes, a different body in some subtle ways. And both of her sons are alive. I'm going to have to have a long argument with myself about where to (meaninglessly) categorize this on my review site, since in construction it is an alternate reality story and therefore a standard SF trope. Any SF reader is going to immediately assume Elisa has somehow been transported into an alternate reality with a key divergence point from her own. But that's not Lennon's focus. He stays ambiguous on the question of whether this is really happening or whether Elisa had some sort of nervous breakdown, and while some amount of investigation of the situation does take place, it's the sort of investigation that an average person with no access to special resources or scientific knowledge and a completely unbelievable story would be able to do: Internet conspiracy chatrooms and some rather dodgy characters. The focus is instead on Elisa's reaction to the situation, her choices about how to treat this new life, and on how she processes her complex emotions about her family and herself. I had profoundly mixed feelings about this book when I finished it, and revisiting it to review it, I still do. The writing is excellent: spare, evocative, and enjoyable to read. Lennon has a knack for subjective description of emotion and physical experience. The reader feels Elisa's deep discomfort with her changed body and her changed car, her swings between closed-off emotions and sudden emotional connection with a specific situation, and her struggle with the baffling question of how to come to terms with a whole new life. The part of the book from about the middle to nearly the end is excellent. Video games make an appearance and are handled surprisingly well. And when Elisa starts being blunt with people, I found myself both liking her and caring about what happens to her. On the other hand, Familiar also has some serious problems, and one of the biggest is the reaction I feared I'd have to mainstream literature: until Elisa started opening up and taking action, I found it extremely difficult to care about anyone in this book. They're all so profoundly petty, so closed off and engrossed in what seem like depressing and utterly boring lives. I'm sure that some of this is intentional and is there to lay the groundwork for Elisa's own self-discovery, but even towards the end of that self-discovery, everything here is so relentlessly middle-class suburbia that I felt stifled just reading about it. I think it's telling that no one in this book ever seems to have any substantial problem with money, or even with work. Elisa walks into a job that she's never done before and within a few weeks is doing it so well that she can take large amounts of time to wander around for plot purposes. This is a book about highly privileged people being miserable in a bubble. While those people certainly do exist, and I can believe that they act like this, I'm not sure how much I want to read about them. Thankfully, the plot does lead Elisa to poke some holes in that bubble, if never get out of it entirely. This is also another one of those stories in which every character has massive communication problems. Now, this deserves some caveats: Elisa's communication problems with her husband are part of the problem that starts the book and are clearly intentional, as are her communication difficulties with her children. And she's not really close enough to anyone to confide in them. But even with those caveats, no one in this book really talks to anyone else. It's amazing that anyone forms any connections at all, given how many walls and barriers they have around themselves. As someone with a bit of a thing for communication, this drove me nuts to read about, particularly in the first half of the book. But the worst problem is that Lennon completely blows the ending. And by that I don't just mean that I disliked the ending. I mean the ending is so unbelievable and so contrary to the entire rest of the book, at least the way I was reading and understanding it, that I think Familiar is a much better novel if you just remove the final scene entirely. It was such a bizarre and unnecessary twist that I found it infuriating. I don't want to spoil an ending, even a bad ending, so I'll only say this: it felt to me like Lennon just wasn't comfortable with his setting and plot driver and couldn't leave it alone. I think an experienced SF author wouldn't have made this mistake. There were two obvious possible conclusions to draw from the setting, plus a few interesting combinations, and I think someone comfortable with this sort of alternate reality story would have taken one of those options, any of which would have been a reasonable dismount for the plot. Alternately, they could have left it entirely ambiguous to the end and explored why the explanation may not actually matter. But Lennon seemed to me to have a tin ear for plausibility and for the normal flow of this sort of story and seems to have taken it as license for arbitrary events, thus completely violating the internal consistency and emergent rules that he'd spent the rest of the book building. I've mostly talked about my reactions to the characters and the writing and have not said much about the plot. That's somewhat intentional, since figuring out where the story will go is one of the best parts of this book. It's surprisingly tense and well-crafted for not having that much inherent dramatic tension. The excellent writing kept me reading through the first part, when I hated everyone in the story, and then Elisa started taking responsibility for her own life and actions and I started really enjoying the book while being constantly surprised. I think it's the sort of story that's best to take without too much foreknowledge of where it's going. I'm going to call this first experiment a qualfied success. Familiar was certainly interesting to read, and quite different from what I normally read despite the SF premise. If it weren't for the ending, I'd be recommending it to other people. Rating: 6 out of 10

Bdale Garbee: Batteries and Pyro Circuits

Keith and I have discovered a change in the behavior of the protection circuits integrated in the LiPo batteries we sell for use with Altus Metrum products that poses a risk for our customers. This post is meant to document what we now know, communicate changes we're planning as a result, and explain what we think flyers of our existing electronics and batteries may want to do to maximize their chances of successful flights. Background Choosing batteries and designing pyro circuits for high power rocketry avionics involves a variety of trade-offs. Reliability is the highest concern, both because nobody wants to lose an airframe due to a failed pyro event, and also because airframes recovering anomalously have safety implications. But we also care about other factors including cost and weight, and usually want to minimize the complexity of both the electronics and the overall installation. The objective of a pyro circuit is to dump enough energy rapidly into an electric match to cause it to catch fire. We need batteries both to power the electronics that decide when to fire the charge, and to provide the energy that actually ignites the match. The two most common battery types seen in the rocketry hobby are alkaline cells, often the nominal 9V rectangular variety, and rechargeable batteries based on Lithium Polymer (LiPo) chemistry. LiPo cells are 3.7 volts per cell nominal voltage, are very light, and have a high energy density. LiPo capacity is measured in units of current times time, so an 850mAh cell should be able to deliver 850 milliamps for an hour. The battery industry also uses something called a "C rate" to describe how fast the battery can be usefully discharged, wich is a multiplier relative to the battery capacity. So a battery with 850mAh capacity and a "2C" rating can deliver current at a sustained rate of 1700mA until discharged, while the same capacity at a "5C" rating can deliver 4250mA. By comparison, most 9V alkaline batteries are actually composed of 6 individual 1.5V cells enclosed in a wrapper. It's hard to get hard numbers for capacity and discharge rate, since in an alkaline cell the two are not independent, and the discharge rate is related to the volume of each individual cell. The data sheet for an Energizer 522 shows just over 600mAh at a 25mA discharge rate, dropping to about 300mAh at a 500mA discharge rate. Importantly for use in pyro circuits, LiPo cells have a very low source impedance, which means they can source immense amounts of current. It's not unusual for cells in the 1000mAh range to have ratings in excess of 30C! Because this rapid discharge ability can pose a risk of fire, it's common for LiPo cells to come with a "protection board" integrated into the battery assembly that is designed to limit the current to some rate such as 2C continuous duty. In large airframes, or projects that involve staging, air-starts, or other complex pyro event sequences, the most reliable approach will always be to use separate batteries for the control electronics and the source of pyro firing power. In the limit, having separate pyro batteries for each pyro charge with the control electronics only providing the switching to connect the batteries to the charges could even make sense. But for most airframes, this is overkill, and the increases in mass, volume, and wiring complexity just don't make sense. The challenge, then, is how to design electronics that will robustly initiate pyro events without negatively affecting the rest of the electronics when operating from a single shared battery. Altus Metrum Pyro Circuits The very first prototypes of TeleMetrum were designed to use a single-cell LiPo battery, and had an on-board 100mA charging circuit. Because we needed 5 volts to power the accelerometer, we had a small switching regulator that up-converted the LiPo voltage, and we used some of that regulator's output to charge a 1000uF capacitor. The pyro circuit used Fairchild FDN335N N-channel MOSFET switches in a low-side switching configuration to dump the energy stored in the capacitor through an attached ematch. Those FETs had an on resistance of under a tenth of an ohm in our operating conditions. The circuit worked very reliably, but the 1000uF electrolytic cap was huge and we struggled with the mechanics of such a large part hanging off the board... It turns out that 3.7 volts is way more than enough to fire a typical low-current e-match or equivalents like the Quest Q2G2 igniters. In fact, bench testing with a good digital oscilloscope showed that a typical e-match with resistance of 1.3-1.8 ohms will fire in approximately 13 microseconds when hit with the nominal 3.7 volts from a LiPo. So, starting with our v0.2 boards, we switched to using the LiPo battery voltage directly to fire the pyro charges, eliminating the clunky electrolytic capacitor entirely. We also switched to the Fairchild FDS9926A dual N-channel MOSFET whose specs are better in all regards for our application. The on resistance is down around 40 milli-ohms in our circuit, such that the current ratings are much higher (FET current limits are primarily driven by how much heat is generated due to current flowing through the channel's on resistance). Because using the LiPo voltage directly means we're effectively temporarily putting a very low resistance across the battery during the pyro events, the input voltage to the voltage regulator gets pulled down. To ensure the processor could "ride through" these events, we added a 100uF surface mount bulk capacitor on the 3.3 volt regulated voltage rail, which bench testing demonstrated was more than sufficient to maintain processor operation through pyro events. And that is essentially the same pyro circuit on all the boards, both TeleMetrum and TeleMini, that we have shipped to date. What's Changed The LiPo batteries we source and sell with our electronics come with a protection board and cable terminated in a 2-pin, 2mm "JST" connector. The specs on the protection board have always been "2C continuous", but we observed the ability to source much higher currents for short periods such as the 13 micro-seconds or so required to fire an e-match. Thus these protection circuits seemed just fine .. we could fire e-matches with a burst of current but were protected against short circuits in the wiring or our boards by the 2C continuous limit. Unfortunately, the most recent batch of batteries we sourced seem to have a much "twitchier" protection circuit. We can draw more than 2C for short bursts, but not as much as with prior boards, and not for as long an interval. With a 1 ohm power resistor on the pyro terminals of one of our boards, we get about 9 milliseconds of power before the protection circuit cuts in and shuts the battery down. The power stays down until all load is removed, which at least means the board is turned off and back on again, and in some cases could even mean the battery is unplugged and re-plugged since we draw trace current to keep the GPS memory alive even when the power is turned off, and at least some of the new batteries see that as enough to keep the power turned off after an over-current event. For many e-matches, this isn't an issue, since 9 milliseconds is way longer than the 13 microseconds needed to fire the charge. Unfortunately, we've discovered that many of the e-matches bought and used in the rocketry hobby are actually made for use in the fireworks industry, where it is desireable to retain continuity after firing so that series connections of e-matches all can fire even if some fire faster than others! This means that while the resistance goes up some after firing, sometimes the drain on the battery remains sufficient to cause the protection circuit to kick in even after the pyro charge has fired. What We're Doing About It If we remove the protection circuit from the LiPo (or jumper around it), all existing Altus Metrum products will operate successfully with pyro charges thave have an effective resistance of as low as 1 ohm. That's lower than any e-match or Q2G2 we've ever seen, so in effect what this means is that if you have an existing Altus Metrum flight computer, and you remove the LiPo protection circuit, you're good to go. This does not really make things any more "dangerous", since our battery chargers are all current limited and our discharge patterns will never cause heating of the battery. Frankly, in a rocket, the most likely way to cause a problem with a LiPo is by smashing or puncturing the actual battery during bench work or during a crash... and those cause the same problems with or without a protection board present. In the future, we will ship batteries that have either a much higher C rating on the protection circuit, or have no protection circuit at all. The number of problems reported by actual customers that we think should be attributed to LiPo protection circuit boards is very low, and we suspect most of our customers who are happily flying their boards can continue to do so. Ground testing where you fire pyro charges (or at least e-matches) using RF to issue the commands (not USB, since the LiPo charger is running any time USB is connected!) will confirm whether there's a problem. If the board resets (does startup beeps) after a pyro event, or shuts down completely (no LED activity), then you have a problem. If the matches light and the board keeps running, you're good to go. However, any Altus Metrum customer with LiPo batteries sourced from us or our distributors who is worried about this problem (even if you haven't seen problems in ground testing or previous flights), and who doesn't want to try soldering on the battery circuit board yourself, is welcome to contact us about removing the protection circuit for you. We won't charge anything other than shipping. To take advantage of this offer, just send email to fixmybattery@altusmetrum.org telling us how many of which capacity batteries you have that you'd like updated, and we'll respond with an RMA number and shipping details. Going Even Further As previously indicated, with the LiPo protection circuits removed, all of our current products will work reliably with at least 1 ohm across the pyro terminals. That should cover all real-world flying conditions just fine, but we're not satisfied yet. We've designed a new pyro control circuit that transfers the maximum possible energy to the load regardless of battery voltage without ever allowing the voltage to the processor to droop at all. We're testing this new circuit in various prototypes now, and if it pans out it will probably show up first in MegaMetrum and then trickle down to TeleMetrum and TeleMini as those products are updated later this year. The new pyro circuit tolerates 0 ohms (dead shorts) on the pyro terminals for as long as the battery can provide current, which is as good as it gets. We think the circuit is clever enough that we'll probably write more about it once we're finished validating it.

10 April 2013

Paul Wise: Inadequate software

Just 168 of the 4961 packages (3%) I have installed are inadequate. Unfortunately those packages collectively have 3440 inadequacies. How much of the software on your system has these inadequacies? You can find out today by installing Jakub Wilk's software, which is appropriately named adequate. It is now available in Debian unstable. I recommend enabling the apt hook which notifies you when software you are installing is inadequate. Other ways of being notified when you are installing inadequate software include apt-listbugs and debsecan. If you are interested in software quality, Debian's QA activities wiki page provides a good overview of the quality assurance activities that are being worked on within the context of Debian. If you want to provide better quality software for Debian, please keep an eye on the PTS pages for software you maintain. You can also run various automated checks on your software before you make new releases or upload them to the Debian archive. More people are needed to improve and expand upon Debian's existing quality assurance activities and infrastructure. Come join us today!

9 April 2013

Wouter Verhelst: Dear HP,

I bought a ScanJet N6350 a few years back, in the belief that it would work. After all, you write free Linux drivers for all your printers, and when those printers are multifunctional printers with scanners, you do make sure the scanner bits of those printers work too. So it shouldn't be too hard to write a Linux driver for that scanner too, right? Apparently it is. For some reason that I can't imagine, you decided that my money is worth more than my stress levels. Dear HP, you suck. I fail to understand how the basic loss of functionality after a failed scan managed to get through your testing. I fail to understand why you think Linux users should be able to print, but not scan. And I especially fail to understand why I should waste my time with such an expensive device that can't even perform its basic functionality with more than 10% success rate.

8 April 2013

Daniel Pocock: Google Summer of Code, tips for students

I've had quite a few queries about the project ideas for Google Summer of Code on the Lumicall and Debian project pages and I'm publishing some notes here to help people get the best chance of participating, while also helping people's expectations. Firstly, Google is responsible for administering the program and payments, they have the last word on which projects and students participate and free software developers like myself have no direct control over this. In other words, while I would encourage students to make contributions to free software projects to familiarize themselves with the way we work, I can not promise that this will automatically lead to selection. If you are a student, do go ahead and feel free to comment on or make contributions to projects, this helps mentors to see your capabilities very quickly, but don't do any more than you would have done if the GSoC program didn't exist. In any case, building up a portfolio of work that is visible on sites like github does help your employment prospects in the long run. If a development manager is looking at 100 CVs for a job vacancy, and one of them has a strong portfolio on github or Debian, there is a good chance that one person will get an interview - although it doesn't guarantee they get the job. Where to start? If you are a student and you are already heavily involved in some free software project then you probably don't need to read the rest of this. The reality is that GSoC is meant to help bring more people into the free software community and this means we need to help people get their foot in the door. That is what I am addressing here, with examples from my own possible projects. When I went through university in the 90s, there was a very strong UNIX culture in our computer science department and the wider engineering faculty. There were also winds of change, for example, a new dean who wanted to change everything to Windows and a central IT management agenda to abolish campus computers and force students to carry and maintain their own laptops. Around the world, every campus has a different approach. I was fortunate to have UNIX topics like the vi text editor, shell programming and even LaTeX taught as a formal part of the course, students coming after me may have missed those opportunities. I understand that some students have not encountered these concepts at all today. That said, most mentors have a fairly wide tolerance for these things: it is not expected that students know these things, rather, it is an opportunity for you to start learning them whether it is part of your course or not. The key concept here is to be a self-starter: it is not what you know, but your enthusiasm to go and try the things you don't know. That is the very thing you can start doing now, even before the program commences. Doing this will definitely help you identity the areas of free software and Debian that you are most likely to get excited about, and that will lead to a successful project. Some general ideas to get started
  • If your campus hasn't taught UNIX in a formal sense, please familiarise yourself with the basics. While some companies do have more jobs working with other platforms, and some campuses have sadly tried to adapt their courses to prepare students for that limited subset of companies, all the best jobs I've had involve UNIX or Linux. I've never worked for a hedge fund that didn't use Linux. Hollywood uses Linux, NASA does too, but learning Linux is not rocket science. Basic things to understand include the vi text editor, shell scripts and using a command line tool for version control, preferably git. As you will be the system administrator of your own desktop or laptop, it's essential to understand some basic concepts like volume management too.
  • If you need to run some Windows software (some courses do require that), use VirtualBox or a similar solution to run both Windows and Linux on your computer or laptop. The VirtualBox documentation is very good and will help you start quickly. My own laptop runs Debian 7 (wheezy) as the main operating system, and I have Windows and Solaris in VirtualBox for some business applications.
  • My own course included a project that involved downloading the open source GNU make program, writing a patch and testing it. If you've never done something like that, please try. Instead of downloading a project tarball, these days you will typically clone the project's git repository. Even if you only make some trivial change that you will never share, it is good to prove that you can compile and successfully execute something like this.
Some specific ideas for my own projects
  • For Lumicall or Android development, just work your way through the build instructions that are published for all contributors. Trying the f-droid build instructions is also a good way to start.
  • For any of the Debian projects, please make sure you have installed Debian 7 (wheezy) on a desktop, laptop or virtual machine. Just follow the installation guide
  • Once you have installed Debian, try installing whichever one of my packages interests you. Try to configure and use it. Prove that the package works in a normal installation. Look at the log files. Run the test cases.
  • Next, try obtaining the source code and building the package. Debian makes it very easy. Here is an example for reSIProcate, the same works for any other package. As root:
    # apt-get update
    # apt-get build-dep resiprocate
    
    will obtain all the necessary build tools. Then, as a normal user:
    $ mkdir ~/src/resiprocate
    $ cd ~/src/resiprocate
    $ apt-get source resiprocate
    $ cd resiprocate
    $ dpkg-buildpackage -rfakeroot -j12
    
    In a few minutes, you should find the packages (*.deb files) are created.
  • Now, you could try making some small change to the package and compiling it again. Verify that your change works.
If you've successfully got to this point, then there is every chance you will be able to successfully complete a GSoC project. If you struggled along the way, that is OK too, the important thing to do is to engage with the community and learn how to ask for help. The Debian GSoC wiki provides some excellent ideas about engaging with the project using IRC chat and mailing lists. It is also very worthwhile trying to find a local group or a free software event that Debian is attending near you and spending a few hours with people who can look at any problems you have and give you quick solutions.

Next.

Previous.