Search Results: "will"

26 April 2013

Vincent Sanders: When you make something, cleaning it out of structural debris is one of the most vital things you do.

Collabora recently had a problem with a project's ARM build farm. In a nice change of pace it was not that the kernel was crashing, nor indeed any of the software or hardware.
The ProblemInstead our problem was our build farm could best be described as "a pile of stuff" and we wanted to add more systems to it and have switched power control for automated testing.

Which is kinda where the Christopher Alexander quote comes into this. I suggested that I might be able to come up with a better, or at least cleaner, solution.
The IdeaA subrack with sub modulesPrevious experience had exposed me to the idea of using 19 inch subracks for mounting circuits inside submodules.

I originally envisaged the dev boards individually mounted inside these boxes. However preliminary investigation revealed that the enclosures were both expensive and used a lot of space which would greatly increase the rack space required to house these systems.

imx53 QSB eurocard carrier
I decided to instead look at eurocard type subracks with carriers for the systems. Using my 3D printer I came up with a carrier design for the imx53 QSB and printed it. I used the basic eurocard size of 100mm x 160mm which would allow the cards to be used within a 3U subrack.

Once assembled it became apparent that each carrier would be able to share resources like power supply, ethernet port and serial console via USB just as the existing setup did and that these would need to be housed within the subrack.
The Prototype
The carrier prototype was enough to get enough interest to allow me to move on to the next phase of the project. I purchased a Schroff 24563-194 subrack kit and three packs of guide rails from Farnell and assembled it.

Initially I had envisaged acquiring additional horizontal rails from Schroff which would enable constructing an area suitable for mounting the shared components behind the card area.

Rear profile for Schroff subrackUnfortunately Schroff have no suitable horizontal profiles in their catalog and are another of those companies who seem to not want to actually sell products to end users but rather deal with wholesalers who do not have their entire product range!

Printed rear profile for Schroff subrack
Undaunted by this I created my own horizontal rail profile and 3D printed some lengths. The profile is designed to allow a 3mm thick rear cover sheet attached with M2.5 mounting bolts and fit rack sides in the same way the other profiles do.

At this point I should introduce some information on how these subracks are dimensioned. A standard 19 inch rack (as defined in IEC 60297) has a width of 17.75 inches(450.85mm) between the vertical posts. The height is measured in U (1.75 inches)

A subrack must obviously fit in the horizontal gap while providing as much internal space as possible. A subrack is generally either 3 or 6 U high. The width within a subrack is defined in units called HP (Horizontal Pitch) which are 0.2 inches(5.08 mm) and subracks like the Schroff generally list 84 usable HP.

However we must be careful (or actually just learn from me stuffing this up ;-) as the usable HP is not the same thing as the actual length of the horizontal rails! The enclosures actually leave and additional 0.1 inch at either end giving a total internal width of 85HP (17 inches, 431.8 mm) which leaves 0.75 inches for the subrack sides and some clearance.

The Schroff subrack allows eurocards to be slotted into rails where the card centre line is on HP boundaries, hence we describe the width of a card in the slot in terms of HP

I cannot manufacture aluminium extrusions (I know it is a personal failing) nor produce more than 100 mm long length of the plastic profile on my printer.

Even if full lengths are purchased from a commercial service (120 euros for a pair including tax and shipping) the plastic does not have sufficient mechanical strength.

The solution I came up with was somewhat innovative, as an alternative a M5 bolt into a thread in the aluminium extrusion I used a 444mm long length of 4mm threaded rod with nuts at either end. This arrangement puts the extrusion under compression and gives it a great deal of additional mechanical strength as the steel threaded rod is very strong.

Additionally to avoid having to print enough extrusion for the entire length I used some 6mm aluminium tube as a spacer between 6HP(30.48mm) wide sections of the printed extrusion.

It was intended to use a standard modular PC power supply which is 150mm wide which is pretty close to 30HP (6 inches) so it was decided to have a 6HP section of rail at that point to allow a rear mounting plate for the PSU to be attached.

This gives 6HP of profile, 21HP(106.68mm) of tube spacer, 6HP of profile, 46HP(233.68 mm) of tube spacer and a final 6HP profile summing to our total of 85HP. Of course this would be unnecessary if a full continuous 85HP rail had been purchased, but 6 of 6 HP long profile is only 51 euro a saving of 70 euro.

To provide a flat area on which to mount the power switching, Ethernet switch and USB hubs I ordered a 170 x 431 mm sheet of 3mm thick aluminium from inspiredsteel who, while being an ebay company, were fast, cheap and the cutting was accurate.

Do be sure to mention you would prefer it if any error made the sheet smaller rather than larger or it might not fit, for me though they were accurate to the tenth of a mm! If you would prefer the rear section of the rack to be enclosed when you are finished, buy a second sheet for the top. For my prototype I only purchased a 170 x 280mm sheet as I was unsure if I wanted a surface under the PSU (you do, buy the longer sheet)

PC power supply mounted to back plateMounting the PSU was a simple case of constructing a 3 mm thick plate with the correct cutouts and mounting holes for an ATX supply. Although the images show the PSU mounted on the left hand side of the rack this was later reversed to improve cable management.

The subrack needed to provide Ethernet switch ports to all the systems. A TP-Link TL-SF1016DS 16-Port 10/100Mbps Switch was acquired and the switch board removed from its enclosure. The switch selected has an easily removed board and is powered by a single 3.3V input which is readily available from the ATX PSU.

Attention now returned to the eurocard carriers for the systems, the boards to be housed were iMX53 QSB and iMX6 SABRE Lite and a Raspberry Pi control system to act as USB serial console etc.

The carriers for both main boards needed to be 8HP wide, comprised of:
Although only 38 mm this is 7.5HP and fractions of an HP are not possible with the selected subrack.

With 8HP wide modules this would allow for ten slots, within the 84 usable HP, and an eleventh 4HP wide in which the Raspberry Pi system fits.

iMX6 SABRE Lite eurocard carrierCarrier designs for both the i.MX53 QSB and the i.MX6 SABRE Lite boards were created and fabricated at a professional 3D print shop which gave a high quality finish product and removed the perceived risk of relying on a personal 3D printer for a quantity of parts.

This resulted in changes in the design to remove as much material as possible as commercial 3D services charge by the cubic cm. This Design For Manufacture (DFM) step removed almost 50% from the price of the initial design.

i.MX53 QSB carriers with wiring loom
The i.MX6 design underwent a second iteration to allow for the heatsink to be mounted and not mechanically interfere with the hard drive (although the prototype carrier has been used successfully for a system that does not require a hard drive). The lesson learned here is to be aware that an design iteration or two is likely and that it is not without cost.

The initial installation was to have six i.MX53 and two i.MX6 this later changed to a five/four split, however the carrier solution allows for almost any combination, the only caveat (discovered later) is the imx53 carriers should be to the right hand side with the small 4HP gap at that end as they have a JTAG connector underneath the board which otherwise foul the hard drive of the next carrier.

Racked cards showing unwanted cable tails
A wiring loom was constructed for each board giving them a connector tail long enough to allow them to be removed. This was the wrong approach! if you implement this design (or when I do it again) the connector tails on the wiring loom should present all the connections to the rear at the same depth as the Ethernet connection.

The rack cables themselves should be long enough to allow the slides to be removed but importantly it is not desirable to have the trailing cable on the cards. I guess the original eurocard designers figured this out as they designed the cards around the standard fixed DIN connectors at the back of the card slots.

USB relay board with wiring loom attached
We will now briefly examine a misjudgement that caused the initially deployed solution to be reimplemented. As the design was going to use USB serial converters to access the serial console a USB connected relay board was selected to switch the power to each slot. I had previously used serial controlled relay boards with a USB serial convertor however these were no longer available.

Initial deployment with USB controlled relay board
All the available USB relay boards were HID controlled, this did not initially seem to be an issue and Linux software was written to provide a reasonable interface. However it soon became apparent that the firmware on the purchased board was very buggy and crashed the host computer's USB stack multiple times.

Deployed solutionOnce it became apparent that the USB controlled power board was not viable a new design was conceived. As the Ethernet switch had ports available Ethernet controlled relay boards were acquired.

Evolution of 3mm PCB pillars
It did prove necessary to design and print some PCB support posts with M3 nut traps to allow the relay boards to be easily mounted using double sided adhesive pads.

By stacking the relay boards face to face and the Ethernet switch on top separated using nylon spacers it was possible to reduce the cable clutter and provide adequate cable routing space.

A busbar for Ground (black) and unswitched 12V (yellow) was constructed from two lengths of 5A chock block.

An issue with power supply stability was noted so a load resistor was added to the 12V supply and an adhesive thermal pad used to attach it to the aluminium base plate.

Completed redesign
It was most fortunate that the ethernet switch mounting holes lined up very well with the relay board mounting holes allowing for a neat stack.

This second edition is the one currently in use, it has proved reliable in operation and has been successfully updated with additional carriers.

The outstanding issues are mainly centered around the Raspberry Pi control board:
  • Needs its carrier fitting. It is currently just stuck to the subrack end plate.
  • Needs its Ethernet cable replacing. The existing one has developed a fault post installation.
  • Needs the USB hub supply separating from the device cable. The current arrangement lets the hub power the Pi which means you cannot power cycle it.
  • Connect its switched supply separately to the USB hub/devices.
Shopping listThe final bill of materials (excluding labour and workshop costs) which might be useful to anyone hoping to build their own version.

Prices are in GBP currency converted where appropriate and include tax at 20% and delivery to Cambridge UK and were correct as of April 2013.

The purchasing was not optimised and for example around 20GBP could be saved just by ordering all the shapeways parts in one order.
Base subrack
ItemSupplierQuantityLine Price
Schroff 24563-194 subrack kitFarnell141.28
Schroff 24560-351 guide railsFarnell313.65
Schroff rack rear horizontal railShapeways2100.00
1000mm length of 4mm threaded rodB and Q11.48
170mm x 431mm x 3mm Aluminium sheetinspired steel240.00
PSU mounting plateShapeways135.42
PCB standoffShapeways422.30
160mm Deep Modular PC supplyCCL155.76
TP-Link TL-SF1016DS 16-Port 10/100Mbps-SwitchCCL123.77
8 Channel 16A Relay Board Controlled Via EthernetRapid2126.00
Raspberry PiFarnell126.48
USB Serial convertersCCL1037.40
10 port strip style USB HUBEbay17.00
Parts for custom Ethernet cablesRS1326.00
Parts for custom molex power cables (salvaged from scrap ATX PSU)Workshop1111.00
33R 10W wirewound resistor for dummy loadRS11.26
24pin ATX female connector pre-wiredMaplin12.99
Akasa double sided thermal padMaplin15.00
Small cable tie basesMaplin16.49
Miscellaneous cable, connectors, nylon standoffs, solder, heatshrink, zip ties, nuts, washers etc. Workshop120.00
Total for subrack603.28

The carriers are similarly not optimally priced as over five GBP each can be saved by combining shipping on orders alone. Also the SSD drive selection was made some time ago and a newer model may be more suitable.
i.MX53 QSB carrier
ItemSupplierQuantityLine Price
i.MX53 QSBFarnell1105.52
Intel 320 SSD 80GCCL1111.83
Carrier boardShapeways130.00
combined sata data and power (15 to 20cm version)EBay15.00
Low profile right angle 5.5mm x 2.1mm barrel jackEBay10.25
Parts for 9pin serial cable extensionRS15.00
Miscellaneous solder, heatshrink, nylon nuts, bolts and washersWorkshop15.00
Total for carrier262.60

i.MX6 SABRE Lite carrier
ItemSupplierQuantityLine Price
i.MX6 SABRE LiteFarnell1128.06
Intel 320 SSD 80GCCL1111.83
Carrier boardShapeways135.00
combined sata data and power (15 to 20cm version)EBay15.00
Low profile right angle 5.5mm x 2.1mm barrel jackEBay10.25
Parts for 9pin serial cable modificationRS12.00
Miscellaneous solder, heatshrink, nylon nuts, bolts and washersWorkshop15.00
Total for carrier287.14
ConclusionThe solution works and in a 3U high 355mm deep subrack ten ARM development boards can be racked complete with local ethernet switching, power control and serial consoles.

Deployed system in situ configured as a build and test farm
The solution is neat and provides flexibility, density and reproducibility the "pile of stuff" solution failed to do.

For current prototype with nine filled slots the total cost was around 3000GBP or around 330GBP per slot which indicates a 100GBP per slot overhead over the "pile of stuff" solution. These figures omit the costs of the engineer and workshop time, which are estimated at an additional 1500GBP. Therefore a completed rack, fully filled with i.MX6 carriers costs around 5000GBP

Density could be increased if boards with lower height requirements were used however above twelve units there issues with Ethernet switch, power switch and USB port availability become a factor. For Example the 16 port Ethernet switch requires a port for uplink, one for each relay board and one for the console server which leaves only 12 ports for systems.

Addressing the outstanding issues would result in a much more user friendly solution. As the existing unit is in full time use and downtime is not easily scheduled for all ten systems, the issues are not likely to be fixed on the prototype and would have to be solved on a new build.

The solution is probably not suitable for turning into a product but that was not really the original aim. A commercial ARM blade server using this format would almost certainly use standard DIN connectors and a custom PCB design rather than adapting existing boards.

Ian Campbell: Generating Random MAC Addresses

Working with virtualisation I find myself occasionally needing to generate a random MAC address for use with a virtual machine. I have a stupid little local script which spits one out, which is all fine and dandy for my purposes. However this week one of my colleagues was writing a wiki page and needed to include instructions on how to set the MAC address on the Arndale board. He wanted to include a shell snippet to generate one and I suggested that there must be websites which will generate you a suitable address. But when I came to look we found that not a single one of the half a dozen site which I looked at handled the locally administered or multicast bits correctly, meaning that they would randomly generate either multicast addresses or addresses using assigned OUI (or both). Using a random OUI may not cause too much trouble in practice but using a multicast address is sure to lead to strange behaviour, and in any case it's kind of lame. So last night I sat down and wrote a quick hack cgi script to generate random addresses and tonight I deployed it at: http://www.hellion.org.uk/cgi-bin/randmac.pl. This is where the world and his dog leaves a comment pointing me to the existing service my searches missed...

Bits from Debian: Last days for the DebConf13 matching fund!

As part of the DebConf13 fundraising efforts, Brandorr Group is funding a matching initiative for DebConf13, which will be in place for 4 more days (through April 30th). You can donate here! Please consider donating $100, or even $5 or any amount in between, as we can use all the help we can get to reach our fundraising target. The rules are simple: This generous offer will only stay in place through the end of April 30th. Please act quickly, and help spread the world!

Daniel Pocock: An RTC infrastructure for debian.org

A number of potential Summer of Code students are now looking at the question of how to improve RTC in Debian. Making RTC available as a service to the Debian community may be part of the solution, as this would give the community a foundation to test things against and a useful tool for communication. One core component would be a TURN server to help people traverse NAT. There are two free TURN servers already packaged in Debian, and a third one that will potentially be packaged in the near future. Challenge for SoC students: how to make the TURN authentication mechanism work with the existing passwords of people with Debian accounts or alioth accounts? All of the TURN servers would require some code changes or development of new administration tools to link the authentication systems and support LDAP. Another component would be a SIP or XMPP proxy. A very basic implementation would need to allow anybody with a debian.org account to register, make and receive calls, send and receive chat messages. Most SIP proxies and XMPP servers can do this. Challenges for SoC students: how to link up the authentication systems for user passwords? How/where can Debian/alioth users set preferences for call forwarding (adapting the code for the web interface)? How can these tools be integrated with other infrastructure such as the bug tracking system or IRC? There are many additional services that are possible with RTC. A common example is conference calling. Challenge for students: how to set up one of the free conferencing solutions so that Debian Developers can quickly set up conferences using SIP, XMPP or WebRTC? Would it be administered by a web interface? Would it generate emails inviting people to join a conference with a WebRTC link to click? Having RTC services always available on debian.org would enable RTC/VoIP packages to be automatically tested against the central infrastructure using the Jenkins continuous integration system. This would ensure that packages are always interoperable. Challenges for students: pick some of the packages and develop unit testing code for them that allows end-to-end tests against any other VoIP client software. Make it all run under Jenkins.

Jon Dowland: Debian Day #13

Wow, my first "Debian day" in 2013, but it's a bit of a misnomer because I didn't do any Debian work tonight. I spent some time poking at geary, Yorba's new email client which looks promising. I haven't managed to get it working much, though. Someone packaged an old version in Debian but it wouldn't work with either my home or my work IMAP/SMTP servers for various reasons. I did get a git checkout to build in March or April but that stopped working on Debian when they dropped "Precise" support (It seems they were backporting various GIR/Webkit bits and pieces into their own code and didn't want to carry that around any more). They've currently got a fundraiser going, they're aiming for $100k and (at the time of writing) need half of that with only 10 hours to go. Recently I've also been trying to fix a rockbox bug on Sansa Fuze v1 which really cripples writing onto the Fuze. The original firmware does not support microsd capacities >32G which rules it out for me. Sadly the v1 version of the Fuze is uncommon enough that I don't think this bug is getting much attention. My investigations have been limited to fairly dumb bisect-like approaches, combined with lots of writing onto a microsd card (which will probably be killed dead by all this). My initial attempts at a bisect have failed as it turns out the problem is the inverse of a regression: it bizarrely seems as if the bug was fixed in a version-branch, rather than stopped working in one, so it has never been squashed in the main branch at all. Furthermore, the commit that seems to fix the problem is a totally benign version bump, and almost certainly hasn't fixed the bug. So with sufficient further testing I'll probably just prove that this bug has never been fixed. It might be time to switch tack and look into diagnosing the bug rather than trying to side-step it. Or perhaps just spend 15 on a Sansa Clip Zip and forget all about the Fuze v1. Either way I hope someone releases a 128G microsd card this year! Every now and then I wonder whether it would make sense to package rockbox in Debian in some way. Probably not.

25 April 2013

David Bremner: Exporting Debian packaging patches from git, (redux)*

(Debian) packaging and Git. The big picture is as follows. In my view, the most natural way to work on a packaging project in version control [1] is to have an upstream branch which either tracks upstream Git/Hg/Svn, or imports of tarballs (or some combination thereof, and a Debian branch where both modifications to upstream source and commits to stuff in ./debian are added [2]. Deviations from this are mainly motivated by a desire to export source packages, a version control neutral interchange format that still preserves the distinction between upstream source and distro modifications. Of course, if you're happy with the distro modifications as one big diff, then you can stop reading now gitpkg $debian_branch $upstream_branch and you're done. The other easy case is if your changes don't touch upstream; then 3.0 (quilt) packages work nicely with ./debian in a separate tarball. So the tension is between my preferred integration style, and making source packages with changes to upstream source organized in some nice way, preferably in logical patches like, uh, commits in a version control system. At some point we may be able use some form of version control repo as a source package, but the issues with that are for another blog post. At the moment then we are stuck with trying bridge the gap between a git repository and a 3.0 (quilt) source package. If you don't know the details of Debian packaging, just imagine a patch series like you would generate with git format-patch or apply with (surprise) quilt. From Git to Quilt. The most obvious (and the most common) way to bridge the gap between git and quilt is to export patches manually (or using a helper like gbp-pq) and commit them to the packaging repository. This has the advantage of not forcing anyone to use git or specialized helpers to collaborate on the package. On the other hand it's quite far from the vision of using git (or your favourite VCS) to do the integration that I started with. The next level of sophistication is to maintain a branch of upstream-modifying commits. Roughly speaking, this is the approach taken by git-dpm, by gitpkg, and with some additional friction from manually importing and exporting the patches, by gbp-pq. There are some issues with rebasing a branch of patches, mainly it seems to rely on one person at a time working on the patch branch, and it forces the use of specialized tools or workflows. Nonetheless, both git-dpm and gitpkg support this mode of working reasonably well [3]. Lately I've been working on exporting patches from (an immutable) git history. My initial experiments with marking commits with git notes more or less worked [4]. I put this on the back-burner for two reasons, first sharing git notes is still not very well supported by git itself [5], and second Gitpkg maintainer Ron Lee convinced me to automagically pick out what patches to export. Ron's motivation (as I understand it) is to have tools which work on any git repository without extra metadata in the form of notes. Linearizing History on the fly. After a few iterations, I arrived at the following specification. Condition (4) suggests we want something roughly like git format-patch upstream..head, removing those patches which are only about Debian packaging. Because of (3), we have to be a bit careful about commits that touch upstream and ./debian. We also want to avoid outputting patches that have been applied (or worse partially applied) upstream. git patch-id can help identify cherry-picked patches, but not partial application. Eventually I arrived at the following strategy.
  1. Use git-filter-branch to construct a copy of the history upstream..head with ./debian (and for technical reasons .pc) excised.
  2. Filter these commits to remove e.g. those that are present exactly upstream, or those that introduces no changes, or changes unrepresentable in a patch.
  3. Try to revert the remaining commits, in reverse order. The idea here is twofold. First, a patch that occurs twice in history because of merging will only revert the most recent one, allowing earlier copies to be skipped. Second, the state of the temporary branch after all successful reverts represents the difference from upstream not accounted for by any patch.
  4. Generate a "fixup patch" accounting for any remaining differences, to be applied before any if the "nice" patches.
  5. Cherry-pick each "nice" patch on top of the fixup patch, to ensure we have a linear history that can be exported to quilt. If any of these cherry-picks fail, abort the export.
Yep, it seems over-complicated to me too. TL;DR: Show me the code. You can clone my current version from
git://pivot.cs.unb.ca/gitpkg.git
This provides a script "git-debcherry" which does the history linearization discussed above. In order to test out how/if this works in your repository, you could run
git-debcherry --stat $UPSTREAM
For actual use, you probably want to use something like
git-debcherry -o debian/patches
There is a hook in hooks/debcherry-deb-export-hook that does this at source package export time. I'm aware this is not that fast; it does several expensive operations. On the other hand, you know what Don Knuth says about premature optimization, so I'm more interested in reports of when it does and doesn't work. In addition to crashing, generating multi-megabyte "fixup patch" probably counts as failure. Notes
  1. This first part doesn't seem too Debian or git specific to me, but I don't know much concrete about other packaging workflows or other version control systems.
  2. Another variation is to have a patched upstream branch and merge that into the Debian packaging branch. The trade-off here that you can simplify the patch export process a bit, but the repo needs to have taken this disciplined approach from the beginning.
  3. git-dpm merges the patched upstream into the Debian branch. This makes the history a bit messier, but seems to be more robust. I've been thinking about trying this out (semi-manually) for gitpkg.
  4. See e.g. exporting. Although I did not then know the many surprising and horrible things people do in packaging histories, so it probably didn't work as well as I thought it did.
  5. It's doable, but one ends up spending about a bunch lines of code on duplicating basic git functionality; e.g. there is no real support for tags of notes.
  6. Since as far as I know quilt has no way of deleting files except to list the content, this means in particular exporting upstream should yield a DFSG Free source tree.

Wouter Verhelst: Dear supplier,

I don't usually blog about work, but this time around, you maxed it. When I specifically ask you to not ship goods, I have good reasons for that. Specifically: Not Happy(tm)

24 April 2013

Joachim Breitner: The carbondioxide footprint of Debian's Haskell packages

By now, Debian ships quite a lot of Haskell packages (~600). Because of GHC's ABI volatility, whenever we upload a new version of a library, we have to rebuild all libraries that depend on that. In particular, if we upload a new version of the compiler itself, we have to rebuild all Haskell library packages. So we have to rebuild stuff a lot. Luckily, Debian has a decent autobuilding setup so that I just need to tell it what to rebuild, and the rest happens automatically (including figuring out the actual order to build things).
I was curious how much we use the buildd system compared to other packages, and also how long the builders are busy building Haskell packages. All the data is in a postgresql database on buildd.debian.org, so with some python and javascript code, I can visualize this. The graphs show the number of all uploads by autobuilder on the amd64 architecture, with haskell uploads specially marked, and the second graph does the same for the build time. You can select time ranges and get aggregate statistics for that time span. During the last four days a complete rebuild was happening, due to the upload of GHC 7.6.3. During these 2 days and 18 hours building 537 packages took 48 hours of build time and produced 15kg of CO2. That is 94% of all uploads and 91% the total build time. The numbers are lower for the whole of last year: 52% of uploads, 31% of build time and 57kg of CO2. (The CO2 numbers are very rough estimates.)
Note that amd64 is a bit special, as most packages are uploaded on this architecture by the developers, so no automatic builds are happening. On other architectures have, every upload of a (arch:any) package is built, so the share of Haskell packages will be lower. Unfortunately, at the moment the database does not provide me with a table across all architectures (and I was too lazy to make it configurable yet).

22 April 2013

Daniel Pocock: Action week for Google Summer of Code students

This week (22 - 28 April) is the week that really matters most for those students who want to participate in Google Summer of Code. The deadline for student applications is next Friday (3 May), but if you don't spend time exploring project ideas this week, you won't be able to make a strong enough application. Here are the key things I need to see for potential students for the project areas I have offered to co-mentor in:
  1. Register on the Google site, Click here to register yourself (registration opens 22 April and closes 3 May).
  2. Create a page about yourself on the Debian wiki, you must copy the template used by the other students. Make sure you create a link from the student list to your own page. Think carefully about your project: what will be the final result of your work this summer? Describe what it will look like, how Debian people will use it, and explain these things on your wiki page, this will be critical to having Debian accept you.
Those are the essential things you must do before the deadline on 3 May. There are other things you should do as well:
  • Join the Summer of Code co-ordination mailing list
  • Join the mailing lists for whichever open source project and/or Debian packaging team is relevant to your proposed GSoC project
  • On the mailing list, send an email introducing yourself, giving us the link to your wiki page.
To maximize the chance that the Debian project will accept you, it is very important that you interact with us and show us that you understand software development. I am going to try and make this easier: for each of my projects, I've put some wishlist items in the Debian bug tracker. These are all easy tasks that a student can potentially complete. Just do the following:
  • Pick a task from the bug tracker
  • Send an email to the bug tracker (e.g. <bug-number>@bugs.debian.org>) explaining that you will work on the task (and make sure nobody else is already working on it, just look to see if any other emails are attached to the bug report)
  • Write the patch, test it and email the patch to the bug report
Here are the links to the bug lists for each of the packages. You can also pick any other wishlist bug in any related package, but if you are not sure, just pick one of these: If you already have a history of contributing to free software, it is essential that you tell us about it. This will increase the chances that your application is successful. Specifically, on your Debian wiki page, you should include links to any of the following:
  • Emails you sent to open source communities contributing patches or analysis of bugs (give a link to your message in the email archive for the list)
  • Your github account profile page, if you have one
  • Your Sourceforge account profile page, if you have one
  • Any other similar links
Debian receives many applications every year. To help you succeed, you need to think about how Debian will benefit from your work. Use Google to search the Debian site for previous discussions about your idea. Find out the names of Debian Developers who are interested in the same subjects, and email us or email the public mailing lists of the Debian project, for example, debian-devel. Make sure your email includes links to previous discussions or wiki pages about the idea, explain how you can help and ask the community for suggestions and guidance to refine your project proposal. Debian is an open community and we are all very keen to discuss your ideas and help you get involved. DebConf For all students, attending free software events is a great opportunity to meet the community. Of particular relevance for those in Europe, attending DebConf13 in Switzerland is a great way to meet GSoC mentors and other developers. Switzerland's central location is extremely convenient for anybody in Western Europe and it is also Debian's 20th birthday, so it is anticipated DebConf could be more special than usual this year. Please put it in your calendar and subscribe to the debconf-discuss mailing list so you will know when registration opens and other practical details are announced.

Dmitrijs Ledkovs: abi-compliance-checker & dh / cdbs integration

dh-autoreconf is an amazing addon for running autoreconf, I just love it.
abi-compliance-checker is an amazing tool for tracking API/ABI.
Wouldn't it be great to glue abi-compliance-checker into dh / cdbs packaging?!
abi-compliance-checker (1.98.8-1~exp1) experimental; urgency=low
.
* New upstream release
* Add dh_acc to generate and compare library dumps at build time,
together with addons for dh(7) and cdbs.
* Bump standards version, bump debhelper to 9, use 3.0 (quilt) format,
update Vcs-Svn field to canonical form, remove obsolete
DM-Upload-Allowed.
* Apply a patch to allow suffixes on a-c-c abi dumps.
Horay! So how does one use it?

  • build-depend on dh-acc
  • In your debian/rules
    • call dh_acc somewhere appropriate
    • dh $@ --with acc
    • include /usr/share/cdbs/1/rules/acc.mk
  • In your debian/libpackage-dev.acc
    • Write a abi-compliance-checker descriptor (no need to include version)
  • Build your package
  • Copy the generated /usr/lib/$(multiarch)/dh-acc/*.abi.tar.gz as ./debian/lib-package-dev.abi.tar.gz.ARCH
  • Now at each build ABI/API checks will be executed and compat reports will be generated
An example xml descriptor for libapt-pkg-dev is:
<?xml version="1.0" encoding="utf-8"?>
<descriptor>
<headers>
./build/include/apt-pkg/
</headers>
<libs>
./build/bin/
</libs>
<skip_types>
SubstVar
</skip_types>
</descriptor>
I wonder if things can be improved, for example:
  • For trivial packages, don't require .acc at all & simply point abi-compiance-checker at ./debian/tmp/
  • I think logs currently pollute the unpacked source package
  • Multiple "base" ABI need checking. E.g. given versions A, B, C one can introduce a new symbol in B and drop it in C. Both B & C are compatible with A, but C is not compatible with B, thus an API/ABI break got introduced into the distribution if all A,B,C were ever published in the archive.
  • Maybe run these as DEP-8 autopkgtests as well?
On a wider scope of things abi-compliance-checker allows to create "system" abi dumps & also check applications for ABI/API compliance. For example, one could scan deb packages to see if they are still compatible between major Debian Releases for some known install type (e.g. a default full Gnome Desktop), or we could scan proprietary packages to see if they are compatible (e.g. games delivered via Steam by Valve to multiple Ubuntu releases), and we can continuously monitor OS ABI to make sure we don't unknowingly break it with security and bugfix updates.

Please, play around with dh-acc and let me know what you think =)

21 April 2013

Lisandro Dami&aacute;n Nicanor P&eacute;rez Meyer: On the road to Qt 5: Qt 5 base, tools, jsbackend and xmlpatterns in experimental

The first Qt 5 packages have been accepted in Debian experimental.

What's there
To start building Qt 5 apps you will need to export QT_SELECT=qt5, install the package qt5-defaults or read qtchooser's man page. Note that exporting QT_SELECT has precedence over installing qt5-defaults.

What's not there
ArchitecturesAMD64 is already there because it's the arch used by maintainers to build the packages. i386 should be following as soon as buildds catch up. Most surely ARM-based archs will be there at some point too.

Other archs will need some love. Not strange, the Qt project supports amd64, i386 and ARM, but we Debian have normally prepared patches to make it build in other archs. And yes, we try to push them upstream for everyone's benefit. So, if you are missing it in your arch, take a look. You may be the one who enables Qt 5 in it :-)

GLES2 and WaylandWe don't have GLES2 or Wayland support yet. Building it will most probably break the desktop for people using proprietary video drivers (or at least I was told so). I'll surely provide non-official packages with GLES2/Wayland enabled to allow people testing it, but not soon.

This also means that we are not currently able to split X11 and framebuffer support. But we have time to work on it :-)

Non DFSGs compliant files
If you get the original source code tarball from Debian you will notice that it has dfsg in it's name. That means that we had to remove some non DFSG compliant stuff from the original tarball, namely:

  • Every RFC.
  • Three files used for testing the build, which are made of RFCs.
  • Some fonts.

What's following
Other parts of Qt 5 are on the way. And remember, this packages would not have been possible if it weren't for the great Debian's pkg-kde team. My kudos to them.

20 April 2013

Dmitrijs Ledkovs: FTBFS fixes and other patches available for your package!

Wheeze is planned to release very soon! There is still a fair amount of work to push the release out, but it does mean that a lot of fun can begin for jessie.

Over the past cycle in Ubuntu, many packages where synced from experimental, many fixes were applied and many packages were upgraded ahead of debian version strings.

If one goes to your package PTS page at:

packages.qa.debian.org/$SCRPACKAGENAME

One can find the following box titled "ubuntu" in it:


Please click on "patches for VERSION STRING" to find:
  • Useless irrelevant changes =) oh well, it happens
  • FTBFS fixes due to GCC 4.8 (*)
  • FTBFS fixes due to new GlibC
  • FTBFS fixes due to multiarched Python2.7 and Python3.3
  • FTBFS fixes due to multiarched Tcl/Tk
  • FTBFS fixes due to Python hash randomisation enabled
  • FTBFS fixes due to Boost 1.53 (*)
  • Fixes to minimise dependencies when bootstrapping packages
  • Fixes to multiarch more and more libraries
  • Fixes to enable package cross-building
  • Updates to new upstream releases
  • Usage of pybuild
  • Completed migration away from guile1.6
(*) more of those will be available after S-series opens in ubuntu, from April the 29th onwards

Some of the above patches were submitted to debian, with majority of submitted patches have not been applied, simply because none of them were appropriate during freeze and could have been disruptive to Wheezy release.

Please check useful patches available on your PTS!

Please take those patches!

Not taking Ubuntu patches results in more duplicate work in Ubuntu (i.e. rebasing those patches), thus reducing total dev-time available to work to fix other issues.

I myself uploaded a few of those patches into Ubuntu. And in the beginning I was filing BTS bugs with a patch against relevant packages in Debian. Very quickly I realised that since Debian is frozen a lot of those patches would just bitrot in BTS. I of course wished that they would all be instantly applied and uploaded into Experimental, but that did not happen. At the moment there are still 8 bugs open with patches I submitted, with oldest one opened on 6th of July 2012. So that's the reason why I stopped pushing patches to BTS, because well they are not reviewed nor applied in a timely manner =) where my expectation of timely manner is less than 6 months to be uploaded into Experimental or reviewed/rejected/commented on. Not uploading patches because they don't apply to Debian yet, is silly as they will apply to Debian eventually. Unapplied patches continue to generate more work on a more universal scope (e.g. requiring those patches to be rebased in ubuntu, thus many developers spending their time doing that, instead of writting fixes for new issues).

Rog rio Brito: Lack of cooperation from Ubuntu?

I posted the following on Google+, but it is important enough to be reproduced on Planet. I'm editing it a bit, as it is a followup to my previous post. While improving the packaging of MongoDB, there was one thing that caught my attention: that Ubuntu had already done some of the embedded/convenience libraries work, but they had not pushed that work to Debian. Of course, discovered this only after I started working on the improvements of the package. What gives, Ubuntu people? Another thing that I saw is that they have patches enabling mongodb working on armhf. Again, they did not push those to Debian. Why this lack of cooperation? Another thing that I saw is that they had packages for version 2.2 (more precisely, raring has 1:2.2.4-0ubuntu1), while Debian sid just had 2.0 until very recently. Why not push this work and avoid duplication of work? By being a good downstream, I intend to push some of the patches to MongoDB upstream (if they want it), so that we (Debian) have a smaller delta. This will benefit you Ubuntu guys. Why not join forces and help have a world-class set of packages? Please, be good netizens and share the work that you have. I firmly believe that the armhf people will be happy to have one of the fancy "cloud" software available on ARM, especially since the prospects of having ARM machines on datacenters. Oh, just for the record, the kFreeBSD people have sent their contribution and I would love to see (if possible) this running on the HURD.

19 April 2013

Rog rio Brito: Some recent Debian activities

I have been occasionally working on some Debian-related tasks. Chrony One of those was to get chrony is a slightly better shape, by using, at least, a patch system (indeed, I "modernized" its packaging with the format "3.0 (quilt)"), put it in a git repository and would like to receive some comments on what I have so far. The bug Debian bug #694690 contains a very brief description of my intentions and of the problems that I see in the current package. IMVHO, it is a very nice NTP client and server and it could even be used as the default for Debian, once it gets in shape. There is at least another high-profile distribution, namely Fedora, that switched to chrony as its default NTP software. We can certainly take a look at what they are doing and join forces here. MongoDB Another package where I spent some time was with mongodb: MongoDB is a tricky package that is only built for 2 arches: amd64 and i386. The version in unstable for i386 was 2.0.x (roughly the same as for wheezy), which the version in unstable for amd64 was 2.4.1, which has many features that 2.0 lacks. The packaging of it is a bit tricky, since the source tree has bazillion embedded/convenience libraries (e.g., Google's v8, Mozilla's spidermonkey, BOOST, Google's Snappy, PCRE 3 etc.). Up to version 2.4.1-2, it used all these convenience copies, which is of course, a problem for a distribution like Debian. I changed part of the build process to use the libraries that we already have in Debian and, as Antonin Kral uploaded this newer version, unsurprisingly the binary packages are smaller (especially if you take into account that a handful of the libraries may already be installed on the system). A few hours later, Antonin uploaded a new upstream version, which means that we now have better MongoDB packages to play with. I am, in fact, really playing with MongoDB as my first NoSQL database, since 10gen is giving an introductory course on how it works and my motivation was to get what we have in Debian in shape for the course. You can say that I am a firm believer of the "eat your own dogfood" principle. Regarding MongoDB only being built for i386 and amd64, the BTS has a patch to enable building for kFreeBSD, but the patch is for the 2.0 series and the code has changed so drastically in relation to the 2.4 series that there is no hope of it applying. It would be super nice to have MongoDB working on kFreeBSD and on HURD also, though. nocache There is a very nice command line program called nocache that was packaged by Dmitry Smirnov (and just approved by the FTP masters!) whose packaging I briefly reviewed per Dmitry's request and this is an amazing utility whose purpose is to bypass/minimize file system caching for a program. This is especially useful when you are making backups (reading lots of files that would, otherwise, fill the filesystem cache, even if they are not used frequently) or if you are just streaming one file (possibly larger than the system's RAM) to another computer and you have no need to use the file immediately after that. It performs its job by using the LD_PRELOAD mechanism and using posix_fadvise's flag POSIX_FADV_DONTNEED for the files that will be touched. Post note Oh, just one aside: for the readers of Debian Planet and other aggregation services which are not Debian Developers/Maintainers, I contributed to these packages without being the maintainer of them, just scratching some itches and contributing back what I produced.

Gunnar Wolf: Ten years already!

(actually, please set your calendars to the day before yesterday I had a mental tab on this, but it seems watching mental tabs is a low-priority task for brain.sched) Ten years ago today, I got that long awaited mail telling me I had passed all of the needed hurdles and was accepted as a Debian Developer. We were at the first third of a very long release cycle, and the general spirit of the project was clearly younger both as in "things moved easier" and "we were much more immature" Try to follow the mailing list discussions we had back then, and even with all the vitriol that's every now and then spilled on debian-whatever@lists.debian.org, it's clear we have more experience working together. And yes, the main change that ten years bring to a group of people is social. I was at DebConf in Oslo when the now-historic presentation that prompted the birth of the Debian-Women group was given Surely, Debian (and Free Software) still is by far predominantly male and white But I fel it's no longer a hostile group, much to the contrary. Over the years, I was first active (as was the norm by then) as a "solo" maintainer. When Joachim Breitner started the pkg-perl group in 2004, I joined, and was part of the group while an important part of my work was based in Perl. I joined pkg-ruby-extras, and slowly migrated my technical work from one to the other. For several years, I also maintained the Cherokee webserver. I started getting involved in DebConf organization in 2005, and (except for 2008, as I took a vacation from many topics due to personal issues). Back in 2009, I became an official delegate! I joined Jonathan McDowell handling keyring maintenance. One year later, another delegation: With Moray Allan and Holger Levsen, the three of us became the DebConf chairs. This last couple of months, I have been quite inactive in most of my Debian work. I took up teaching at the univerity, and have been devoting what amounts to basically a full time job to prepare material. I expect (hope!) this craze to reach back a "workable" level by late May, when the course finishes, and I can retake some of my usual Debian tasks. Anyway 10 years. Wow. This project is one of the longest commitments in my life. I am still very happy I joined, it still thrills me to say I am part fo this great project, it still makes me proud to be accepted as a peer by so many highly skilled and intelligent people But, as I have repeatedly stated, I see Debian more as a social project (with a technological product) than as a technical one. And as such, I am really happy to have made so many good, close friends in this project, to have the opportunity to work and exchange points of view about anything, and have this large, highly disfunctional but very closely regarded family of friends. So, guys, see you this August in Switzerland. I will be among the group celebrating we have been there for half of the project's history!

18 April 2013

David Moreno: Debian meetup in Utrecht, The Netherlands

This next Tuesday, April 23rd, 2013, a few of us Debian people in the Netherlands will get together for beer and keysigning in Belgisch Bier Caf Olivier in Utrecht. If you are in the area, feel free to come, have a beer or two and mingle! Find the exact details in this debian-events-nl mail. See you there!

Daniel Pocock: Getting to Switzerland, travel costs

With so many people contemplating the cost of travel to Switzerland this summer, I thought I would share some eye-opening insights into how to get a better deal. Let's look at some costs for a hypothetical DebConf13 visitor coming from New York. Flight prices taken today from a comparison site for 8 - 18 August, car hire prices from Europcar and rail prices from SBB (regular tickets) and Swiss Travel System (tourist rail passes):
Departure airport Arrival airport Flight cost Bus to Swiss border Rail pass Total (Flight+bus/train) Airport car hire, 10 days Total (Flight+car)
New York Geneva $1,153 n/a CHF 82
Return ticket
$1,235 CHF 668 $1,823
New York Milan $990 $20 CHF 315
Flexi pass
$1,350 EUR 413
CHF 496
$1,490
Warning: car rental prices based on advance booking - if you just arrive at a Swiss airport and hire a car on the spot, it is likely to be much more expensive, maybe CHF 200 per day For a single traveler, it is slightly cheaper to fly into Geneva (closest airport) and buy a return train ticket, as long as no other travel in Switzerland is desired and sleeping on-site at Vaumarcus, not using a bus every day. If traveling in Switzerland to see the mountains, then it works out cheaper to fly into Milan, Italy, which is very close to the Swiss border. Most of the journey from Milan to Vaumarcus can be covered using one of the Swiss railway travel passes, with stops for sight-seeing on the way, and the other days on the pass used for sight-seeing. Now imagine people arriving in a group: the rail ticket prices quickly add up. For a family of four, 4 * 315 = CHF 1,260. Car rental may be a better option. Hiring a car in Switzerland is expensive, hiring in Italy is much cheaper, the plane tickets to Italy are cheaper too, so there is a lot of money to be saved. But who wants the hassle of travelling to Switzerland via Italy? Of course that is the silliest rhetorical question you are going to see this week: visiting Italy is not a hassle at all. Even the Swiss enjoy going there so much that they just built the world's longest rail tunnel under the Alps to bring Como and Milan that little bit closer, the Gotthard Base Tunnel. The Italian connection could involve a few days in Como or Brunate. If designer fashion labels are appealing to you (or as a gift), Vertemate has one well known outlet for a major brand and the prices really are much cheaper than any of the cities. If you want a laugh, it's worth comparing the hotel prices in Lugano on the Swiss side of the border against prices in nearby Como. Traveling north from Como, taking a train to Luzern (and a connection to Vaumarcus), it is possible to stop in Goschenen and go up into the mountains to experience the Glacier Express featured in my earlier video or get off the train at Arth-Goldau to cross Rigi on the Rigi Bahn (from my other video), take a boat across the lake to Luzern and then continue to Vaumarcus. The Italian entry point isn't just an option for those coming from the US. Even a traveler from London will find that the budget airlines have largely snubbed the Swiss airports with their high operating costs. Easyjet offers a limited number of flights to Geneva and Zurich, but the prices are always higher than their other destinations. Ryanair offers flight's to Milan's Bergamo airport and Easyjet flies to Milan Malpensa.

Hideki Yamane: Grand Unification Debian study Meeting 2013 (Japan)

We Debian JP folks are please to announce Debian event named "Grand Unification Debian study Meeting 2013" will be held in Japan, 29th June (oh, don't ask/blame me for its name, it was decided by others, not me... ;-)

And we're in CfP and negotiating with potential sponsors now, and some companies and media have already ACKed as below.


  • WindowsAzure (Platinum sponsor, yes "Microsoft supports Debian event" :-)
  • Plathome (Gold sponsor, they're also DebConf13 sponsor!)
  • TOP STUDIO (Gold sponsor, thanks to Kenshi Muto :)
  • Serverworks (Gold sponsor, AWS vendor)
If you're in Japan or near here at that time, please come and enjoy this event (and please comment if you have any question).

17 April 2013

Soeren Sonnenburg: CfP: Shogun Machine Learning Workshop, July 12-14, Berlin, Germany

CALL FOR PARTICIPATION: Shogun Machine Learning Workshop, Berlin, Germany, July 12-14, 2013 Data Science, Big-Data are omnipresent terms documenting the need for automated tools to analyze the ever growing wealth of data. To this end we invite practitioners, researchers and students to participate in the first Shogun machine learning workshop. While the workshop is centered around the development and use of the shogun machine learning toolbox, it will also feature general machine learning subjects. General Information The workshop will include: Do not miss the chance to familiarize yourself with the shogun machine learning toolbox for solving various data analysis tasks and to talk to their authors and contributors. The program of the workshop will cover from basic to advanced topics in machine learning and how to approach them using Shogun, which makes it suitable for anyone, no matter if you are a senior researcher or practitioner with many year's of experience, or a junior student willing to discover much more. Interested? A tentative schedule is available at http://shogun-toolbox.org/page/Events/workshop2013_program. Call for contributions The organizing committee is seeking workshop contributions. The committee will select several submitted contributions for 15-minute talks and poster presentations. The accepted contributions will also be published on the workshop web site. Amongst other topics, we encourage submission that Submission Guidelines Send an abstract of your talk/contribution to shogun-workshop2013@shogun-toolbox.org before June 1. Notifications will be given on June 7. Registration Workshop registration is free of charge. However, only a limited number of seats is available. First-come, first-served! Register by filling out the registration form. Location and Timeline The main workshop will take place at c-base Berlin (http://c-base.org/, https://en.wikipedia.org/wiki/C-base) on July 12. It is followed by additional 2-day hands-on sessions held at TU Berlin on July 13-14. About the Shogun machine learning toolbox Shogun is designed for unified large-scale learning for a broad range of feature types and learning settings, like classification, regression, or explorative data analysis. Further information is available at http://www.shogun-toolbox.org.

Daniel Pocock: Switzerland Schilthorn with 007 (leg 3)

Friday's video of the 2nd leg of the cable car journey was quite popular, Ganglia stats for the video server can't be wrong: The first leg was skipped because of the low cloud cover. Weather reports in Switzerland, such as www.meteosuisse.ch often show a distinction between the low clouds and high clouds, because many Swiss mountain resorts are high enough to enjoy ample sunshine by virtue of their altitude even when cities like Zurich and Geneva are overcast. Now for the third leg, rising up to Berg. In the video, it is hard to gauge the size of the rocky cliffs along the route, but looking at the cable car's shadow will give a clue. As the cable car approaches the station in Berg, it feels like it takes forever to complete the journey. That is because the rocks are so much bigger than they appear from a distance. <video controls="" height="340" poster="http://upload.wikimedia.org/wikipedia/commons/thumb/0/03/Schilthorn_with_Bernese_Alps%2C_2012_August.jpg/800px-Schilthorn_with_Bernese_Alps%2C_2012_August.jpg" width="560">
<source src="http://video.danielpocock.com/VID_20121025_122453.m4v" type="video/mp4"></source>
<source src="http://video.danielpocock.com/VID_20121025_122453.webm" type="video/webm"></source>
</video> The final leg to the Schilthorn summit will appear shortly. I understand many people are starting to contemplate their DebConf13 travel and summer holidays - please feel free to email me if you'd like me to post a video of some other feature in Switzerland.
AttachmentSize
ganglia-schilthorn2.png8.71 KB

Next.

Previous.