Search Results: "xam"

26 April 2013

Richard Hartmann: Release Critical Bug report for Week 17

The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the Squeeze release cycle?
Week Squeeze Wheezy Diff
43 284 (213+71) 468 (332+136) +184 (+119/+65)
44 261 (201+60) 408 (265+143) +147 (+64/+83)
45 261 (205+56) 425 (291+134) +164 (+86/+78)
46 271 (200+71) 401 (258+143) +130 (+58/+72)
47 283 (209+74) 366 (221+145) +83 (+12/+71)
48 256 (177+79) 378 (230+148) +122 (+53/+69)
49 256 (180+76) 360 (216+155) +104 (+36/+79)
50 204 (148+56) 339 (195+144) +135 (+47/+90)
51 178 (124+54) 323 (190+133) +145 (+66/+79)
52 115 (78+37) 289 (190+99) +174 (+112/+62)
1 93 (60+33) 287 (171+116) +194 (+111/+83)
2 82 (46+36) 271 (162+109) +189 (+116/+73)
3 25 (15+10) 249 (165+84) +224 (+150/+74)
4 14 (8+6) 244 (176+68) +230 (+168/+62)
5 2 (0+2) 224 (132+92) +222 (+132/+90)
6 release! 212 (129+83) +212 (+129/+83)
7 release+1 194 (128+66) +194 (+128/+66)
8 release+2 206 (144+62) +206 (+144/+62)
9 release+3 174 (105+69) +174 (+105/+69)
10 release+4 120 (72+48) +120 (+72/+48)
11 release+5 115 (74+41) +115 (+74/+41)
12 release+6 93 (47+46) +93 (+47/+46)
13 release+7 50 (24+26) +50 (+24/+26)
14 release+8 51 (32+19) +51 (+32/+19)
15 release+9 39 (32+7) +39 (+32/+7)
16 release+10 20 (12+8) +20 (+12/+8)
17 release+11 24 (19+5) +24 (+19/+5)
18 release+12
Graphical overview of bug stats thanks to azhag:

Vincent Sanders: When you make something, cleaning it out of structural debris is one of the most vital things you do.

Collabora recently had a problem with a project's ARM build farm. In a nice change of pace it was not that the kernel was crashing, nor indeed any of the software or hardware.
The ProblemInstead our problem was our build farm could best be described as "a pile of stuff" and we wanted to add more systems to it and have switched power control for automated testing.

Which is kinda where the Christopher Alexander quote comes into this. I suggested that I might be able to come up with a better, or at least cleaner, solution.
The IdeaA subrack with sub modulesPrevious experience had exposed me to the idea of using 19 inch subracks for mounting circuits inside submodules.

I originally envisaged the dev boards individually mounted inside these boxes. However preliminary investigation revealed that the enclosures were both expensive and used a lot of space which would greatly increase the rack space required to house these systems.

imx53 QSB eurocard carrier
I decided to instead look at eurocard type subracks with carriers for the systems. Using my 3D printer I came up with a carrier design for the imx53 QSB and printed it. I used the basic eurocard size of 100mm x 160mm which would allow the cards to be used within a 3U subrack.

Once assembled it became apparent that each carrier would be able to share resources like power supply, ethernet port and serial console via USB just as the existing setup did and that these would need to be housed within the subrack.
The Prototype
The carrier prototype was enough to get enough interest to allow me to move on to the next phase of the project. I purchased a Schroff 24563-194 subrack kit and three packs of guide rails from Farnell and assembled it.

Initially I had envisaged acquiring additional horizontal rails from Schroff which would enable constructing an area suitable for mounting the shared components behind the card area.

Rear profile for Schroff subrackUnfortunately Schroff have no suitable horizontal profiles in their catalog and are another of those companies who seem to not want to actually sell products to end users but rather deal with wholesalers who do not have their entire product range!

Printed rear profile for Schroff subrack
Undaunted by this I created my own horizontal rail profile and 3D printed some lengths. The profile is designed to allow a 3mm thick rear cover sheet attached with M2.5 mounting bolts and fit rack sides in the same way the other profiles do.

At this point I should introduce some information on how these subracks are dimensioned. A standard 19 inch rack (as defined in IEC 60297) has a width of 17.75 inches(450.85mm) between the vertical posts. The height is measured in U (1.75 inches)

A subrack must obviously fit in the horizontal gap while providing as much internal space as possible. A subrack is generally either 3 or 6 U high. The width within a subrack is defined in units called HP (Horizontal Pitch) which are 0.2 inches(5.08 mm) and subracks like the Schroff generally list 84 usable HP.

However we must be careful (or actually just learn from me stuffing this up ;-) as the usable HP is not the same thing as the actual length of the horizontal rails! The enclosures actually leave and additional 0.1 inch at either end giving a total internal width of 85HP (17 inches, 431.8 mm) which leaves 0.75 inches for the subrack sides and some clearance.

The Schroff subrack allows eurocards to be slotted into rails where the card centre line is on HP boundaries, hence we describe the width of a card in the slot in terms of HP

I cannot manufacture aluminium extrusions (I know it is a personal failing) nor produce more than 100 mm long length of the plastic profile on my printer.

Even if full lengths are purchased from a commercial service (120 euros for a pair including tax and shipping) the plastic does not have sufficient mechanical strength.

The solution I came up with was somewhat innovative, as an alternative a M5 bolt into a thread in the aluminium extrusion I used a 444mm long length of 4mm threaded rod with nuts at either end. This arrangement puts the extrusion under compression and gives it a great deal of additional mechanical strength as the steel threaded rod is very strong.

Additionally to avoid having to print enough extrusion for the entire length I used some 6mm aluminium tube as a spacer between 6HP(30.48mm) wide sections of the printed extrusion.

It was intended to use a standard modular PC power supply which is 150mm wide which is pretty close to 30HP (6 inches) so it was decided to have a 6HP section of rail at that point to allow a rear mounting plate for the PSU to be attached.

This gives 6HP of profile, 21HP(106.68mm) of tube spacer, 6HP of profile, 46HP(233.68 mm) of tube spacer and a final 6HP profile summing to our total of 85HP. Of course this would be unnecessary if a full continuous 85HP rail had been purchased, but 6 of 6 HP long profile is only 51 euro a saving of 70 euro.

To provide a flat area on which to mount the power switching, Ethernet switch and USB hubs I ordered a 170 x 431 mm sheet of 3mm thick aluminium from inspiredsteel who, while being an ebay company, were fast, cheap and the cutting was accurate.

Do be sure to mention you would prefer it if any error made the sheet smaller rather than larger or it might not fit, for me though they were accurate to the tenth of a mm! If you would prefer the rear section of the rack to be enclosed when you are finished, buy a second sheet for the top. For my prototype I only purchased a 170 x 280mm sheet as I was unsure if I wanted a surface under the PSU (you do, buy the longer sheet)

PC power supply mounted to back plateMounting the PSU was a simple case of constructing a 3 mm thick plate with the correct cutouts and mounting holes for an ATX supply. Although the images show the PSU mounted on the left hand side of the rack this was later reversed to improve cable management.

The subrack needed to provide Ethernet switch ports to all the systems. A TP-Link TL-SF1016DS 16-Port 10/100Mbps Switch was acquired and the switch board removed from its enclosure. The switch selected has an easily removed board and is powered by a single 3.3V input which is readily available from the ATX PSU.

Attention now returned to the eurocard carriers for the systems, the boards to be housed were iMX53 QSB and iMX6 SABRE Lite and a Raspberry Pi control system to act as USB serial console etc.

The carriers for both main boards needed to be 8HP wide, comprised of:
Although only 38 mm this is 7.5HP and fractions of an HP are not possible with the selected subrack.

With 8HP wide modules this would allow for ten slots, within the 84 usable HP, and an eleventh 4HP wide in which the Raspberry Pi system fits.

iMX6 SABRE Lite eurocard carrierCarrier designs for both the i.MX53 QSB and the i.MX6 SABRE Lite boards were created and fabricated at a professional 3D print shop which gave a high quality finish product and removed the perceived risk of relying on a personal 3D printer for a quantity of parts.

This resulted in changes in the design to remove as much material as possible as commercial 3D services charge by the cubic cm. This Design For Manufacture (DFM) step removed almost 50% from the price of the initial design.

i.MX53 QSB carriers with wiring loom
The i.MX6 design underwent a second iteration to allow for the heatsink to be mounted and not mechanically interfere with the hard drive (although the prototype carrier has been used successfully for a system that does not require a hard drive). The lesson learned here is to be aware that an design iteration or two is likely and that it is not without cost.

The initial installation was to have six i.MX53 and two i.MX6 this later changed to a five/four split, however the carrier solution allows for almost any combination, the only caveat (discovered later) is the imx53 carriers should be to the right hand side with the small 4HP gap at that end as they have a JTAG connector underneath the board which otherwise foul the hard drive of the next carrier.

Racked cards showing unwanted cable tails
A wiring loom was constructed for each board giving them a connector tail long enough to allow them to be removed. This was the wrong approach! if you implement this design (or when I do it again) the connector tails on the wiring loom should present all the connections to the rear at the same depth as the Ethernet connection.

The rack cables themselves should be long enough to allow the slides to be removed but importantly it is not desirable to have the trailing cable on the cards. I guess the original eurocard designers figured this out as they designed the cards around the standard fixed DIN connectors at the back of the card slots.

USB relay board with wiring loom attached
We will now briefly examine a misjudgement that caused the initially deployed solution to be reimplemented. As the design was going to use USB serial converters to access the serial console a USB connected relay board was selected to switch the power to each slot. I had previously used serial controlled relay boards with a USB serial convertor however these were no longer available.

Initial deployment with USB controlled relay board
All the available USB relay boards were HID controlled, this did not initially seem to be an issue and Linux software was written to provide a reasonable interface. However it soon became apparent that the firmware on the purchased board was very buggy and crashed the host computer's USB stack multiple times.

Deployed solutionOnce it became apparent that the USB controlled power board was not viable a new design was conceived. As the Ethernet switch had ports available Ethernet controlled relay boards were acquired.

Evolution of 3mm PCB pillars
It did prove necessary to design and print some PCB support posts with M3 nut traps to allow the relay boards to be easily mounted using double sided adhesive pads.

By stacking the relay boards face to face and the Ethernet switch on top separated using nylon spacers it was possible to reduce the cable clutter and provide adequate cable routing space.

A busbar for Ground (black) and unswitched 12V (yellow) was constructed from two lengths of 5A chock block.

An issue with power supply stability was noted so a load resistor was added to the 12V supply and an adhesive thermal pad used to attach it to the aluminium base plate.

Completed redesign
It was most fortunate that the ethernet switch mounting holes lined up very well with the relay board mounting holes allowing for a neat stack.

This second edition is the one currently in use, it has proved reliable in operation and has been successfully updated with additional carriers.

The outstanding issues are mainly centered around the Raspberry Pi control board:
  • Needs its carrier fitting. It is currently just stuck to the subrack end plate.
  • Needs its Ethernet cable replacing. The existing one has developed a fault post installation.
  • Needs the USB hub supply separating from the device cable. The current arrangement lets the hub power the Pi which means you cannot power cycle it.
  • Connect its switched supply separately to the USB hub/devices.
Shopping listThe final bill of materials (excluding labour and workshop costs) which might be useful to anyone hoping to build their own version.

Prices are in GBP currency converted where appropriate and include tax at 20% and delivery to Cambridge UK and were correct as of April 2013.

The purchasing was not optimised and for example around 20GBP could be saved just by ordering all the shapeways parts in one order.
Base subrack
ItemSupplierQuantityLine Price
Schroff 24563-194 subrack kitFarnell141.28
Schroff 24560-351 guide railsFarnell313.65
Schroff rack rear horizontal railShapeways2100.00
1000mm length of 4mm threaded rodB and Q11.48
170mm x 431mm x 3mm Aluminium sheetinspired steel240.00
PSU mounting plateShapeways135.42
PCB standoffShapeways422.30
160mm Deep Modular PC supplyCCL155.76
TP-Link TL-SF1016DS 16-Port 10/100Mbps-SwitchCCL123.77
8 Channel 16A Relay Board Controlled Via EthernetRapid2126.00
Raspberry PiFarnell126.48
USB Serial convertersCCL1037.40
10 port strip style USB HUBEbay17.00
Parts for custom Ethernet cablesRS1326.00
Parts for custom molex power cables (salvaged from scrap ATX PSU)Workshop1111.00
33R 10W wirewound resistor for dummy loadRS11.26
24pin ATX female connector pre-wiredMaplin12.99
Akasa double sided thermal padMaplin15.00
Small cable tie basesMaplin16.49
Miscellaneous cable, connectors, nylon standoffs, solder, heatshrink, zip ties, nuts, washers etc. Workshop120.00
Total for subrack603.28

The carriers are similarly not optimally priced as over five GBP each can be saved by combining shipping on orders alone. Also the SSD drive selection was made some time ago and a newer model may be more suitable.
i.MX53 QSB carrier
ItemSupplierQuantityLine Price
i.MX53 QSBFarnell1105.52
Intel 320 SSD 80GCCL1111.83
Carrier boardShapeways130.00
combined sata data and power (15 to 20cm version)EBay15.00
Low profile right angle 5.5mm x 2.1mm barrel jackEBay10.25
Parts for 9pin serial cable extensionRS15.00
Miscellaneous solder, heatshrink, nylon nuts, bolts and washersWorkshop15.00
Total for carrier262.60

i.MX6 SABRE Lite carrier
ItemSupplierQuantityLine Price
i.MX6 SABRE LiteFarnell1128.06
Intel 320 SSD 80GCCL1111.83
Carrier boardShapeways135.00
combined sata data and power (15 to 20cm version)EBay15.00
Low profile right angle 5.5mm x 2.1mm barrel jackEBay10.25
Parts for 9pin serial cable modificationRS12.00
Miscellaneous solder, heatshrink, nylon nuts, bolts and washersWorkshop15.00
Total for carrier287.14
ConclusionThe solution works and in a 3U high 355mm deep subrack ten ARM development boards can be racked complete with local ethernet switching, power control and serial consoles.

Deployed system in situ configured as a build and test farm
The solution is neat and provides flexibility, density and reproducibility the "pile of stuff" solution failed to do.

For current prototype with nine filled slots the total cost was around 3000GBP or around 330GBP per slot which indicates a 100GBP per slot overhead over the "pile of stuff" solution. These figures omit the costs of the engineer and workshop time, which are estimated at an additional 1500GBP. Therefore a completed rack, fully filled with i.MX6 carriers costs around 5000GBP

Density could be increased if boards with lower height requirements were used however above twelve units there issues with Ethernet switch, power switch and USB port availability become a factor. For Example the 16 port Ethernet switch requires a port for uplink, one for each relay board and one for the console server which leaves only 12 ports for systems.

Addressing the outstanding issues would result in a much more user friendly solution. As the existing unit is in full time use and downtime is not easily scheduled for all ten systems, the issues are not likely to be fixed on the prototype and would have to be solved on a new build.

The solution is probably not suitable for turning into a product but that was not really the original aim. A commercial ARM blade server using this format would almost certainly use standard DIN connectors and a custom PCB design rather than adapting existing boards.

Steve Kemp: Modern console mail clients?

I've recently started staging upgrades from Squeeze to Wheezy. One unpleasant surprise was that the mutt-patched package available to Debian doesn't contain the "sidebar-new-only" patch. This means I need to maintain it myself again, which I'd rather avoid. Over time I've been slowly moving to standard Debian systems, trying to not carry too many local perversions around. Unfortunately if you've kept all your mail since 1994 you have many mailboxes. having mutt-patched available at all, with the sidebar patch, is a great timesaver. But I don't want to see mailboxes I'm never going to touch; just mailboxes with new mail in them. Also I find the idea of having to explicitly define mailboxes a pain. Just run inotify on ~/Maildir and discover the damn things yourself. Please computer, compute! If you divide up "mail client" into distinct steps it doesn't seem so hard: Obviously there is more to it than that. Sending mail? exec( sendmail ). Filtering mail? procmail/sieve/etc. Editing mail? exec(vim). I'm sure if I were to start a core of a program, suitable for myself, would be simple enough. Maybe with lua, maybe with javascript, but with a real language at the core. Anyway I've thought this before, and working with quilt and some ropy patches has always seemed like the way to go. Maybe it still is, but I can dream. (PS. Sup + Notmuch both crash on my archives. I do not wish to examine them further. Still some interesting ideas. It should be possible to say "maildirs are tags; view "~/Maildir/.livejournal.2003" and ~/Maildir/.livejournal.2007 at the same time. Why just a single directory in the "index-view? So 1994.) Disjointed posts R Us. Obquote: "How hard could it be?" -- Patrick.

Daniel Pocock: An RTC infrastructure for debian.org

A number of potential Summer of Code students are now looking at the question of how to improve RTC in Debian. Making RTC available as a service to the Debian community may be part of the solution, as this would give the community a foundation to test things against and a useful tool for communication. One core component would be a TURN server to help people traverse NAT. There are two free TURN servers already packaged in Debian, and a third one that will potentially be packaged in the near future. Challenge for SoC students: how to make the TURN authentication mechanism work with the existing passwords of people with Debian accounts or alioth accounts? All of the TURN servers would require some code changes or development of new administration tools to link the authentication systems and support LDAP. Another component would be a SIP or XMPP proxy. A very basic implementation would need to allow anybody with a debian.org account to register, make and receive calls, send and receive chat messages. Most SIP proxies and XMPP servers can do this. Challenges for SoC students: how to link up the authentication systems for user passwords? How/where can Debian/alioth users set preferences for call forwarding (adapting the code for the web interface)? How can these tools be integrated with other infrastructure such as the bug tracking system or IRC? There are many additional services that are possible with RTC. A common example is conference calling. Challenge for students: how to set up one of the free conferencing solutions so that Debian Developers can quickly set up conferences using SIP, XMPP or WebRTC? Would it be administered by a web interface? Would it generate emails inviting people to join a conference with a WebRTC link to click? Having RTC services always available on debian.org would enable RTC/VoIP packages to be automatically tested against the central infrastructure using the Jenkins continuous integration system. This would ensure that packages are always interoperable. Challenges for students: pick some of the packages and develop unit testing code for them that allows end-to-end tests against any other VoIP client software. Make it all run under Jenkins.

Lior Kaplan: The commit police: learning for recent reference mistakes

Continuing the previous post about commits and bugs, I d like to review some mistakes I saw recently. Mistakes do happen, but mentioning them here is meant to teach others and hopefully to reduce similar ones in the future. This post isn t about shaming the authors/commiters. Also, the points I highlight are what I consider as a mistakes or problems, other people might think differently.
  1. Mentioning two bugs in one commit message, which our system doesn t support right now. So the second bug doesn t get the commit notification, and that should be done manually (example: commit 2933935 and fdo#53278).
  2. Referencing gerrit changes as part of the commit message (example: commit 87f185d). Giving references as part of the commit is great and helpful, but I would prefer to see the reference to the actual commit and not to its review process. This is meaningful when you search the log. If the followup change is suggested when the first change is still in review it should be combined, otherwise it already have a commit to reference.
  3. Following up a commit, and not mentioning the bug it references (example: commit 87f185d). In this specific case, the we re talking about a meta bug for translating comments from German to English (fdo#39468), so no harm done. But this important when you want to cherry pick a fix for a bug to other branches and might forget the follow up commits. It s also relevant to more technical meta bugs (example: fdo#62096).
  4. Referencing a mailing list which reference a commit and a bug instead of referencing them directly (commit 21fea27, fdo#60534). This bug shows very well why correct referencing is important, a commit was made to fix the bug, a follow up commit was done without proper reference, and than the first commit was reverted. No one involved in trying to fix that specific bug knew about the follow up commit as it wasn t documented anywhere.
  5. Referencing bugs though full bug URL instead of the right format (example: commit e1f6dac). Also the bug is referenced in the commit in the body of the message instead of the first line (header) which is more visible.
  6. Referencing non existing bugs (example: commit 3a4534b). Which got a manual notification in the bug by the comitter (fdo#33091).
  7. Using shortening services URL as part of commit messages (example: commit 86f8fba). There s no way to know to what the reference is without using the service, which in this case was leading to a post on one of the projects mailing list. There isn t any problem giving the direct URL to the list s archive. It s interesting whether we should link to our URL and is it OK to use other external services who also archive our mailing lists (example: commit e83990a).
To conclude, having references in commit message is really helpful, but please reference the right resource to help people find it easily and to let our automated services parse it.
Filed under: LibreOffice

22 April 2013

Daniel Pocock: Action week for Google Summer of Code students

This week (22 - 28 April) is the week that really matters most for those students who want to participate in Google Summer of Code. The deadline for student applications is next Friday (3 May), but if you don't spend time exploring project ideas this week, you won't be able to make a strong enough application. Here are the key things I need to see for potential students for the project areas I have offered to co-mentor in:
  1. Register on the Google site, Click here to register yourself (registration opens 22 April and closes 3 May).
  2. Create a page about yourself on the Debian wiki, you must copy the template used by the other students. Make sure you create a link from the student list to your own page. Think carefully about your project: what will be the final result of your work this summer? Describe what it will look like, how Debian people will use it, and explain these things on your wiki page, this will be critical to having Debian accept you.
Those are the essential things you must do before the deadline on 3 May. There are other things you should do as well:
  • Join the Summer of Code co-ordination mailing list
  • Join the mailing lists for whichever open source project and/or Debian packaging team is relevant to your proposed GSoC project
  • On the mailing list, send an email introducing yourself, giving us the link to your wiki page.
To maximize the chance that the Debian project will accept you, it is very important that you interact with us and show us that you understand software development. I am going to try and make this easier: for each of my projects, I've put some wishlist items in the Debian bug tracker. These are all easy tasks that a student can potentially complete. Just do the following:
  • Pick a task from the bug tracker
  • Send an email to the bug tracker (e.g. <bug-number>@bugs.debian.org>) explaining that you will work on the task (and make sure nobody else is already working on it, just look to see if any other emails are attached to the bug report)
  • Write the patch, test it and email the patch to the bug report
Here are the links to the bug lists for each of the packages. You can also pick any other wishlist bug in any related package, but if you are not sure, just pick one of these: If you already have a history of contributing to free software, it is essential that you tell us about it. This will increase the chances that your application is successful. Specifically, on your Debian wiki page, you should include links to any of the following:
  • Emails you sent to open source communities contributing patches or analysis of bugs (give a link to your message in the email archive for the list)
  • Your github account profile page, if you have one
  • Your Sourceforge account profile page, if you have one
  • Any other similar links
Debian receives many applications every year. To help you succeed, you need to think about how Debian will benefit from your work. Use Google to search the Debian site for previous discussions about your idea. Find out the names of Debian Developers who are interested in the same subjects, and email us or email the public mailing lists of the Debian project, for example, debian-devel. Make sure your email includes links to previous discussions or wiki pages about the idea, explain how you can help and ask the community for suggestions and guidance to refine your project proposal. Debian is an open community and we are all very keen to discuss your ideas and help you get involved. DebConf For all students, attending free software events is a great opportunity to meet the community. Of particular relevance for those in Europe, attending DebConf13 in Switzerland is a great way to meet GSoC mentors and other developers. Switzerland's central location is extremely convenient for anybody in Western Europe and it is also Debian's 20th birthday, so it is anticipated DebConf could be more special than usual this year. Please put it in your calendar and subscribe to the debconf-discuss mailing list so you will know when registration opens and other practical details are announced.

Dmitrijs Ledkovs: abi-compliance-checker & dh / cdbs integration

dh-autoreconf is an amazing addon for running autoreconf, I just love it.
abi-compliance-checker is an amazing tool for tracking API/ABI.
Wouldn't it be great to glue abi-compliance-checker into dh / cdbs packaging?!
abi-compliance-checker (1.98.8-1~exp1) experimental; urgency=low
.
* New upstream release
* Add dh_acc to generate and compare library dumps at build time,
together with addons for dh(7) and cdbs.
* Bump standards version, bump debhelper to 9, use 3.0 (quilt) format,
update Vcs-Svn field to canonical form, remove obsolete
DM-Upload-Allowed.
* Apply a patch to allow suffixes on a-c-c abi dumps.
Horay! So how does one use it?

  • build-depend on dh-acc
  • In your debian/rules
    • call dh_acc somewhere appropriate
    • dh $@ --with acc
    • include /usr/share/cdbs/1/rules/acc.mk
  • In your debian/libpackage-dev.acc
    • Write a abi-compliance-checker descriptor (no need to include version)
  • Build your package
  • Copy the generated /usr/lib/$(multiarch)/dh-acc/*.abi.tar.gz as ./debian/lib-package-dev.abi.tar.gz.ARCH
  • Now at each build ABI/API checks will be executed and compat reports will be generated
An example xml descriptor for libapt-pkg-dev is:
<?xml version="1.0" encoding="utf-8"?>
<descriptor>
<headers>
./build/include/apt-pkg/
</headers>
<libs>
./build/bin/
</libs>
<skip_types>
SubstVar
</skip_types>
</descriptor>
I wonder if things can be improved, for example:
  • For trivial packages, don't require .acc at all & simply point abi-compiance-checker at ./debian/tmp/
  • I think logs currently pollute the unpacked source package
  • Multiple "base" ABI need checking. E.g. given versions A, B, C one can introduce a new symbol in B and drop it in C. Both B & C are compatible with A, but C is not compatible with B, thus an API/ABI break got introduced into the distribution if all A,B,C were ever published in the archive.
  • Maybe run these as DEP-8 autopkgtests as well?
On a wider scope of things abi-compliance-checker allows to create "system" abi dumps & also check applications for ABI/API compliance. For example, one could scan deb packages to see if they are still compatible between major Debian Releases for some known install type (e.g. a default full Gnome Desktop), or we could scan proprietary packages to see if they are compatible (e.g. games delivered via Steam by Valve to multiple Ubuntu releases), and we can continuously monitor OS ABI to make sure we don't unknowingly break it with security and bugfix updates.

Please, play around with dh-acc and let me know what you think =)

21 April 2013

Lisandro Dami&aacute;n Nicanor P&eacute;rez Meyer: On the road to Qt 5: Qt 5 base, tools, jsbackend and xmlpatterns in experimental

The first Qt 5 packages have been accepted in Debian experimental.

What's there
To start building Qt 5 apps you will need to export QT_SELECT=qt5, install the package qt5-defaults or read qtchooser's man page. Note that exporting QT_SELECT has precedence over installing qt5-defaults.

What's not there
ArchitecturesAMD64 is already there because it's the arch used by maintainers to build the packages. i386 should be following as soon as buildds catch up. Most surely ARM-based archs will be there at some point too.

Other archs will need some love. Not strange, the Qt project supports amd64, i386 and ARM, but we Debian have normally prepared patches to make it build in other archs. And yes, we try to push them upstream for everyone's benefit. So, if you are missing it in your arch, take a look. You may be the one who enables Qt 5 in it :-)

GLES2 and WaylandWe don't have GLES2 or Wayland support yet. Building it will most probably break the desktop for people using proprietary video drivers (or at least I was told so). I'll surely provide non-official packages with GLES2/Wayland enabled to allow people testing it, but not soon.

This also means that we are not currently able to split X11 and framebuffer support. But we have time to work on it :-)

Non DFSGs compliant files
If you get the original source code tarball from Debian you will notice that it has dfsg in it's name. That means that we had to remove some non DFSG compliant stuff from the original tarball, namely:

  • Every RFC.
  • Three files used for testing the build, which are made of RFCs.
  • Some fonts.

What's following
Other parts of Qt 5 are on the way. And remember, this packages would not have been possible if it weren't for the great Debian's pkg-kde team. My kudos to them.

19 April 2013

Richard Hartmann: Release Critical Bug report for Week 16

Seems I was off by two weeks. Week 18, it is/should be. Yay. The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the Squeeze release cycle?
Week Squeeze Wheezy Diff
43 284 (213+71) 468 (332+136) +184 (+119/+65)
44 261 (201+60) 408 (265+143) +147 (+64/+83)
45 261 (205+56) 425 (291+134) +164 (+86/+78)
46 271 (200+71) 401 (258+143) +130 (+58/+72)
47 283 (209+74) 366 (221+145) +83 (+12/+71)
48 256 (177+79) 378 (230+148) +122 (+53/+69)
49 256 (180+76) 360 (216+155) +104 (+36/+79)
50 204 (148+56) 339 (195+144) +135 (+47/+90)
51 178 (124+54) 323 (190+133) +145 (+66/+79)
52 115 (78+37) 289 (190+99) +174 (+112/+62)
1 93 (60+33) 287 (171+116) +194 (+111/+83)
2 82 (46+36) 271 (162+109) +189 (+116/+73)
3 25 (15+10) 249 (165+84) +224 (+150/+74)
4 14 (8+6) 244 (176+68) +230 (+168/+62)
5 2 (0+2) 224 (132+92) +222 (+132/+90)
6 release! 212 (129+83) +212 (+129/+83)
7 release+1 194 (128+66) +194 (+128/+66)
8 release+2 206 (144+62) +206 (+144/+62)
9 release+3 174 (105+69) +174 (+105/+69)
10 release+4 120 (72+48) +120 (+72/+48)
11 release+5 115 (74+41) +115 (+74/+41)
12 release+6 93 (47+46) +93 (+47/+46)
13 release+7 50 (24+26) +50 (+24/+26)
14 release+8 51 (32+19) +51 (+32/+19)
15 release+9 39 (32+7) +39 (+32/+7)
16 release+10 20 (12+8) +20 (+12/+8)
17 release+11
18 release+12
Graphical overview of bug stats thanks to azhag:

16 April 2013

Wouter Verhelst: The devops "installer"

puppet and similar things are very nice tools for managing your systems. Rather than doing the same thing 100 times, it's much more interesting to spend some time describing how to do something, and then have it done hundreds of times automatically. However, I'm starting to see a distressing trend lately. I'd like to say that puppet is not an installer, nor is it a replacement for one. Here's why: Please, pretty please, with sugar on top: just provide packages. If that can't be done, just provide clear installation instructions and/or a puppet module that I can include or something. Don't assume I'm a system administrator not worth his paycheck. Thanks. note: the gitorious example above is just that, an example. I've seen more cases of people doing similar things, not just gitorious. Surprisingly, it's mostly from people who're on the ruby kool-aid. I hope that's not related... Update: (2013-04-18) despite my usage of buzzwords in the title of this post, I did not mean to imply that any of the above has anything to do with devops. Instead, what the post is supposed to say is that you should not fall into the trap of believing that devops methods, which may work wonderfully internal to your organization, will therefore also work wonderfully for processes outside your organization.

14 April 2013

Richard Hartmann: Release Critical Bug report for Week 15

This week: Monday 47, Tuesday 47, Wednesday 41, Thursday 39, Friday 37 Addendum: the bug count was at 37 around Friday noon and that's what I am tracking with the short stats above. The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the Squeeze release cycle?
Week Squeeze Wheezy Diff
43 284 (213+71) 468 (332+136) +184 (+119/+65)
44 261 (201+60) 408 (265+143) +147 (+64/+83)
45 261 (205+56) 425 (291+134) +164 (+86/+78)
46 271 (200+71) 401 (258+143) +130 (+58/+72)
47 283 (209+74) 366 (221+145) +83 (+12/+71)
48 256 (177+79) 378 (230+148) +122 (+53/+69)
49 256 (180+76) 360 (216+155) +104 (+36/+79)
50 204 (148+56) 339 (195+144) +135 (+47/+90)
51 178 (124+54) 323 (190+133) +145 (+66/+79)
52 115 (78+37) 289 (190+99) +174 (+112/+62)
1 93 (60+33) 287 (171+116) +194 (+111/+83)
2 82 (46+36) 271 (162+109) +189 (+116/+73)
3 25 (15+10) 249 (165+84) +224 (+150/+74)
4 14 (8+6) 244 (176+68) +230 (+168/+62)
5 2 (0+2) 224 (132+92) +222 (+132/+90)
6 release! 212 (129+83) +212 (+129/+83)
7 release+1 194 (128+66) +194 (+128/+66)
8 release+2 206 (144+62) +206 (+144/+62)
9 release+3 174 (105+69) +174 (+105/+69)
10 release+4 120 (72+48) +120 (+72/+48)
11 release+5 115 (74+41) +115 (+74/+41)
12 release+6 93 (47+46) +93 (+47/+46)
13 release+7 50 (24+26) +50 (+24/+26)
14 release+8 51 (32+19) +51 (+32/+19)
15 release+9 39 (32+7) +39 (+32/+7)
16 release+10
17 release+11
18 release+12
Graphical overview of bug stats thanks to azhag:

12 April 2013

Craig Small: Pie Charts in TurboGears

You might of looked at Ralph Bean s tutorial on graphs and thought, that s nice but I d like something different. The great thing about ToscaWidgets using the jqPlot library is that pretty much anything you can do in jqPlot, you can do in ToscaWidgets and by extension in TurboGears. You want different graph types? jqPlot has heaps! My little application needed a Pie Chart to display the overall status of attributes. There are 6 states an attribute can be in: Up, Alert, Down, Testing, Admin Down and Unknown. The Pie Chart would show them all with some sort of colour representing the status. For this example I ve used hard-coded data but you would normally extract it from the database using a TurboGears model and possibly some bucket sorting code. I ve divided my code into widget specific, which is found in myapp/widgets/attribute.py and the controller code found unsurprisingly at myapp/controllers/attribute.py Also note that some versions of ToscaWidgets have a bug which means any jqPlot widget won t work, version 2.0.4 has the fix for issue 80 that explains briefly the bug. The widget code looks like this:
from tw2.jqplugins.jqplot import JQPlotWidget
from tw2.jqplugins.jqplot.base import pieRenderer_js
import tw2.core as twc
 
class AttributeStatusPie(JQPlotWidget):
  """
  Pie Chart of the Attributes' Status """
  id = 'attribute-status-pie'
  resources = JQPlotWidget.resources + [
  pieRenderer_js,
  ]
 
  options =  
  'seriesColors': [ "#468847", "#F89406", "#B94A48", "#999999", "#3887AD", "#222222"], 
  'seriesDefaults' :  
  'renderer': twc.js_symbol('$.jqplot.PieRenderer'),
   ,
  'legend':  
  'show': True, 
  'location': 'e',
   ,
   
Some important things to note are: Next the controller needs to be defined:
from myapp.widgets.attribute import AttributeStatusPie
 
@expose('myapp.templates.widget')
  def statuspie(self):
  data = [[
  ['Up', 20], ['Alert', 7], ['Down', 12], ['Admin Down', 3], ['Testing', 1], ['Unknown', 4],
  ]]
  pie = AttributeStatusPie(data=data)
  return dict(w=pie)
And that is about it, we now have a controller path attributes/statuspie which shows us the pie chart.
My template is basically a bare template with a $ w.display n in it to just show the widget for testing.
Pie Chart in Turbogears

Pie Chart in Turbogears

11 April 2013

Olivier Berger: Slides + Manual + programs generated from single org-mode source

I ve been working on maintaining lecture slides and a manual, by writing a single source org-mode file. From a single source I want to be able to generate different output PDFs, only changing a few switches : The slides may contain notes. Here s an archive that contains an example document and complementary files. See this documentation document for more details (itself maintained with such an .org source).

10 April 2013

Lars Wirzenius: Machine-parseable embedded license summary information

There is no common way to express license information in each source file at the moment. Some people embed license information in each file, others keep it in README or another file at the top of the source tree. Worse, there is no common syntax to express the license information in a machine-parseable way. If we had this, we could have tools that, for example, tell you if you're trying to merge code that has an incompatible license. Obviously, this kind of thing can never work perfectly. People keep inventing new licenses, and it is not possible for a computer program to fully understand any license. It is not clear humans can do that, either. However, it would be possible to do it to a number of well-known licenses, which would help most of the time. A classic 20/80 situation. I would like to suggest a syntax, similar to Emacs's "Hey Emacs" modelines, for embedding a summary of the license, or licenses, for a source file, in such a way that it can be programmatically extracted and parsed and analysed. With this syntax, one could then write a tool to ensure that all files in a project have the same license, or that all licenses are compatible with the project's overall license. Such a tool will, of course, rely on heuristics and assumptions. For example, it needs to assume the machine-parseable license summary is correct, and rely on a ruleset on what licenses are compatible with what licenses. Things can go wrong. That's life. Remember, this is aiming at doing the 20% of the work that will work 80% of the time, not perfection. I don't have a tool written, but I have a suggestion for the syntax.
/*
 * Copyright 2013 Lars Wirzenius
 *
 * tl;dr =*= Licenses: GPL-3+ or Expat, and Artistic =*=
 *
 * Blah. Blah. Blah. Imagine long, boring license texts 
 * here.
 */
The important part is this:
=*= Licenses: GPL-3+ or Expat, and Artistic =*=
The =*= prefix and suffix and the word Licenses are there to make grepping reasonably reliable without too many false positives, and to allow comment characters and other text on the same line. The actual license summary follows the syntax and semantics of the Debian copyright-format 1.0 specification, which I chose because it exists and has had a fair bit of review so far, and is reasonably expressive. The license summaries can be extracted with the following GNU sed invocation:
sed -n '/.*=\*= [Ll]icen[cs]es\?: \(.*\)=\*=.*/s//\1/p'
I allowed various forms of the word license, since it's a word that a lot of people will get wrong, and it's easy to catch all four common forms. So, does anyone else think this might be useful? Would you use it in your own projects?

8 April 2013

Daniel Pocock: Google Summer of Code, tips for students

I've had quite a few queries about the project ideas for Google Summer of Code on the Lumicall and Debian project pages and I'm publishing some notes here to help people get the best chance of participating, while also helping people's expectations. Firstly, Google is responsible for administering the program and payments, they have the last word on which projects and students participate and free software developers like myself have no direct control over this. In other words, while I would encourage students to make contributions to free software projects to familiarize themselves with the way we work, I can not promise that this will automatically lead to selection. If you are a student, do go ahead and feel free to comment on or make contributions to projects, this helps mentors to see your capabilities very quickly, but don't do any more than you would have done if the GSoC program didn't exist. In any case, building up a portfolio of work that is visible on sites like github does help your employment prospects in the long run. If a development manager is looking at 100 CVs for a job vacancy, and one of them has a strong portfolio on github or Debian, there is a good chance that one person will get an interview - although it doesn't guarantee they get the job. Where to start? If you are a student and you are already heavily involved in some free software project then you probably don't need to read the rest of this. The reality is that GSoC is meant to help bring more people into the free software community and this means we need to help people get their foot in the door. That is what I am addressing here, with examples from my own possible projects. When I went through university in the 90s, there was a very strong UNIX culture in our computer science department and the wider engineering faculty. There were also winds of change, for example, a new dean who wanted to change everything to Windows and a central IT management agenda to abolish campus computers and force students to carry and maintain their own laptops. Around the world, every campus has a different approach. I was fortunate to have UNIX topics like the vi text editor, shell programming and even LaTeX taught as a formal part of the course, students coming after me may have missed those opportunities. I understand that some students have not encountered these concepts at all today. That said, most mentors have a fairly wide tolerance for these things: it is not expected that students know these things, rather, it is an opportunity for you to start learning them whether it is part of your course or not. The key concept here is to be a self-starter: it is not what you know, but your enthusiasm to go and try the things you don't know. That is the very thing you can start doing now, even before the program commences. Doing this will definitely help you identity the areas of free software and Debian that you are most likely to get excited about, and that will lead to a successful project. Some general ideas to get started
  • If your campus hasn't taught UNIX in a formal sense, please familiarise yourself with the basics. While some companies do have more jobs working with other platforms, and some campuses have sadly tried to adapt their courses to prepare students for that limited subset of companies, all the best jobs I've had involve UNIX or Linux. I've never worked for a hedge fund that didn't use Linux. Hollywood uses Linux, NASA does too, but learning Linux is not rocket science. Basic things to understand include the vi text editor, shell scripts and using a command line tool for version control, preferably git. As you will be the system administrator of your own desktop or laptop, it's essential to understand some basic concepts like volume management too.
  • If you need to run some Windows software (some courses do require that), use VirtualBox or a similar solution to run both Windows and Linux on your computer or laptop. The VirtualBox documentation is very good and will help you start quickly. My own laptop runs Debian 7 (wheezy) as the main operating system, and I have Windows and Solaris in VirtualBox for some business applications.
  • My own course included a project that involved downloading the open source GNU make program, writing a patch and testing it. If you've never done something like that, please try. Instead of downloading a project tarball, these days you will typically clone the project's git repository. Even if you only make some trivial change that you will never share, it is good to prove that you can compile and successfully execute something like this.
Some specific ideas for my own projects
  • For Lumicall or Android development, just work your way through the build instructions that are published for all contributors. Trying the f-droid build instructions is also a good way to start.
  • For any of the Debian projects, please make sure you have installed Debian 7 (wheezy) on a desktop, laptop or virtual machine. Just follow the installation guide
  • Once you have installed Debian, try installing whichever one of my packages interests you. Try to configure and use it. Prove that the package works in a normal installation. Look at the log files. Run the test cases.
  • Next, try obtaining the source code and building the package. Debian makes it very easy. Here is an example for reSIProcate, the same works for any other package. As root:
    # apt-get update
    # apt-get build-dep resiprocate
    
    will obtain all the necessary build tools. Then, as a normal user:
    $ mkdir ~/src/resiprocate
    $ cd ~/src/resiprocate
    $ apt-get source resiprocate
    $ cd resiprocate
    $ dpkg-buildpackage -rfakeroot -j12
    
    In a few minutes, you should find the packages (*.deb files) are created.
  • Now, you could try making some small change to the package and compiling it again. Verify that your change works.
If you've successfully got to this point, then there is every chance you will be able to successfully complete a GSoC project. If you struggled along the way, that is OK too, the important thing to do is to engage with the community and learn how to ask for help. The Debian GSoC wiki provides some excellent ideas about engaging with the project using IRC chat and mailing lists. It is also very worthwhile trying to find a local group or a free software event that Debian is attending near you and spending a few hours with people who can look at any problems you have and give you quick solutions.

5 April 2013

Richard Hartmann: Release Critical Bug report for Week 14

Over the week, around noon CEST, we had: The UDD bugs interface currently knows about the following release critical bugs: How do we compare to the Squeeze release cycle?
Week Squeeze Wheezy Diff
43 284 (213+71) 468 (332+136) +184 (+119/+65)
44 261 (201+60) 408 (265+143) +147 (+64/+83)
45 261 (205+56) 425 (291+134) +164 (+86/+78)
46 271 (200+71) 401 (258+143) +130 (+58/+72)
47 283 (209+74) 366 (221+145) +83 (+12/+71)
48 256 (177+79) 378 (230+148) +122 (+53/+69)
49 256 (180+76) 360 (216+155) +104 (+36/+79)
50 204 (148+56) 339 (195+144) +135 (+47/+90)
51 178 (124+54) 323 (190+133) +145 (+66/+79)
52 115 (78+37) 289 (190+99) +174 (+112/+62)
1 93 (60+33) 287 (171+116) +194 (+111/+83)
2 82 (46+36) 271 (162+109) +189 (+116/+73)
3 25 (15+10) 249 (165+84) +224 (+150/+74)
4 14 (8+6) 244 (176+68) +230 (+168/+62)
5 2 (0+2) 224 (132+92) +222 (+132/+90)
6 release! 212 (129+83) +212 (+129/+83)
7 release+1 194 (128+66) +194 (+128/+66)
8 release+2 206 (144+62) +206 (+144/+62)
9 release+3 174 (105+69) +174 (+105/+69)
10 release+4 120 (72+48) +120 (+72/+48)
11 release+5 115 (74+41) +115 (+74/+41)
12 release+6 93 (47+46) +93 (+47/+46)
13 release+7 50 (24+26) +50 (+24/+26)
14 release+8 51 (32+19) +51 (+32/+19)
15 release+9
16 release+10
17 release+11
18 release+12
Graphical overview of bug stats thanks to azhag:

Daniel Pocock: Quickstart: Integrating SIP support into any application with reSIProcate

Today's applications are more dynamic and interactive than ever before, typically offering the user many ways to collaborate or get live data about their community or environment as they need it. Two open standards exist with a broad focus on real-time messaging and session control, SIP and XMPP (Jabber). Both of them have been supercharged by the development of technologies like WebRTC that makes every browser into a communications endpoint. Here, we look at integrating SIP into any C++ application (either client or server). We use the industry standard SIP stack reSIProcate. The quickest way to get started is with a Debian, Ubuntu or Fedora development environment. reSIProcate is conveniently packaged on all those platforms (for Fedora, it is not yet in yum and must be built from the tarball using rpmbuild). Getting the environment set up Once you have Debian or Ubuntu installed, just do:
# apt-get build-dep resiprocate
# apt-get install libresiprocate-1.8-dev
This will install all the necessary tools, libraries and header files. Headers are under /usr/include/resip Writing some code The object orientated nature of C++ is heavily leveraged by reSIProcate, and this makes it easy to code. The stack is based on a callback paradigm: you create some objects based on well-defined interfaces, give your objects to the stack, and put the stack into a loop. The stack will call your object when some event occurs (e.g. new incoming call, new text message) The easiest way to see this in practice is to browse the test cases. The test cases under the resip/dum(mirrored on github) section of the source tree provide good high-level examples of the type of code an application writes when using reSIProcate in an existing application. Sending a SIP text message Based on the basicMessage test case, I've created a trivial piece of code below to send a text message over SIP. The whole demo project is uploaded into github, see the README for instructions on how to build and run. Here, we have a quick look at the one object in the demo, it is notified on the progress of the SIP text message:
class ClientMessageHandler : public ClientPagerMessageHandler  
public:
   ClientMessageHandler()
      : finished(false),
        successful(false)
    
    ;
   virtual void onSuccess(ClientPagerMessageHandle, const SipMessage& status)
    
      InfoLog(<<"ClientMessageHandler::onSuccess\n");
      successful = true;
      finished = true;
    
   virtual void onFailure(ClientPagerMessageHandle, const SipMessage& status, std::auto_ptr<contents> contents)
    
      ErrLog(<<"ClientMessageHandler::onFailure\n");
      successful = false;
      finished = true;
    
   bool isFinished()   return finished;  ;
   bool isSuccessful()   return successful;  ;
private:
   bool finished;
   bool successful;
 ;
int main(int argc, char *argv[])
 
   // boilerplate initialisation code...
...
   NameAddr naTo(to.c_str());
   ClientPagerMessageHandle cpmh = clientDum.makePagerMessage(naTo);
   Data messageBody("Hello world!");
   auto_ptr<contents> content(new PlainContents(messageBody));
   cpmh.get()->page(content);
...
 
</contents></contents>
Going further with reSIProcate

4 April 2013

Bernhard R. Link: Git package workflows

Given the recent discussions on planet.debian.org I use the opportunity to describe how you can handle upstream history in a git-dpm workflow. One of the primary points of git-dpm is that you should be able to just check out the Debian branch, get the .orig.tar file(s) (for example using pristine-tar, by git-dpm prepare or by just downloading them) and then calling dpkg-buildpackage. Thus the contents of the Debian branch need to be clean from dpkg-source's point of view, that is do not contain any files the .orig.tar file(s) contains not nor any modified files. The easy way The easiest way to get there is by importing the upstream tarball(s) as a git commit, which one will usually do with git-dpm import-new-upstream as that also does some of the bookkeeping. This new git commit will have (by default) the previous upstream commit and any parent you give with -p as parents. (i.e. with -p it will be a merge commit) and its content will be the contents of the tarball (with multiple orig files, it gets more complicated). The idea is of course that you give the upstream tag/commit belonging to this release tarball with -p so that it becomes part of your history and so git blame can find those commits. Thus you get a commit with the exact orig contents (so pristine-tar can more easily create small deltas) and the history combined.. Sometimes there are files in the upstream tarball that you do not want to have in your Debian branch (as you remove them in debian/rules clean), then when using this method you will have those files in the upstream branch but you delete them in the Debian branch. (This is why git-dpm merge-patched (the operation to merge a new branch with upstream + patches with your previous debian/ directory) will look which files relative to the previous upstream branch are deleted and delete them also in the newly merged branch by default). The complicated way There is only a way without importing the .orig.tar file(s), though that is a bit more complicated: The idea is that if your upstream's git repository contains all the files needed for building your Debian package (for example if you call autoreconf in your Debian package and clean all the generated files in the clean target, or if upstream has a less sophisticated release process and their .tar contains only stuff from the git repository), you can just use the upstream git commit as base for your Debian branch. Thus you can make upstream's commit/tag your upstream branch, by recording it with git-dpm new-upstream together with the .orig.tar it belongs to (Be careful, git-dpm does not check if that branch contains any files different than your .orig.tar and could not decide if it misses any files you need to build even if it tried to tell). Once that is merged with the debian/ directory to create the Debian branch, you run dpkg-buildpackage, which will call dpkg-source which compares your working directory with the contents of the .orig.tar with the patches applied. As it will only see files not there but no files modified or added (if everything was done correctly), one can work directly in the git checkout without needing to import the .orig.tar files at all (altough the pristine-tar deltas might get a bit bigger).

Daniel Pocock: Comparing packaging workflows in Debian and beyond

My blog post yesterday about Debian's git packaging workflows was meant to help fill some gaps in the documentation of this process, one of the most significant being the diagram. I was surprised to find that some people felt my post argues that distribution tarballs are a must-have or that separate repositories are the optimal solution. In fact, my post was more a reflection of how things are being done, and it is great to see contributions from Joey, Russ and Thomas proposing ways to integrate the workflow in a single repository and also raising questions about the future of tarballs. Some of the changes outlined by Russ only entered the tools after many people were already using the workflow I have described. Given the rise of collaboration, through collab-maint and packaging teams, it is more important than ever that workflows and tools are easily understood and documented. This lets new contributors (including people new to Debian) jump in more quickly and with less risk of disruption. It would be great to see some of these latest ideas covered more thoroughly with diagrams and I'm happy for people to rip-off my dia source file for my own diagram and amend it as they please (under the generous terms of the GPL v3) A look at other packaging systems With the role of the tarball in people's sights, it's worth looking outside Debian for a moment:
  • Android packaging is radically different. All packages have a single integer version number. The pretty version number displayed to the user is meaningless. Due to the continuously increasing integer version, and the way that database schemas are versioned, it is not really possible to even keep multiple release branches in a repository. For an example, see the version in the Lumicall AndroidManifest.xml and the database upgrade function. Google Play/Android Market makes no rules about how a project manages its code. f-droid, the open source market for apps, builds projects like Lumicall directly from git, it relies on them having an ant build file in the standard format generated by the SDK.
  • OpenCSW does not keep upstream tarballs at all. They keep a single git repository for tracking all packages. Each package is built from a Makefile (sample), and their tool suite takes care of downloading the upstream sources (using a URL specified in the Makefile) and verifying against checksums tracked in the repository (sample). The common style of the Makefiles makes it very easy for somebody familiar with the tools to work on just about any of the packages, and anybody who knows how to write a Makefile can start quickly.
Of course, these are just a few examples. These days, software is distributed in many different ways, whether it is a runnable VM image or an embedded device implanted during surgery.

Thomas Goirand: Git packaging workflow

Seeing what has been posted recently in planet.d.o, I would like to share as well my thoughts and work-flow, and tell that I do agree with Joey Hess on many of his arguments. Especially when he tells that Debian fetishises upstream tarballs. We re in 2013, at the age of Internet, and more and more upstream authors are using Git, and more and more they don t care about releasing tarballs. I ve seen some upstream authors who simply stopped doing so completely, as a Git tag is really enough. I also fully agree than disk space and network speed isn t much of a problem these days. When there are tags available, I use the following debian/gbp.conf:
[DEFAULT]
upstream-branch = master
debian-branch = debian
upstream-tag = %(version)s
compression = xz
[git-buildpackage]
export-dir = ../build-area/
On many of my packages, I now just use Git tags from upstream if they are available. To make it more easy, I now nearly always use the following piece of code in my debian/rules files:
DEBVERS         ?= $(shell dpkg-parsechangelog   sed -n -e 's/^Version: //p')
VERSION         ?= $(shell echo '$(DEBVERS)'   sed -e 's/^[[:digit:]]*://' -e 's/[-].*//')
DEBFLAVOR       ?= $(shell dpkg-parsechangelog   grep -E ^Distribution:   cut -d" " -f2)
DEBPKGNAME      ?= $(shell dpkg-parsechangelog   grep -E ^Source:   cut -d" " -f2)
DEBIAN_BRANCH   ?= $(shell cat debian/gbp.conf   grep debian-branch   cut -d'=' -f2   awk ' print $1 ')
GIT_TAG         ?= $(shell echo '$(VERSION)'   sed -e 's/~/_/')
get-upstream-sources:
        git remote add upstream git://git.example.org/proj/foo.git   true
        git fetch upstream
        if ! git checkout master ; then \
                echo "No upstream branch: checking out" ; \
                git checkout -b master upstream/master ; \
        fi
        git checkout $(DEBIAN_BRANCH)
make-orig-file:
        if [ ! -f ../$(DEBPKGNAME)_$(VERSION).orig.tar.xz ] ; then \
                git archive --prefix=$(DEBPKGNAME)-$(GIT_TAG)/ $(GIT_TAG)   xz >../$(DEBPKGNAME)_$(VERSION).orig.tar.xz ; \
        fi
        [ ! -e ../build-area ] && mkdir ../build-area   true
        [ ! -e ../build-area/$(DEBPKGNAME)_$(VERSION).orig.tar.xz ] && cp ../$(DEBPKGNAME)_$(VERSION).orig.tar.xz ../build-area   true
Packaging a new upstream VERSION now means that I only have to edit the debian/changelog, do ./debian/rules get-upstream-source so that I get new commits and tags, then git merge -X theirs VERSION to import the changes, then finally invoke ./debian/rules make-orig-file to create the orig.tar.xz. My debian branch is now ready for git-buildpackage. Note that the sed with the GIT_TAG thing is there because unfortunately, Git doesn t support the ~ char in tags, and that most of the time, upstream do not use _ in version numbers. Let s say upstream is releasing version 1.2.3rc1, then I simply do git tag 1.2.3_rc1 1.2.3rc1 so that I have a new tag which points to the same commit as 1.2.3rc1, but that can be used for the Debian 1.2.3~rc1-1 release and the make-orig-file. All this might looks overkill at first, but in fact it is really convenient and efficient. Also, even though there is a master branch above, it isn t needed to build the package. Git is smarter than this, so even if you haven t checked out upstream master branch from the upstream remote, make-orig-file and git-buildpackage will simply continue to work. Which is cool, because this means you can store a single branch on Alioth (which is what I do).

Next.

Previous.