Search Results: "xam"

31 July 2015

Scott Kitterman: Plasma 5 (KDE) In Testing

A few days ago, fellow Qt/KDE team member Lisandro gave an update on the situation with migration to Plasma 5 in Debian Testing (AKA Stretch). It s changed again. All of Plasma 5 is now in Testing. The upgrade probably won t be entirely smooth, which we ll work on that after the gcc5 transition is done, but it will be much better than the half KDE4 SC half Kf5/Plasma 5 situation we ve had for the last several days. The issues with starting kwin should be resolved once users upgrade to Plasma 5. To use the current kwin with KDE SC 4, you will need to add a symlink from /usr/bin/kwin to /usr/bin/kwin_x11. That will be included in the next upload after gcc5. Systemsettings and plasma-nm now work. In my initial testing, I didn t see anything major that was broken. One user reported an issue with sddm starting automatically, but it worked fine for me. During the upgrade you should get a debconf prompt asking if you want to use kdm or sddm. Pick sddm. When I tried to dist-upgrade, apt wanted to remove task-kde-desktop. I let it remove it and some other packages and then in a second step did apt-get install task-kde-desktop. That pulled it back in successfully along with adding and removing a reasonably large stack of packages. Obviously we need to make that work better before Stretch is released, but as long as you don t restart KDE in between those two steps it should be fine. Lastely, I used apt-get autoremove to clear out a lot of no longer needed KDE4 things (when it asks if you want to stop the running kdm, say no). Here are a few notes on terminology and what I understand of the future plans: What used to be called KDE is now three different things (in part because KDE is now the community of people, not the software): KDE Frameworks 5 (Kf5): This is a group of several dozen small libraries that as a group, roughly equate to what used to be kdelibs. Plasma (Workspaces) 5: This is the desktop that we ve just transitioned to. Applications: These are a mix of kdelibs and Kf5 based applications. Currently in Testing there are some of both and this will evolve over time based on upstream development. As an example, the Kf5 based version of konsole is in Unstable and should transition to Testing shortly. Finally, thanks to Maximiliano Curia (maxy on IRC) for doing virtually all of the packaging of Kf5, Plasma 5, and applications. He did the heavy lifting, the rest of us just nibbled around the edges to keep it moving towards testing.

Raphaël Hertzog: My Free Software Activities in July 2015

My monthly report covers a large part of what I have been doing in the free software world. I write it for my donators (thanks to them!) but also for the wider Debian community because it can give ideas to newcomers and it s one of the best ways to find volunteers to work with me on projects that matter to me. Debian LTS This month I have been paid to work 15 hours on Debian LTS. In that time I did the following: Kali Linux / Debian Stretch work kaliKali Linux wants to experiment something close to Debian Constantly Usable Testing: we have a kali-rolling release that is based on Debian Testing and we want to take a new snapshot every 4 months (in order to have 3 releases per year). More specifically we have a kali-dev repository which is exactly Debian Stretch + our own Kali packages (the kali package take precedence) updated 4 times a day, just like testing is. And we have a britney2 setup that generates kali-rolling out of kali-dev (without any requirement in terms of delay/RC bugs, it just ensures that dependencies are not broken), also 4 times a day. We have jenkins job that ensures that our metapackages are installable in kali-dev (and kali-rolling) and that we can build our ISO images. When things break, I have to fix them and I try to fix them on the Debian side first. So here are some examples of stuff I did in response to various failures: If you analyze this list, you will see that a large part of the issues we had come down to package getting removed from testing due to RC bugs. We should be able to anticipate those issues and monitor the packages that have an impact on Kali. We will probably add new jenkins job that installs all the metapackages and then run how-can-i-help -s testing-autorm --old I just submitted #794238 as a wishlist against how-can-i-help. At the same time, there are bugs that make it into testing and that I fix / work around on the Kali side. But those fixes / work around might be more useful if they were pushed to testing via testing-proposed-updates. I tried to see whether other derivatives had similar needs to see if derivatives could join their efforts at this level but it does not look like so for now. Last but not least, bugs reported on the Kali side also resulted in Debian improvements: Other Debian work Sponsorship. I sponsored multiple packages for Daniel Stender who is packaging prospector, a software that I requested earlier (through RFP bug). So I reviewed and uploaded python-requirements-detector, python-setoptconf, pylint-celery and pylint-common. During a review I also discovered a nice bug in dh-python (#793609a comment in the middle of a Build-Depends could break a package). I also sponsored an upload of notmuch-addrlookup (new package requested by a Freexian customer). Packaging. I uploaded python-django 1.7.9 in unstable and 1.8.3 in experimental to fix security issues. I uploaded a new upstream release of ditaa through a non-maintainer uploaded (again at the request of a Freexian customer). Distro Tracker. Beside the work to integrate detailed security status, I fixed the code to be compatible with Django 1.8 and modified the tox configuration to ensure that the test suite is regularly run against Django 1.8. I also merged multiple patches of Christophe Siraut (cf #784151 and #754413). Thanks See you next month for a new summary of my activities.

2 comments Liked this article? Click here. My blog is Flattr-enabled.

24 July 2015

Vincent Sanders: NetSurf developers and the Order of the Phoenix

Once more the NetSurf developers gathered to battle the forces of darkness, or as they are more commonly known web specifications.

Michael Drake, Vincent Sanders, John-Mark Bell and Daniel Silverstone at the Codethink manchester officesThe fifth developer weekend was an opportunity for us to gather in a pleasant setting and work together in person. We were graciously hosted, once again, by Codethink in their Manchester offices.

Four developers managed to attend in person from around the UK: Michael Drake, John-Mark Bell, Daniel Silverstone and Vincent Sanders.

The main focus of the weekends activities was to address two areas that have become overwhelmingly important: JavaScript and Layout.

Although the browser obviously already has both these features they are somewhat incomplete and incapable of supporting the features of the modern web.
JavaScriptThe discussion started with JavaScript and its implementation. We had previously looked at the feasibility of changing our JavaScript engine from Spidermonkey to DukTape. We had decided this was a change we wanted to make when DukTape was mature enough to support the necessary features.

The main reasons for the change are that Spidermonkey is a poor fit to NetSurf as it is relatively large and does not provide a stable API guarantee. The lack of a stable API requires extensive engineering to update to new releases. Additionally support for compiling on minority platforms is very challenging meaning that most platforms are stuck using version 1.7 or 1.85 (current release is version 31 with 38 due).

We started the move to Duktape by creating a development branch, integrating the Duktape library and open coding a minimal implementation of the core classes as a proof of concept. This work was mostly undertaken by Daniel with input from myself and John-Mark. This resulted in a build that was substantially smaller than using Spidermonkey with all the existing functionality our tests cover.

The next phase of this work is to take the prototype implementation and turn it into something that can be reliably used and covers the entire JavaScript DOM interface. This is no small job as there are at least 200 classes and 1500 methods and properties to implement.
LayoutThe layout library design discussion was an extensive and very involved. The layout engine is a browsers most important component. It takes all the information processed by the CSS and DOM libraries, applies a vast number of involved rules and produces a list of operations that can be rendered.

This reimplementation of our rendering engine has been in planning for many years. The existing engine stems from the browsers earliest days more than a decade ago and has many shortcomings in architecture and implementation we hope to address.

The work has finally started on libnslayout with Michael taking the lead and defining the initial API and starting the laborious work of building the test harness, a feature the previous implementation lacked!
The second war beginsFor a war you need people and it is a little unfortunate that this was our lowest ever turnout for the event. This is true of the project overall with declining numbers of commits and interest outside our core group. If anyone is interested we are always happy to have new contributors and there are opportunities to contribute in many areas from image assets, through translations, to C programming.

We discussed some ways to encourage new developers and try and get committed developers especially for the minority platform frontends. The RISC OS frontend for example has needed a maintainer since the previous one stepped down. There was some initial response from its community, culminating in a total of two patches, when we announced the port was under threat of not being releasable in future. Unfortunately nothing further came from this and it appears our oldest frontend may soon become part of our history.

We also covered some issues from the bug tracker mostly to see if there were any patterns that we needed to address before the forthcoming 3.4 release.

There was discussion about recent improvements to the CI system which generate distribution packages from the development branch and how this could be extended to benefit more users. This also included authorisation to acquire storage and other miscellaneous items necessary to keep the project infrastructure running.

We managed over 20 hours of work in the two days and addressed our current major shortcomings. Now it just requires a great deal of programming to complete the projects started here.

23 July 2015

Antoine Beaupr : Is it safe to use open wireless access points?

I sometimes get questions when people use my wireless access point, which, for as long as I can remember, has been open to everyone; that is without any form of password protection or encryption. I arguably don't use the access point much myself, as I prefer the wired connection for the higher bandwidth, security and reliability it provides. Apart from convenience for myself and visitors, the main reason why I leave my wireless access open is that I believe in a free (both as in beer and freedom) internet, built with principles of solidarity rather than exploitation and profitability. In these days of ubiquitous surveillance, freedom often goes hand in hand with anonymity, which implies providing free internet access to everyone. I also believe that, as more and more services get perniciously transferred to the global internet, access to the network is becoming a basic human right. This is therefore my small contribution to the struggle, now also part of the R seau Libre project. So here were my friends question, in essence:
My credit card info was stolen when I used a wifi hotspot in an airport... Should I use open wifi networks? Is it safe to use my credit card for shopping online?
Here is a modified version of an answer I sent to a friend recently which I thought could be useful to the larger internet community. The short answer is "sorry about that", "it depends, you generally can, but be careful" and "your credit card company is supposed to protect you".

Sorry! First off, sorry to hear that our credit card was stolen in an airport! That has to be annoying... Did the credit card company reimburse you? Normally, the whole point of credit cards is that they protect you in case of theft like this and they are supposed to reimburse you if you credit card gets stolen or abused...

The complexity and unreliability of passwords Now of course, securing every bit of your internet infrastructure helps in protecting against such attacks. However: there is a trade-off! First off, it does makes it more complicated for people to join the network. You need to make up some silly password (which has its own security problems: passwords can be surprisingly easy to guess!) that you will post on the fridge or worst, forget all the time! And if it's on the fridge, anyone with a view to that darn fridge, be it one-time visitor or sneaky neighbor, can find the password and steal your internet access (although, granted, that won't allow them to directly spy on your internet connection). In any case, if you choose to use a password, you should use the tricks I wrote in the koumbit wiki to generate the password and avoid writing it on the fridge.

The false sense of security of wireless encryption Second, it can also give a false sense of security: just because a wifi access point appears "secure" (ie. that the communication between your computer and the wifi access point is encrypted) doesn't mean the whole connection is secure. In fact, one attack that can be done against access points is exactly to masquerade as an existing access point, with no security security at all. That way, instead of connecting to the real secure and trusted access point, you connect to an evil one which spies on our connection. Most computers will happily connect to such a hotspot even with degraded security without warning. It may be what happened at the airport, in fact. Of course this particular attack would be less likely to happen if you live in the middle of the woods than an airport, but it's some important distinction to keep in mind, because the same attack can be performed after the wireless access point, for example by your countryside internet access provider or someone attacking it. Your best protection for your banking details is to rely on good passwords (for your back account) but also, and more importantly, what we call end-to-end encryption. That is usually implemented using the "HTTPS" with a pad lock icon in your address bar. This ensures that the communication between your computer and the bank or credit card company is secure, that is: that no wifi access point or attacker between your computer and them can intercept your credit card number.

The flaws of internet security Now unfortunately, even the HTTPS protocol doesn't bring complete security. For example, one attack that can be done is similar to the previous one and that is to masquerade as a legitimate bank site, but either strip out the encryption or even fake the encryption. So you also need to look at the address of the website you are visiting. Attackers are often pretty clever and will use many tricks to hide the real address of the website in the address bar. To work around this, I always explicitly type my bank website address (https://accesd.desjardins.com/ in my case) directly myself instead of clicking on links, bookmarks or using a search engine to find my bank site. In the case of credit cards, it is much trickier because when you buy stuff online, you end up putting that credit card number on different sites which you do not necessarily trust. There's no good solution but complaining to your credit card company if you believe a website you used has stolen your credit card details. You can also use services like Paypal, Dwolla or Bitcoin that hide your credit card details from the seller, if they support the service. I usually try to avoid putting my credit card details on sites I do not trust, and limit myself to known parties (e.g. Via Rail, Air Canada, etc). Also, in general, I try to assume the network connection between me and the website I visit is compromised. This forced me to get familiar with online security and use of encryption. It is more accessible to me than trying to secure the infrastructure i am using, because i often do not control it at all (e.g. internet cafes...). Internet security is unfortunately a hard problem, and things are not getting easier as more things move online. The burden is on us programmers and system administrators to create systems that are more secure and intuitive for our users so, as I said earlier, sorry the internet sucks so much, we didn't think so many people would join the acid trip of the 70s. ;)

Daniel Pocock: Recording live events like a pro (part 1: audio)

Whether it is a technical talk at a conference, a political rally or a budget-conscious wedding, many people now have most of the technology they need to record it and post-process the recording themselves. For most events, audio is an essential part of the recording. There are exceptions: if you take many short clips from a wedding and mix them together you could leave out the audio and just dub the couple's favourite song over it all. For a video of a conference presentation, though, the the speaker's voice is essential. These days, it is relatively easy to get extremely high quality audio using a lapel microphone attached to a smartphone. Lets have a closer look at the details. Using a lavalier / lapel microphone Full wireless microphone kits with microphone, transmitter and receiver are usually $US500 or more. The lavalier / lapel microphone by itself, however, is relatively cheap, under $US100. The lapel microphone is usually an omnidirectional microphone that will pick up the voices of everybody within a couple of meters of the person wearing it. It is useful for a speaker at an event, some types of interviews where the participants are at a table together and it may be suitable for a wedding, although you may want to remember to remove it from clothing during the photos. There are two key features you need when using such a microphone with a smartphone:
  • TRRS connector (this is the type of socket most phones and many laptops have today)
  • Microphone impedance should be at least 1k (that is one kilo Ohm) or the phone may not recognize when it is connected
Many leading microphone vendors have released lapel mics with these two features aimed specifically at smartphone users. I have personally been testing the Rode smartLav+ Choice of phone There are almost 10,000 varieties of smartphone just running Android, as well as iPhones, Blackberries and others. It is not practical for most people to test them all and compare audio recording quality. It is probably best to test the phone you have and ask some friends if you can make test recordings with their phones too for comparison. You may not hear any difference but if one of the phones has a poor recording quality you will hopefully notice that and exclude it from further consideration. A particularly important issue is being able to disable AGC in the phone. Android has a standard API for disabling AGC but not all phones or Android variations respect this instruction. I have personally had positive experiences recording audio with a Samsung Galaxy Note III. Choice of recording app Most Android distributions have at least one pre-installed sound recording app. Look more closely and you will find not all apps are the same. For example, some of the apps have aggressive compression settings that compromise recording quality. Others don't work when you turn off the screen of your phone and put it in your pocket. I've even tried a few that were crashing intermittently. The app I found most successful so far has been Diktofon, which is available on both F-Droid and Google Play. Diktofon has been designed not just for recording, but it also has some specific features for transcribing audio (currently only supporting Estonian) and organizing and indexing the text. I haven't used those features myself but they don't appear to cause any inconvenience for people who simply want to use it as a stable recording app. As the app is completely free software, you can modify the source code if necessary. I recently contributed patches enabling 48kHz recording and disabling AGC. At the moment, the version with these fixes has just been released and appears in F-Droid but not yet uploaded to Google Play. The fixes are in version 0.9.83 and you need to go into the settings to make sure AGC is disabled and set the 48kHz sample rate. Whatever app you choose, the following settings are recommended:
  • 16 bit or greater sample size
  • 48kHz sample rate
  • Disable AGC
  • WAV file format
Whatever app you choose, test it thoroughly with your phone and microphone. Make sure it works even when you turn off the screen and put it in your pocket while wearing the lapel mic for an hour. Observe the battery usage. Gotchas Now lets say you are recording a wedding and the groom has that smartphone in his pocket and the mic on his collar somewhere. What is the probability that some telemarketer calls just as the couple are exchanging vows? What is the impact on the recording? Maybe some apps will automatically put the phone in silent mode when recording. More likely, you need to remember this yourself. These are things that are well worth testing though. Also keep in mind the need to have sufficient storage space and to check whether the app you use is writing to your SD card or internal memory. The battery is another consideration. In a large event where smartphones are being used instead of wireless microphones, possibly for many talks in parallel, install a monitoring app like Ganglia on the phones to detect and alert if any phone has weak wifi signal, low battery or a lack of memory. Live broadcasts and streaming Some time ago I tested RTP multicasting from Lumicall on Android. This type of app would enable a complete wireless microphone setup with live streaming to the internet at a fraction of the cost of a traditional wireless microphone kit. This type of live broadcast could also be done with WebRTC on the Firefox app. Conclusion If you research the topic thoroughly and spend some time practicing and testing your equipment, you can make great audio recordings with a smartphone and an inexpensive lapel microphone. In subsequent blogs, I'll look at tips for recording video and doing post-production with free software.

20 July 2015

Daniel Pocock: RTC status on Debian, Ubuntu and Fedora

Zoltan (Zoltanh721) recently blogged about WebRTC for the Fedora community and Fedora desktop. https://fedrtc.org has been running for a while now and this has given many people a chance to get a taste of regular SIP and WebRTC-based SIP. As suggested in Zoltan's blog, it has convenient integration with Fedora SSO and as the source code is available, people are welcome to see how it was built and use it for other projects. Issues with Chrome/Chromium on Linux If you tried any of FedRTC.org, rtc.debian.org or meet.jit.si using Chrome/Chromium on Linux, you may have found that the call appears to be connected but there is no media. This is a bug and the Chromium developers are on to it. You can work around this by trying an older version of Chromium (it still works with v37 from Debian wheezy) or Firefox/Iceweasel. WebRTC is not everything WebRTC offers many great possibilities for people to quickly build and deploy RTC services to a large user base, especially when using components like JSCommunicator or the DruCall WebRTC plugin for Drupal. However, it is not a silver bullet. For example, there remain concerns about how to receive incoming calls. How do you know which browser tab is ringing when you have many tabs open at once? This may require greater browser/desktop integration and that has security implications for JavaScript. Whether users on battery-powered devices can really leave JavaScript running for extended periods of time waiting for incoming calls is another issue, especially when you consider that many web sites contain some JavaScript that is less than efficient. Native applications and mobile apps like Lumicall continue to offer the most optimized solution for each platform although WebRTC currently offers the most convenient way for people to place a Call me link on their web site or portal. Deploy it yourself The RTC Quick Start Guide offers step-by-step instructions and a thorough discussion of the architecture for people to start deploying RTC and WebRTC on their own servers using standard packages on many of the most popular Linux distributions, including Debian, Ubuntu, RHEL, CentOS and Fedora.

19 July 2015

Laura Arjona: Family games: Robots

I play Robots with my kid. I ve tested the game with other kids and it seems that for ages 5 to 7 they like it. I ve talked about the game to several adults and it seems they like too, so I thought maybe writing about it here may be useful for somebody to enjoy some summer days. Players One player is the Robot. The other one is the programmer. If there are more players, it can be several robots and several programmers. If players are older, you can make the game more complicated making robots cooperate or programmers cooperate. If not, you make pairs 1-1 or 1 programmer 2 robots if the number is odd. The game The programmer must turn on the robot, pressing the ON/OFF button (robot chooses where s the button: nose, ear, belly, whatever).
Then, the robot say hello , and the programmer asks for the list of commands available (like Hello, robot, give me the list of commands ). The robot says the list of commands available, for example Run, stop, jump, sing a song, somersault, say something in a different language . Then, the programmer thinks a program, and loads it to the robot (speaks the list of orders, loudly, to the robot). Then the programmer presses the START button (Robot choses where it is) and then the robot has to perform the program without errors. If the robot performs correctly, wins one point. If it fails, looses one point. The programmer can design another program (maybe longer, maybe with some conditional expression) and tries the limits of the memory of robot. If the robot is tired, needs to charge batteries, or whatever, the roles programmer/robot are interchanged, and the one with more points in a certain amount of time or rounds, wins. Variants, tips If the programmer does not like the list, of commands, she can ask for updates, and maybe some new commands will be installed (and/or other uninstalled, who knows). Please be creative with the list of commands, or the game will be very boring. Depending on the operating system which runs the robot, it will give more or less options to the programmer, and the behaviour will be more evil or good. Robots shouldn t behave too much evil, though, otherwise the programmer will erase their disk and install Debian on them to make them obedient ;) You can play with a third person being the Robot manufacturer, who controls the robot, even sometimes overriding the programmer instructions (if the robot has an OS which is not free software). Robot will win one point obeying the manufacturer, but if there are more robots, will loose one round of playing because the programmer got angry and turned it off or reinstalled the software. The manufacturer and the programmer cooperate if the robot runs free software, though. Together they can expand robot memory (for example, lend a piece of paper where to store the program), or create new commands, fix bugs, or whatever. Comments? You can comment about this post in Pump.io thread about this post.
Filed under: My experiences and opinion Tagged: Debian, Education, English, Free culture, Free Software, Games, kids

16 July 2015

Petter Reinholdtsen: Typesetting DocBook footnotes as endnotes with dblatex

I'm still working on the Norwegian version of the Free Culture book by Lawrence Lessig, and is now working on the final typesetting and layout. One of the features I want to get the structure similar to the original book is to typeset the footnotes as endnotes in the notes chapter. Based on the feedback from the Debian maintainer and the dblatex developer, I came up with this recipe I would like to share with you. The proposal was to create a new LaTeX class file and add the LaTeX code there, but this is not always practical, when I want to be able to replace the class using a make file variable. So my proposal misuses the latex.begindocument XSL parameter value, to get a small fragment into the correct location in the generated LaTeX File. First, decide where in the DocBook document to place the endnotes, and add this text there: 
<?latex \theendnotes ?>
Next, create a xsl stylesheet file dblatex-endnotes.xsl to add the code needed to add the endnote instructions in the preamble of the generated LaTeX document, with content like this: 
<?xml version='1.0'?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version='1.0'>
  <xsl:param name="latex.begindocument">
    <xsl:text>
\usepackage endnotes 
\let\footnote=\endnote
\def\enoteheading \mbox \par\vskip-\baselineskip  
\begin document 
    </xsl:text>
  </xsl:param>
</xsl:stylesheet>
Finally, load this xsl file when running dblatex, for example like this: 
dblatex --xsl-user=dblatex-endnotes.xsl freeculture.nb.xml
The end result can be seen on github, where my book project is located.

Osamu Aoki: debmake and debamke-doc and debian-handbook

While updating the maintainer's guide, I had faced with several issues to update it as it is, since it was based on the dh-make package and does not play nice with new features such as multi-arch etc. Also. the lack of capability to create a decent copyright information template was my concern.

I have created the debmake tool to address shortcomings of dh-make. It was good attempt for Jessie, but it has not so nice bug on parsing copyright. So I decided to drop it for Jessie.
https://packages.qa.debian.org/d/debmake.html

The current testing repository has the updated debmake command. With the -k option, you can check if the your debian/copyright is covering all the files properly in source or not.

Since I got this updated package, I rewrote the maintainer's guide from scratch. Now it is available in the debmake-doc package. You can see all the packaging cases including autotools, cmake, and libraries. If I made a mistake, let me know. As you may noticed, debian-doc related web pages are now generated from the latest uploaded packages mostly. debmake-doc is no exception.
"Guide for Debian Maintainers"
https://www.debian.org/doc/manuals/debmake-doc/index.en.html

By using debmake as the template generation tool, the document can be made shorter and there are many practical examples. More over, all the source code examples can be found in the source tree. Try:

$ apt-get source debmake-doc
$ sudo apt-get install devscripts build-essentials
$ sudo apt-get build-dep debmake-doc
$ cd debmake-doc*
$ make

You get the simulated log of building all the details of the packaging examples in this way.

With upcoming uploads, I should fix some CSS for the better cell phone compatibility but this is in good shape from the PC browsers.

Oh, not last ... NOW debian-handbook is on our www.debian.org server and its mirrors.

I think this is the most comprehensive work on Debian system and deserves to be presented in our web pages prominently. Kudos for people who wrote this fine manual.

https://www.debian.org/doc/manuals/debian-handbook/

(I think outdated reference to old TLDP pages should be mostly dropped in our https://www.debian.org/doc/ page.)

Please note web pages on www.debian.org has been modified to make the language based content negotiation. So you should see the page based on your browser settings.

Osamu

14 July 2015

Matthew Palmer: Why DANE isn't going to win

In a comment to my previous post, Daniele asked the entirely reasonable question,
Would you like to comment on why you think that DNSSEC+DANE are not a possible and much better alternative?
Where DANE fails to be a feasible alternative to the current system is that it is not widely acknowledged to be superior in every possible way . A weak demonstration of this is that no browser has implemented DANE support, and very few other TLS-using applications have, either. The only thing I use which has DANE support that I m aware of is Postfix and SMTP is an application in which the limitations of DANE have far less impact. My understanding of the limitations of DANE, for large-scale deployment, are enumerated below.

DNS Is Awful Quoting Google security engineer Adam Langley:
But many (~4% in past tests) of users can t resolve a TXT record when they can resolve an A record for the same name. In practice, consumer DNS is hijacked by many devices that do a poor job of implementing DNS.
Consider that TXT records are far, far older than TLSA records. It seems likely that TLSA records would fail to be retrieved greater than 4% of the time. Extrapolate to the likely failure rate for lookup of TLSA records would be, and imagine what that would do to the reliability of DANE verification. It would either be completely unworkable, or else would cause a whole new round of just click through the security error training. Ugh. This also impacts DNSSEC itself. Lots of recursive resolvers don t validate DNSSEC, and some providers mangle DNS responses in some way, which breaks DNSSEC. Since OSes don t support DNSSEC validation by default (for example, by having the name resolution APIs indicate DNSSEC validation status), browsers would essentially have to ship their own validating resolver code. Some people have concerns around the single point of control for DNS records, too. While the weakest link nature of the CA model is terribad, there is a significant body of opinion that replacing it with a single, minimally-accountable organisation like ICANN isn t a great trade. Finally, performance is also a concern. Having to go out-of-band to retrieve TLSA records delays page generation, and nobody likes slow page loads.

DNSSEC Is Awful Lots of people don t like DNSSEC, for all sorts of reasons. While I don t think it is quite as bad as people make out (I ve deployed it for most zones I manage, there are some legitimate issues that mean browser vendors aren t willing to rely on DNSSEC. 1024 bit RSA keys are quite common throughout the DNSSEC system. Getting rid of 1024 bit keys in the PKI has been a long-running effort; doing the same for DNSSEC is likely to take quite a while. Yes, rapid rotation is possible, by splitting key-signing and zone-signing (a good design choice), but since it can t be enforced, it s entirely likely that long-lived 1024 bit keys for signing DNSSEC zones is the rule, rather than exception.

DNS Providers are Awful While we all poke fun at CAs who get compromised, consider how often someone s DNS control panel gets compromised. Now ponder the fact that, if DANE is supported, TLSA records can be manipulated in that DNS control panel. Those records would then automatically be DNSSEC signed by the DNS provider and served up to anyone who comes along. Ouch. In theory, of course, you should choose a suitably secure DNS provider, to prevent this problem. Given that there are regular hijackings of high-profile domains (which, presumably, the owners of those domains would also want to prevent), there is something in the DNS service provider market which prevents optimal consumer behaviour. Market for lemons, perchance?

Conclusion None of these problems are unsolvable, although none are trivial. I like DANE as a concept, and I d really, really like to see it succeed. However, the problems I ve listed above are all reasonable objections, made by people who have their hands in browser codebases, and so unless they re fixed, I don t see that anyone s going to be able to rely on DANE on the Internet for a long, long time to come.

13 July 2015

Dirk Eddelbuettel: RcppGSL 0.2.5

A new version of RcppGSL arrived on CRAN a couple of days ago. This package provides an interface from R to the GNU GSL using our Rcpp package. In the course of preparation for the higher-performance R via C++ course I gave in Zuerich last month, I overhauled this package, its embedded package (!!) showing how to build a package which uses R, C++ and the GSL (and which can serve as a fine example of how to build and R and C++ package using an external library), and also overhauled the vignette which discusses all these aspect. All examples now consistently use Rcpp Attributes. The NEWS file entries follows below:
Changes in version 0.2.5 (2015-07-05)
  • The colnorm function in the included example package was rewritten to use Rcpp Attributes, the example package was updated and its version number increased to 0.0.3.
  • The unit tests also use the updated version of the example package.
  • The package, and the included example package, were updated throughout to conform to the current R CMD check standards.
  • The RcppGSL-intro vignette was updated throughout.
  • The Travis CI integration now uses r-cran-* packages which leads to faster tests.
Courtesy of CRANberries, a summary of changes to the most recent release is available. More information is on the RcppGSL page. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

12 July 2015

Lunar: Reproducible builds: week 11 in Stretch cycle

Debian is undertaking a huge effort to develop a reproducible builds system. I'd like to thank you for that. This could be Debian's most important project, with how badly computer security has been going.

PerniciousPunk in Reddit's Ask me anything! to Neil McGovern, DPL. What happened in the reproducible builds effort this week: Toolchain fixes More tools are getting patched to use the value of the SOURCE_DATE_EPOCH environment variable as the current time:

In the reproducible experimental toolchain which have been uploaded: Johannes Schauer followed up on making sbuild build path deterministic with several ideas. Packages fixed The following 311 packages became reproducible due to changes in their build dependencies : 4ti2, alot, angband, appstream-glib, argvalidate, armada-backlight, ascii, ask, astroquery, atheist, aubio, autorevision, awesome-extra, bibtool, boot-info-script, bpython, brian, btrfs-tools, bugs-everywhere, capnproto, cbm, ccfits, cddlib, cflow, cfourcc, cgit, chaussette, checkbox-ng, cinnamon-settings-daemon, clfswm, clipper, compton, cppcheck, crmsh, cupt, cutechess, d-itg, dahdi-tools, dapl, darnwdl, dbusada, debian-security-support, debomatic, dime, dipy, dnsruby, doctrine, drmips, dsc-statistics, dune-common, dune-istl, dune-localfunctions, easytag, ent, epr-api, esajpip, eyed3, fastjet, fatresize, fflas-ffpack, flann, flex, flint, fltk1.3, fonts-dustin, fonts-play, fonts-uralic, freecontact, freedoom, gap-guava, gap-scscp, genometools, geogebra, git-reintegrate, git-remote-bzr, git-remote-hg, gitmagic, givaro, gnash, gocr, gorm.app, gprbuild, grapefruit, greed, gtkspellmm, gummiboot, gyp, heat-cfntools, herold, htp, httpfs2, i3status, imagetooth, imapcopy, imaprowl, irker, jansson, jmapviewer, jsdoc-toolkit, jwm, katarakt, khronos-opencl-man, khronos-opengl-man4, lastpass-cli, lava-coordinator, lava-tool, lavapdu, letterize, lhapdf, libam7xxx, libburn, libccrtp, libclaw, libcommoncpp2, libdaemon, libdbusmenu-qt, libdc0, libevhtp, libexosip2, libfreenect, libgwenhywfar, libhmsbeagle, libitpp, libldm, libmodbus, libmtp, libmwaw, libnfo, libpam-abl, libphysfs, libplayer, libqb, libsecret, libserial, libsidplayfp, libtime-y2038-perl, libxr, lift, linbox, linthesia, livestreamer, lizardfs, lmdb, log4c, logbook, lrslib, lvtk, m-tx, mailman-api, matroxset, miniupnpd, mknbi, monkeysign, mpi4py, mpmath, mpqc, mpris-remote, musicbrainzngs, network-manager, nifticlib, obfsproxy, ogre-1.9, opal, openchange, opensc, packaging-tutorial, padevchooser, pajeng, paprefs, pavumeter, pcl, pdmenu, pepper, perroquet, pgrouting, pixz, pngcheck, po4a, powerline, probabel, profitbricks-client, prosody, pstreams, pyacidobasic, pyepr, pymilter, pytest, python-amqp, python-apt, python-carrot, python-django, python-ethtool, python-mock, python-odf, python-pathtools, python-pskc, python-psutil, python-pypump, python-repoze.tm2, python-repoze.what, qdjango, qpid-proton, qsapecng, radare2, reclass, repsnapper, resource-agents, rgain, rttool, ruby-aggregate, ruby-albino, ruby-archive-tar-minitar, ruby-bcat, ruby-blankslate, ruby-coffee-script, ruby-colored, ruby-dbd-mysql, ruby-dbd-odbc, ruby-dbd-pg, ruby-dbd-sqlite3, ruby-dbi, ruby-dirty-memoize, ruby-encryptor, ruby-erubis, ruby-fast-xs, ruby-fusefs, ruby-gd, ruby-git, ruby-globalhotkeys, ruby-god, ruby-hike, ruby-hmac, ruby-integration, ruby-jnunemaker-matchy, ruby-memoize, ruby-merb-core, ruby-merb-haml, ruby-merb-helpers, ruby-metaid, ruby-mina, ruby-net-irc, ruby-net-netrc, ruby-odbc, ruby-ole, ruby-packet, ruby-parseconfig, ruby-platform, ruby-plist, ruby-popen4, ruby-rchardet, ruby-romkan, ruby-ronn, ruby-rubyforge, ruby-rubytorrent, ruby-samuel, ruby-shoulda-matchers, ruby-sourcify, ruby-test-spec, ruby-validatable, ruby-wirble, ruby-xml-simple, ruby-zoom, rumor, rurple-ng, ryu, sam2p, scikit-learn, serd, shellex, shorewall-doc, shunit2, simbody, simplejson, smcroute, soqt, sord, spacezero, spamassassin-heatu, spamprobe, sphinxcontrib-youtube, splitpatch, sratom, stompserver, syncevolution, tgt, ticgit, tinyproxy, tor, tox, transmissionrpc, tweeper, udpcast, units-filter, viennacl, visp, vite, vmfs-tools, waffle, waitress, wavtool-pl, webkit2pdf, wfmath, wit, wreport, x11proto-input, xbae, xdg-utils, xdotool, xsystem35, yapsy, yaz. Please note that some packages in the above list are falsely reproducible. In the experimental toolchain, debhelper exported TZ=UTC and this made packages capturing the current date (without the time) reproducible in the current test environment. The following packages became reproducible after getting fixed: Ben Hutchings upstreamed several patches to fix Linux reproducibility issues which were quickly merged. Some uploads fixed some reproducibility issues but not all of them: Uploads that should fix packages not in main: Patches submitted which have not made their way to the archive yet: reproducible.debian.net A new package set has been added for lua maintainers. (h01ger) tracker.debian.org now only shows reproducibility issues for unstable. Holger and Mattia worked on several bugfixes and enhancements: finished initial test setup for NetBSD, rewriting more shell scripts in Python, saving UDD requests, and more debbindiff development Reiner Herrmann fixed text comparison of files with different encoding. Documentation update Juan Picca added to the commands needed for a local test chroot installation of the locales-all package. Package reviews 286 obsolete reviews have been removed, 278 added and 243 updated this week. 43 new bugs for packages failing to build from sources have been filled by Chris West (Faux), Mattia Rizzolo, and h01ger. The following new issues have been added: timestamps_in_manpages_generated_by_ronn, timestamps_in_documentation_generated_by_org_mode, and timestamps_in_pdf_generated_by_matplotlib. Misc. Reiner Herrmann has submitted patches for OpenWrt. Chris Lamb cleaned up some code and removed cruft in the misc.git repository. Mattia Rizzolo updated the prebuilder script to match what is currently done on reproducible.debian.net.

Lars Wirzenius: /u/liw no more

I've used Reddit for many years. I used it for many years without an account, but eventually I made one. The site has always had its share of unpleasantness, people who're more interested in tearing down than in building. In recent years, it's gotten worse, and getting out of hand. During the fairly short reign of Ellen Pao as CEO, I found things to be getting better. The site was starting to make it clear that harrassment, for example, was unacceptable. Unsurprisingly, this made some of the nastier people quite upset. Pao has now resigned, and a new CEO has started. He had an "Ask Me Anything" session yesterday, and made it clear that he's changing things. From my point of view, it's changing to the worse. He made it clear that as long as Reddit itself does not get into legal trouble, and harrassment isn't too overt or particularly public, it's OK now. I've closed my Reddit account.

7 July 2015

Lunar: Reproducible builds: week 10 in Stretch cycle

What happened about the reproducible builds effort this week: Media coverage Daniel Stender published an English translation of the article which originally appeared in Linux Magazin in Admin Magazine. Toolchain fixes Fixes landed in the Debian archive: Lunar submitted to Debian the patch already sent upstream adding a --clamp-mtime option to tar. Patches have been submitted to add support for SOURCE_DATE_EPOCH to txt2man (Reiner Herrmann), epydoc (Reiner Herrmann), GCC (Dhole), and Doxygen (akira). Dhole uploaded a new experimental debhelper to the reproducible repository which exports SOURCE_DATE_EPOCH. As part of the experiment, the patch also sets TZ to UTC which should help with most timezone issues. It might still be problematic for some packages which would change their settings based on this. Mattia Rizzolo sent upstream a patch originally written by Lunar to make the generate-id() function be deterministic in libxslt. While that patch was quickly rejected by upstream, Andrew Ayer came up with a much better one which sadly could have some performance impact. Daniel Veillard replied with another patch that should be deterministic in most cases without needing extra data structures. It's impact is currently being investigated by retesting packages on reproducible.debian.net. akira added a new option to sbuild for configuring the path in which packages are built. This will be needed for the srebuild script. Niko Tyni asked Perl upstream about it using the __DATE__ and __TIME__ C processor macros. Packages fixed The following 143 packages became reproducible due to changes in their build dependencies: alot, argvalidate, astroquery, blender, bpython, brian, calibre, cfourcc, chaussette, checkbox-ng, cloc, configshell, daisy-player, dipy, dnsruby, dput-ng, dsc-statistics, eliom, emacspeak, freeipmi, geant321, gpick, grapefruit, heat-cfntools, imagetooth, jansson, jmapviewer, lava-tool, libhtml-lint-perl, libtime-y2038-perl, lift, lua-ldoc, luarocks, mailman-api, matroxset, maven-hpi-plugin, mknbi, mpi4py, mpmath, msnlib, munkres, musicbrainzngs, nova, pecomato, pgrouting, pngcheck, powerline, profitbricks-client, pyepr, pylibssh2, pylogsparser, pystemmer, pytest, python-amqp, python-apt, python-carrot, python-crypto, python-darts.lib.utils.lru, python-demgengeo, python-graph, python-mock, python-musicbrainz2, python-pathtools, python-pskc, python-psutil, python-pypump, python-repoze.sphinx.autointerface, python-repoze.tm2, python-repoze.what-plugins, python-repoze.what, python-repoze.who-plugins, python-xstatic-term.js, reclass, resource-agents, rgain, rttool, ruby-aggregate, ruby-archive-tar-minitar, ruby-bcat, ruby-blankslate, ruby-coffee-script, ruby-colored, ruby-dbd-mysql, ruby-dbd-odbc, ruby-dbd-pg, ruby-dbd-sqlite3, ruby-dbi, ruby-dirty-memoize, ruby-encryptor, ruby-erubis, ruby-fast-xs, ruby-fusefs, ruby-gd, ruby-git, ruby-globalhotkeys, ruby-god, ruby-hike, ruby-hmac, ruby-integration, ruby-ipaddress, ruby-jnunemaker-matchy, ruby-memoize, ruby-merb-core, ruby-merb-haml, ruby-merb-helpers, ruby-metaid, ruby-mina, ruby-net-irc, ruby-net-netrc, ruby-odbc, ruby-packet, ruby-parseconfig, ruby-platform, ruby-plist, ruby-popen4, ruby-rchardet, ruby-romkan, ruby-rubyforge, ruby-rubytorrent, ruby-samuel, ruby-shoulda-matchers, ruby-sourcify, ruby-test-spec, ruby-validatable, ruby-wirble, ruby-xml-simple, ruby-zoom, ryu, simplejson, spamassassin-heatu, speaklater, stompserver, syncevolution, syncmaildir, thin, ticgit, tox, transmissionrpc, vdr-plugin-xine, waitress, whereami, xlsx2csv, zathura. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Patches submitted which have not made their way to the archive yet: reproducible.debian.net A new package set for the X Strike Force has been added. (h01ger) Bugs tagged with locale are now visible in the statistics. (h01ger) Some work has been done add tests for NetBSD. (h01ger) Many changes by Mattia Rizzolo have been merged on the whole infrastructure: debbindiff development Version 26 has been released on June 28th fixing the comparison of files of unknown format. (Lunar) A missing dependency identified in python-rpm affecting debbindiff installation without recommended packages was promptly fixed by Michal iha . Lunar also started a massive code rearchitecture to enhance code reuse and enable new features. Nothing visible yet, though. Documentation update josch and Mattia Rizzolo documented how to reschedule packages from Alioth. Package reviews 142 obsolete reviews have been removed, 344 added and 107 updated this week. Chris West (Faux) filled 13 new bugs for packages failing to build from sources. The following new issues have been added: snapshot_placeholder_replaced_with_timestamp_in_pom_properties, different_encoding, timestamps_in_documentation_generated_by_org_mode and timestamps_in_pdf_generated_by_matplotlib.

Petter Reinholdtsen: MPEG LA on "Internet Broadcast AVC Video" licensing and non-private use

After asking the Norwegian Broadcasting Company (NRK) why they can broadcast and stream H.264 video without an agreement with the MPEG LA, I was wiser, but still confused. So I asked MPEG LA if their understanding matched that of NRK. As far as I can tell, it does not. I started by asking for more information about the various licensing classes and what exactly is covered by the "Internet Broadcast AVC Video" class that NRK pointed me at to explain why NRK did not need a license for streaming H.264 video:
According to a MPEG LA press release dated 2010-02-02, there is no charge when using MPEG AVC/H.264 according to the terms of "Internet Broadcast AVC Video". I am trying to understand exactly what the terms of "Internet Broadcast AVC Video" is, and wondered if you could help me. What exactly is covered by these terms, and what is not? The only source of more information I have been able to find is a PDF named AVC Patent Portfolio License Briefing, which states this about the fees:
  • Where End User pays for AVC Video
    • Subscription (not limited by title) 100,000 or fewer subscribers/yr = no royalty; > 100,000 to 250,000 subscribers/yr = $25,000; >250,000 to 500,000 subscribers/yr = $50,000; >500,000 to 1M subscribers/yr = $75,000; >1M subscribers/yr = $100,000
    • Title-by-Title - 12 minutes or less = no royalty; >12 minutes in length = lower of (a) 2% or (b) $0.02 per title
  • Where remuneration is from other sources
    • Free Television - (a) one-time $2,500 per transmission encoder or (b) annual fee starting at $2,500 for > 100,000 HH rising to maximum $10,000 for >1,000,000 HH
    • Internet Broadcast AVC Video (not title-by-title, not subscription) no royalty for life of the AVC Patent Portfolio License
Am I correct in assuming that the four categories listed is the categories used when selecting licensing terms, and that "Internet Broadcast AVC Video" is the category for things that do not fall into one of the other three categories? Can you point me to a good source explaining what is ment by "title-by-title" and "Free Television" in the license terms for AVC/H.264? Will a web service providing H.264 encoded video content in a "video on demand" fashing similar to Youtube and Vimeo, where no subscription is required and no payment is required from end users to get access to the videos, fall under the terms of the "Internet Broadcast AVC Video", ie no royalty for life of the AVC Patent Portfolio license? Does it matter if some users are subscribed to get access to personalized services? Note, this request and all answers will be published on the Internet.
The answer came quickly from Benjamin J. Myers, Licensing Associate with the MPEG LA:
Thank you for your message and for your interest in MPEG LA. We appreciate hearing from you and I will be happy to assist you. As you are aware, MPEG LA offers our AVC Patent Portfolio License which provides coverage under patents that are essential for use of the AVC/H.264 Standard (MPEG-4 Part 10). Specifically, coverage is provided for end products and video content that make use of AVC/H.264 technology. Accordingly, the party offering such end products and video to End Users concludes the AVC License and is responsible for paying the applicable royalties. Regarding Internet Broadcast AVC Video, the AVC License generally defines such content to be video that is distributed to End Users over the Internet free-of-charge. Therefore, if a party offers a service which allows users to upload AVC/H.264 video to its website, and such AVC Video is delivered to End Users for free, then such video would receive coverage under the sublicense for Internet Broadcast AVC Video, which is not subject to any royalties for the life of the AVC License. This would also apply in the scenario where a user creates a free online account in order to receive a customized offering of free AVC Video content. In other words, as long as the End User is given access to or views AVC Video content at no cost to the End User, then no royalties would be payable under our AVC License. On the other hand, if End Users pay for access to AVC Video for a specific period of time (e.g., one month, one year, etc.), then such video would constitute Subscription AVC Video. In cases where AVC Video is delivered to End Users on a pay-per-view basis, then such content would constitute Title-by-Title AVC Video. If a party offers Subscription or Title-by-Title AVC Video to End Users, then they would be responsible for paying the applicable royalties you noted below. Finally, in the case where AVC Video is distributed for free through an "over-the-air, satellite and/or cable transmission", then such content would constitute Free Television AVC Video and would be subject to the applicable royalties. For your reference, I have attached a .pdf copy of the AVC License. You will find the relevant sublicense information regarding AVC Video in Sections 2.2 through 2.5, and the corresponding royalties in Section 3.1.2 through 3.1.4. You will also find the definitions of Title-by-Title AVC Video, Subscription AVC Video, Free Television AVC Video, and Internet Broadcast AVC Video in Section 1 of the License. Please note that the electronic copy is provided for informational purposes only and cannot be used for execution. I hope the above information is helpful. If you have additional questions or need further assistance with the AVC License, please feel free to contact me directly.
Having a fresh copy of the license text was useful, and knowing that the definition of Title-by-Title required payment per title made me aware that my earlier understanding of that phrase had been wrong. But I still had a few questions:
I have a small followup question. Would it be possible for me to get a license with MPEG LA even if there are no royalties to be paid? The reason I ask, is that some video related products have a copyright clause limiting their use without a license with MPEG LA. The clauses typically look similar to this:
This product is licensed under the AVC patent portfolio license for the personal and non-commercial use of a consumer to (a) encode video in compliance with the AVC standard ("AVC video") and/or (b) decode AVC video that was encoded by a consumer engaged in a personal and non-commercial activity and/or AVC video that was obtained from a video provider licensed to provide AVC video. No license is granted or shall be implied for any other use. additional information may be obtained from MPEG LA L.L.C.
It is unclear to me if this clause mean that I need to enter into an agreement with MPEG LA to use the product in question, even if there are no royalties to be paid to MPEG LA. I suspect it will differ depending on the jurisdiction, and mine is Norway. What is MPEG LAs view on this?
According to the answer, MPEG LA believe those using such tools for non-personal or commercial use need a license with them:
With regard to the Notice to Customers, I would like to begin by clarifying that the Notice from Section 7.1 of the AVC License reads: THIS PRODUCT IS LICENSED UNDER THE AVC PATENT PORTFOLIO LICENSE FOR THE PERSONAL USE OF A CONSUMER OR OTHER USES IN WHICH IT DOES NOT RECEIVE REMUNERATION TO (i) ENCODE VIDEO IN COMPLIANCE WITH THE AVC STANDARD ("AVC VIDEO") AND/OR (ii) DECODE AVC VIDEO THAT WAS ENCODED BY A CONSUMER ENGAGED IN A PERSONAL ACTIVITY AND/OR WAS OBTAINED FROM A VIDEO PROVIDER LICENSED TO PROVIDE AVC VIDEO. NO LICENSE IS GRANTED OR SHALL BE IMPLIED FOR ANY OTHER USE. ADDITIONAL INFORMATION MAY BE OBTAINED FROM MPEG LA, L.L.C. SEE HTTP://WWW.MPEGLA.COM The Notice to Customers is intended to inform End Users of the personal usage rights (for example, to watch video content) included with the product they purchased, and to encourage any party using the product for commercial purposes to contact MPEG LA in order to become licensed for such use (for example, when they use an AVC Product to deliver Title-by-Title, Subscription, Free Television or Internet Broadcast AVC Video to End Users, or to re-Sell a third party's AVC Product as their own branded AVC Product). Therefore, if a party is to be licensed for its use of an AVC Product to Sell AVC Video on a Title-by-Title, Subscription, Free Television or Internet Broadcast basis, that party would need to conclude the AVC License, even in the case where no royalties were payable under the License. On the other hand, if that party (either a Consumer or business customer) simply uses an AVC Product for their own internal purposes and not for the commercial purposes referenced above, then such use would be included in the royalty paid for the AVC Products by the licensed supplier. Finally, I note that our AVC License provides worldwide coverage in countries that have AVC Patent Portfolio Patents, including Norway. I hope this clarification is helpful. If I may be of any further assistance, just let me know.
The mentioning of Norwegian patents made me a bit confused, so I asked for more information:
But one minor question at the end. If I understand you correctly, you state in the quote above that there are patents in the AVC Patent Portfolio that are valid in Norway. This make me believe I read the list available from <URL: http://www.mpegla.com/main/programs/AVC/Pages/PatentList.aspx > incorrectly, as I believed the "NO" prefix in front of patents were Norwegian patents, and the only one I could find under Mitsubishi Electric Corporation expired in 2012. Which patents are you referring to that are relevant for Norway?
Again, the quick answer explained how to read the list of patents in that list:
Your understanding is correct that the last AVC Patent Portfolio Patent in Norway expired on 21 October 2012. Therefore, where AVC Video is both made and Sold in Norway after that date, then no royalties would be payable for such AVC Video under the AVC License. With that said, our AVC License provides historic coverage for AVC Products and AVC Video that may have been manufactured or Sold before the last Norwegian AVC patent expired. I would also like to clarify that coverage is provided for the country of manufacture and the country of Sale that has active AVC Patent Portfolio Patents. Therefore, if a party offers AVC Products or AVC Video for Sale in a country with active AVC Patent Portfolio Patents (for example, Sweden, Denmark, Finland, etc.), then that party would still need coverage under the AVC License even if such products or video are initially made in a country without active AVC Patent Portfolio Patents (for example, Norway). Similarly, a party would need to conclude the AVC License if they make AVC Products or AVC Video in a country with active AVC Patent Portfolio Patents, but eventually Sell such AVC Products or AVC Video in a country without active AVC Patent Portfolio Patents.
As far as I understand it, MPEG LA believe anyone using Adobe Premiere and other video related software with a H.264 distribution license need a license agreement with MPEG LA to use such tools for anything non-private or commercial, while it is OK to set up a Youtube-like service as long as no-one pays to get access to the content. I still have no clear idea how this applies to Norway, where none of the patents MPEG LA is licensing are valid. Will the copyright terms take precedence or can those terms be ignored because the patents are not valid in Norway?

6 July 2015

Ritesh Raj Sarraf: Apport Integration with Debian - GSoC Update

For this year's Google Summer of Code, I have been mentoring Yuru Roy Shao, on Integrating Apport with Debian. Yuru is a CS student studying at University of Michigan, USA completing his PhD. For around 2+ years, Apport was packaged for Debian, but remained in Experimental. While we did have a separate (Debian BTS aware) crashdb, the general concerns (bug spam, too many duplicates etc) were the reason we held its inclusion. With this GSoC, Yuru has been bringing some of the missing integration pieces of Debian. For example, we are now using debianbts to to talk to the Debian BTS Server, and fetch bug reports for the user. While apport's Bug Report data collection itself is very comprehensive, still for Debian, it will have the option to use native as well as reportbug. This will allow us to use the many hooks people have integrated so far with reportbug. Both Bug Report data collectors will be available. Yuru has blogged about his GSoC progress so far, here. Please do have a read, and let us know your views. If the travel formalities work out well, I intend to attend Debconf this year, and can talk in more detail.

Categories:

Keywords:

Like:

5 July 2015

Robert Edmonds: Git packaging workflow for py-lmdb

Recently, I packaged the py-lmdb Python binding for the LMDB database library. This package is going to be team maintained by the pkg-db group, which is responsible for maintaining BerkeleyDB and LMDB packages. Below are my notes on (re-)Debianizing this package and how the Git repository for the source package is laid out. The upstream py-lmdb developer has a Git-centric workflow. Development is done on the master branch, with regular releases done as fast-forward merges to the release branch. Release tags of the form py-lmdb_X.YZ are provided. The only tarballs provided are the ones that GitHub automatically generates from tags. Since these tarballs are synthetic and the content of these tarballs matches the content on the corresponding tag, we will ignore them in favor of using the release tags directly. (The --git-pristine-tar-commit option to gbp-buildpackage will be used so that .orig.tar.gz files can be replicated so that the Debian archive will accept subsequent uploads, but tarballs are otherwise irrelevant to our workflow.) To make it clear that the release tags come from upstream's repository, they should be prefixed with upstream/, which would preferably result in a DEP-14 compliant scheme. (Unfortunately, since upstream's release tags begin with py-lmdb_, this doesn't quite match the pattern that DEP-14 recommends.) Here is how the local packaging repository is initialized. Note that git clone isn't used, so that we can customize how the tags are fetched. Instead, we create an empty Git repository and add the upstream repository as the upstream remote. The --no-tags option is used, so that git fetch does not import the remote's tags. However, we also add a custom fetch refspec refs/tags/*:refs/tags/upstream/* so that the remote's tags are explicitly fetched, but with the upstream/ prefix. 
$ mkdir py-lmdb
$ cd py-lmdb
$ git init
Initialized empty Git repository in /home/edmonds/debian/py-lmdb/.git/
$ git remote add --no-tags upstream https://github.com/dw/py-lmdb
$ git config --add remote.upstream.fetch 'refs/tags/*:refs/tags/upstream/*'
$ git fetch upstream
remote: Counting objects: 3336, done.
remote: Total 3336 (delta 0), reused 0 (delta 0), pack-reused 3336
Receiving objects: 100% (3336/3336), 2.15 MiB   0 bytes/s, done.
Resolving deltas: 100% (1958/1958), done.
From https://github.com/dw/py-lmdb
 * [new branch]      master     -> upstream/master
 * [new branch]      release    -> upstream/release
 * [new branch]      win32-sparse-patch -> upstream/win32-sparse-patch
 * [new tag]         last-cython-version -> upstream/last-cython-version
 * [new tag]         py-lmdb_0.1 -> upstream/py-lmdb_0.1
 * [new tag]         py-lmdb_0.2 -> upstream/py-lmdb_0.2
 * [new tag]         py-lmdb_0.3 -> upstream/py-lmdb_0.3
 * [new tag]         py-lmdb_0.4 -> upstream/py-lmdb_0.4
 * [new tag]         py-lmdb_0.5 -> upstream/py-lmdb_0.5
 * [new tag]         py-lmdb_0.51 -> upstream/py-lmdb_0.51
 * [new tag]         py-lmdb_0.52 -> upstream/py-lmdb_0.52
 * [new tag]         py-lmdb_0.53 -> upstream/py-lmdb_0.53
 * [new tag]         py-lmdb_0.54 -> upstream/py-lmdb_0.54
 * [new tag]         py-lmdb_0.56 -> upstream/py-lmdb_0.56
 * [new tag]         py-lmdb_0.57 -> upstream/py-lmdb_0.57
 * [new tag]         py-lmdb_0.58 -> upstream/py-lmdb_0.58
 * [new tag]         py-lmdb_0.59 -> upstream/py-lmdb_0.59
 * [new tag]         py-lmdb_0.60 -> upstream/py-lmdb_0.60
 * [new tag]         py-lmdb_0.61 -> upstream/py-lmdb_0.61
 * [new tag]         py-lmdb_0.62 -> upstream/py-lmdb_0.62
 * [new tag]         py-lmdb_0.63 -> upstream/py-lmdb_0.63
 * [new tag]         py-lmdb_0.64 -> upstream/py-lmdb_0.64
 * [new tag]         py-lmdb_0.65 -> upstream/py-lmdb_0.65
 * [new tag]         py-lmdb_0.66 -> upstream/py-lmdb_0.66
 * [new tag]         py-lmdb_0.67 -> upstream/py-lmdb_0.67
 * [new tag]         py-lmdb_0.68 -> upstream/py-lmdb_0.68
 * [new tag]         py-lmdb_0.69 -> upstream/py-lmdb_0.69
 * [new tag]         py-lmdb_0.70 -> upstream/py-lmdb_0.70
 * [new tag]         py-lmdb_0.71 -> upstream/py-lmdb_0.71
 * [new tag]         py-lmdb_0.72 -> upstream/py-lmdb_0.72
 * [new tag]         py-lmdb_0.73 -> upstream/py-lmdb_0.73
 * [new tag]         py-lmdb_0.74 -> upstream/py-lmdb_0.74
 * [new tag]         py-lmdb_0.75 -> upstream/py-lmdb_0.75
 * [new tag]         py-lmdb_0.76 -> upstream/py-lmdb_0.76
 * [new tag]         py-lmdb_0.77 -> upstream/py-lmdb_0.77
 * [new tag]         py-lmdb_0.78 -> upstream/py-lmdb_0.78
 * [new tag]         py-lmdb_0.79 -> upstream/py-lmdb_0.79
 * [new tag]         py-lmdb_0.80 -> upstream/py-lmdb_0.80
 * [new tag]         py-lmdb_0.81 -> upstream/py-lmdb_0.81
 * [new tag]         py-lmdb_0.82 -> upstream/py-lmdb_0.82
 * [new tag]         py-lmdb_0.83 -> upstream/py-lmdb_0.83
 * [new tag]         py-lmdb_0.84 -> upstream/py-lmdb_0.84
 * [new tag]         py-lmdb_0.85 -> upstream/py-lmdb_0.85
 * [new tag]         py-lmdb_0.86 -> upstream/py-lmdb_0.86
$
Note that at this point we have content from the upstream remote in our local repository, but we don't have any local branches: 
$ git status
On branch master
Initial commit
nothing to commit (create/copy files and use "git add" to track)
$ git branch -a
  remotes/upstream/master
  remotes/upstream/release
  remotes/upstream/win32-sparse-patch
$
We will use the DEP-14 naming scheme for the packaging branches, so the branch for packages targeted at unstable will be called debian/sid. Since I already made an initial 0.84-1 upload, we need to start the debian/sid branch from the upstream 0.84 tag and import the original packaging content from that upload. The --no-track flag is passed to git checkout initially so that Git doesn't consider the upstream release tag upstream/py-lmdb_0.84 to be the upstream branch for our packaging branch. 
$ git checkout --no-track -b debian/sid upstream/py-lmdb_0.84
Switched to a new branch 'debian/sid'
$
At this point I imported the original packaging content for 0.84-1 with git am. Then, I signed the debian/0.84-1 tag: 
$ git tag -s -m 'Debian release 0.84-1' debian/0.84-1
$ git verify-tag debian/0.84-1
gpg: Signature made Sat 04 Jul 2015 02:49:42 PM EDT using RSA key ID AAF6CDAE
gpg: Good signature from "Robert Edmonds <edmonds@mycre.ws>" [ultimate]
gpg:                 aka "Robert Edmonds <edmonds@fsi.io>" [ultimate]
gpg:                 aka "Robert Edmonds <edmonds@debian.org>" [ultimate]
$
New upstream releases are integrated by fetching new upstream tags and non-fast-forward merging into the packaging branch. The latest release is 0.86, so we merge from the upstream/py-lmdb_0.86 tag. 
$ git fetch upstream --dry-run
[...]
$ git fetch upstream
[...]
$ git checkout debian/sid
Already on 'debian/sid'
$ git merge --no-ff --no-edit upstream/py-lmdb_0.86
Merge made by the 'recursive' strategy.
 ChangeLog                        46 ++++++++++++++
 docs/index.rst                   46 +++++++++++++-
 docs/themes/acid/layout.html      4 +-
 examples/dirtybench-gdbm.py       6 ++
 examples/dirtybench.py           19 ++++++
 examples/nastybench.py           18 ++++--
 examples/parabench.py             6 ++
 lib/lmdb.h                       37 ++++++-----
 lib/mdb.c                       281 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------
 lib/midl.c                        2 +-
 lib/midl.h                        2 +-
 lib/py-lmdb/preload.h            48 ++++++++++++++
 lmdb/__init__.py                  2 +-
 lmdb/cffi.py                    120 ++++++++++++++++++++++++-----------
 lmdb/cpython.c                   86 +++++++++++++++++++------
 lmdb/tool.py                      5 +-
 misc/gdb.commands                21 ++++++
 misc/runtests-travisci.sh         3 +-
 misc/runtests-ubuntu-12-04.sh    28 ++++----
 setup.py                          2 +
 tests/crash_test.py              22 +++++++
 tests/cursor_test.py             37 +++++++++++
 tests/env_test.py                73 +++++++++++++++++++++
 tests/testlib.py                 14 +++-
 tests/txn_test.py                20 ++++++
 25 files changed, 773 insertions(+), 175 deletions(-)
 create mode 100644 lib/py-lmdb/preload.h
 create mode 100644 misc/gdb.commands
$
Here I did some additional development work like editing the debian/gbp.conf file and applying a fix for #790738 to make the package build reproducibly. The package is now ready for an 0.86-1 upload, so I ran the following gbp dch command: 
$ gbp dch --release --auto --new-version=0.86-1 --commit
gbp:info: Found tag for topmost changelog version '6bdbb56c04571fe2d5d22aa0287ab0dc83959de5'
gbp:info: Continuing from commit '6bdbb56c04571fe2d5d22aa0287ab0dc83959de5'
gbp:info: Changelog has been committed for version 0.86-1
$
This automatically generates a changelog entry for 0.86-1, but it includes commit summaries for all of the upstream commits since the last release, which I had to edit out. Then, I used gbp buildpackage with BUILDER=pbuilder to build the package in a clean, up-to-date sid chroot. After checking the result, I signed the debian/0.86-1 tag: 
$ git tag -s -m 'Debian release 0.86-1' debian/0.86-1
$
The package is now ready to be pushed to git.debian.org. First, a bare repository is initialized: 
$ ssh git.debian.org
edmonds@moszumanska:~$ cd /srv/git.debian.org/git/pkg-db/
edmonds@moszumanska:/srv/git.debian.org/git/pkg-db$ umask 002
edmonds@moszumanska:/srv/git.debian.org/git/pkg-db$ mkdir py-lmdb.git
edmonds@moszumanska:/srv/git.debian.org/git/pkg-db$ cd py-lmdb.git/
edmonds@moszumanska:/srv/git.debian.org/git/pkg-db/py-lmdb.git$ git --bare init --shared
Initialized empty shared Git repository in /srv/git.debian.org/git/pkg-db/py-lmdb.git/
edmonds@moszumanska:/srv/git.debian.org/git/pkg-db/py-lmdb.git$ echo 'py-lmdb Debian packaging' > description
edmonds@moszumanska:/srv/git.debian.org/git/pkg-db/py-lmdb.git$ mv hooks/post-update.sample hooks/post-update
edmonds@moszumanska:/srv/git.debian.org/git/pkg-db/py-lmdb.git$ chmod a+x hooks/post-update
edmonds@moszumanska:/srv/git.debian.org/git/pkg-db/py-lmdb.git$ logout
Shared connection to git.debian.org closed.
Then, we add a new debian remote to our local packaging repository. Per our repository conventions, we need to ensure that only branch names matching debian/* and pristine-tar and tag names matching debian/* and upstream/* are pushed to the debian remote when we run git push debian, so we add a a set of remote.debian.push refspecs that correspond to these conventions. We also add an explicit remote.debian.fetch refspec to fetch tags. 
$ git remote add debian ssh://git.debian.org/git/pkg-db/py-lmdb.git
$ git config --add remote.debian.push 'refs/tags/debian/*'
$ git config --add remote.debian.push 'refs/tags/upstream/*'
$ git config --add remote.debian.push 'refs/heads/debian/*'
$ git config --add remote.debian.push 'refs/heads/pristine-tar'
$ git config --add remote.debian.fetch 'refs/tags/*:refs/tags/*'
We now run the initial push to the remote Git repository. The --set-upstream option is used so that our local branches will be configured to track the corresponding remote branches. Also note that the debian/* and upstream/* tags are pushed as well. 
$ git push debian --set-upstream
Counting objects: 3333, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (1083/1083), done.
Writing objects: 100% (3333/3333), 1.37 MiB   0 bytes/s, done.
Total 3333 (delta 2231), reused 3314 (delta 2218)
To ssh://git.debian.org/git/pkg-db/py-lmdb.git
 * [new branch]      pristine-tar -> pristine-tar
 * [new branch]      debian/sid -> debian/sid
 * [new tag]         debian/0.84-1 -> debian/0.84-1
 * [new tag]         debian/0.86-1 -> debian/0.86-1
 * [new tag]         upstream/last-cython-version -> upstream/last-cython-version
 * [new tag]         upstream/py-lmdb_0.1 -> upstream/py-lmdb_0.1
 * [new tag]         upstream/py-lmdb_0.2 -> upstream/py-lmdb_0.2
 * [new tag]         upstream/py-lmdb_0.3 -> upstream/py-lmdb_0.3
 * [new tag]         upstream/py-lmdb_0.4 -> upstream/py-lmdb_0.4
 * [new tag]         upstream/py-lmdb_0.5 -> upstream/py-lmdb_0.5
 * [new tag]         upstream/py-lmdb_0.51 -> upstream/py-lmdb_0.51
 * [new tag]         upstream/py-lmdb_0.52 -> upstream/py-lmdb_0.52
 * [new tag]         upstream/py-lmdb_0.53 -> upstream/py-lmdb_0.53
 * [new tag]         upstream/py-lmdb_0.54 -> upstream/py-lmdb_0.54
 * [new tag]         upstream/py-lmdb_0.56 -> upstream/py-lmdb_0.56
 * [new tag]         upstream/py-lmdb_0.57 -> upstream/py-lmdb_0.57
 * [new tag]         upstream/py-lmdb_0.58 -> upstream/py-lmdb_0.58
 * [new tag]         upstream/py-lmdb_0.59 -> upstream/py-lmdb_0.59
 * [new tag]         upstream/py-lmdb_0.60 -> upstream/py-lmdb_0.60
 * [new tag]         upstream/py-lmdb_0.61 -> upstream/py-lmdb_0.61
 * [new tag]         upstream/py-lmdb_0.62 -> upstream/py-lmdb_0.62
 * [new tag]         upstream/py-lmdb_0.63 -> upstream/py-lmdb_0.63
 * [new tag]         upstream/py-lmdb_0.64 -> upstream/py-lmdb_0.64
 * [new tag]         upstream/py-lmdb_0.65 -> upstream/py-lmdb_0.65
 * [new tag]         upstream/py-lmdb_0.66 -> upstream/py-lmdb_0.66
 * [new tag]         upstream/py-lmdb_0.67 -> upstream/py-lmdb_0.67
 * [new tag]         upstream/py-lmdb_0.68 -> upstream/py-lmdb_0.68
 * [new tag]         upstream/py-lmdb_0.69 -> upstream/py-lmdb_0.69
 * [new tag]         upstream/py-lmdb_0.70 -> upstream/py-lmdb_0.70
 * [new tag]         upstream/py-lmdb_0.71 -> upstream/py-lmdb_0.71
 * [new tag]         upstream/py-lmdb_0.72 -> upstream/py-lmdb_0.72
 * [new tag]         upstream/py-lmdb_0.73 -> upstream/py-lmdb_0.73
 * [new tag]         upstream/py-lmdb_0.74 -> upstream/py-lmdb_0.74
 * [new tag]         upstream/py-lmdb_0.75 -> upstream/py-lmdb_0.75
 * [new tag]         upstream/py-lmdb_0.76 -> upstream/py-lmdb_0.76
 * [new tag]         upstream/py-lmdb_0.77 -> upstream/py-lmdb_0.77
 * [new tag]         upstream/py-lmdb_0.78 -> upstream/py-lmdb_0.78
 * [new tag]         upstream/py-lmdb_0.79 -> upstream/py-lmdb_0.79
 * [new tag]         upstream/py-lmdb_0.80 -> upstream/py-lmdb_0.80
 * [new tag]         upstream/py-lmdb_0.81 -> upstream/py-lmdb_0.81
 * [new tag]         upstream/py-lmdb_0.82 -> upstream/py-lmdb_0.82
 * [new tag]         upstream/py-lmdb_0.83 -> upstream/py-lmdb_0.83
 * [new tag]         upstream/py-lmdb_0.84 -> upstream/py-lmdb_0.84
 * [new tag]         upstream/py-lmdb_0.85 -> upstream/py-lmdb_0.85
 * [new tag]         upstream/py-lmdb_0.86 -> upstream/py-lmdb_0.86
Branch pristine-tar set up to track remote branch pristine-tar from debian.
Branch debian/sid set up to track remote branch debian/sid from debian.
$
After the initial push, we need to configure the remote repository so that clones will checkout the debian/sid branch by default: 
$ ssh git.debian.org
edmonds@moszumanska:~$ cd /srv/git.debian.org/git/pkg-db/py-lmdb.git/
edmonds@moszumanska:/srv/git.debian.org/git/pkg-db/py-lmdb.git$ git symbolic-ref HEAD refs/heads/debian/sid
edmonds@moszumanska:/srv/git.debian.org/git/pkg-db/py-lmdb.git$ logout
Shared connection to git.debian.org closed.
We can check if there are any updates in upstream's Git repository with the following command: 
$ git fetch upstream --dry-run -v
From https://github.com/dw/py-lmdb
 = [up to date]      master     -> upstream/master
 = [up to date]      release    -> upstream/release
 = [up to date]      win32-sparse-patch -> upstream/win32-sparse-patch
 = [up to date]      last-cython-version -> upstream/last-cython-version
 = [up to date]      py-lmdb_0.1 -> upstream/py-lmdb_0.1
 = [up to date]      py-lmdb_0.2 -> upstream/py-lmdb_0.2
 = [up to date]      py-lmdb_0.3 -> upstream/py-lmdb_0.3
 = [up to date]      py-lmdb_0.4 -> upstream/py-lmdb_0.4
 = [up to date]      py-lmdb_0.5 -> upstream/py-lmdb_0.5
 = [up to date]      py-lmdb_0.51 -> upstream/py-lmdb_0.51
 = [up to date]      py-lmdb_0.52 -> upstream/py-lmdb_0.52
 = [up to date]      py-lmdb_0.53 -> upstream/py-lmdb_0.53
 = [up to date]      py-lmdb_0.54 -> upstream/py-lmdb_0.54
 = [up to date]      py-lmdb_0.56 -> upstream/py-lmdb_0.56
 = [up to date]      py-lmdb_0.57 -> upstream/py-lmdb_0.57
 = [up to date]      py-lmdb_0.58 -> upstream/py-lmdb_0.58
 = [up to date]      py-lmdb_0.59 -> upstream/py-lmdb_0.59
 = [up to date]      py-lmdb_0.60 -> upstream/py-lmdb_0.60
 = [up to date]      py-lmdb_0.61 -> upstream/py-lmdb_0.61
 = [up to date]      py-lmdb_0.62 -> upstream/py-lmdb_0.62
 = [up to date]      py-lmdb_0.63 -> upstream/py-lmdb_0.63
 = [up to date]      py-lmdb_0.64 -> upstream/py-lmdb_0.64
 = [up to date]      py-lmdb_0.65 -> upstream/py-lmdb_0.65
 = [up to date]      py-lmdb_0.66 -> upstream/py-lmdb_0.66
 = [up to date]      py-lmdb_0.67 -> upstream/py-lmdb_0.67
 = [up to date]      py-lmdb_0.68 -> upstream/py-lmdb_0.68
 = [up to date]      py-lmdb_0.69 -> upstream/py-lmdb_0.69
 = [up to date]      py-lmdb_0.70 -> upstream/py-lmdb_0.70
 = [up to date]      py-lmdb_0.71 -> upstream/py-lmdb_0.71
 = [up to date]      py-lmdb_0.72 -> upstream/py-lmdb_0.72
 = [up to date]      py-lmdb_0.73 -> upstream/py-lmdb_0.73
 = [up to date]      py-lmdb_0.74 -> upstream/py-lmdb_0.74
 = [up to date]      py-lmdb_0.75 -> upstream/py-lmdb_0.75
 = [up to date]      py-lmdb_0.76 -> upstream/py-lmdb_0.76
 = [up to date]      py-lmdb_0.77 -> upstream/py-lmdb_0.77
 = [up to date]      py-lmdb_0.78 -> upstream/py-lmdb_0.78
 = [up to date]      py-lmdb_0.79 -> upstream/py-lmdb_0.79
 = [up to date]      py-lmdb_0.80 -> upstream/py-lmdb_0.80
 = [up to date]      py-lmdb_0.81 -> upstream/py-lmdb_0.81
 = [up to date]      py-lmdb_0.82 -> upstream/py-lmdb_0.82
 = [up to date]      py-lmdb_0.83 -> upstream/py-lmdb_0.83
 = [up to date]      py-lmdb_0.84 -> upstream/py-lmdb_0.84
 = [up to date]      py-lmdb_0.85 -> upstream/py-lmdb_0.85
 = [up to date]      py-lmdb_0.86 -> upstream/py-lmdb_0.86
We can check if any co-maintainers have pushed updates to the git.debian.org repository with the following command: 
$ git fetch debian --dry-run -v
From ssh://git.debian.org/git/pkg-db/py-lmdb
 = [up to date]      debian/sid -> debian/debian/sid
 = [up to date]      pristine-tar -> debian/pristine-tar
 = [up to date]      debian/0.84-1 -> debian/0.84-1
 = [up to date]      debian/0.86-1 -> debian/0.86-1
 = [up to date]      upstream/last-cython-version -> upstream/last-cython-version
 = [up to date]      upstream/py-lmdb_0.1 -> upstream/py-lmdb_0.1
 = [up to date]      upstream/py-lmdb_0.2 -> upstream/py-lmdb_0.2
 = [up to date]      upstream/py-lmdb_0.3 -> upstream/py-lmdb_0.3
 = [up to date]      upstream/py-lmdb_0.4 -> upstream/py-lmdb_0.4
 = [up to date]      upstream/py-lmdb_0.5 -> upstream/py-lmdb_0.5
 = [up to date]      upstream/py-lmdb_0.51 -> upstream/py-lmdb_0.51
 = [up to date]      upstream/py-lmdb_0.52 -> upstream/py-lmdb_0.52
 = [up to date]      upstream/py-lmdb_0.53 -> upstream/py-lmdb_0.53
 = [up to date]      upstream/py-lmdb_0.54 -> upstream/py-lmdb_0.54
 = [up to date]      upstream/py-lmdb_0.56 -> upstream/py-lmdb_0.56
 = [up to date]      upstream/py-lmdb_0.57 -> upstream/py-lmdb_0.57
 = [up to date]      upstream/py-lmdb_0.58 -> upstream/py-lmdb_0.58
 = [up to date]      upstream/py-lmdb_0.59 -> upstream/py-lmdb_0.59
 = [up to date]      upstream/py-lmdb_0.60 -> upstream/py-lmdb_0.60
 = [up to date]      upstream/py-lmdb_0.61 -> upstream/py-lmdb_0.61
 = [up to date]      upstream/py-lmdb_0.62 -> upstream/py-lmdb_0.62
 = [up to date]      upstream/py-lmdb_0.63 -> upstream/py-lmdb_0.63
 = [up to date]      upstream/py-lmdb_0.64 -> upstream/py-lmdb_0.64
 = [up to date]      upstream/py-lmdb_0.65 -> upstream/py-lmdb_0.65
 = [up to date]      upstream/py-lmdb_0.66 -> upstream/py-lmdb_0.66
 = [up to date]      upstream/py-lmdb_0.67 -> upstream/py-lmdb_0.67
 = [up to date]      upstream/py-lmdb_0.68 -> upstream/py-lmdb_0.68
 = [up to date]      upstream/py-lmdb_0.69 -> upstream/py-lmdb_0.69
 = [up to date]      upstream/py-lmdb_0.70 -> upstream/py-lmdb_0.70
 = [up to date]      upstream/py-lmdb_0.71 -> upstream/py-lmdb_0.71
 = [up to date]      upstream/py-lmdb_0.72 -> upstream/py-lmdb_0.72
 = [up to date]      upstream/py-lmdb_0.73 -> upstream/py-lmdb_0.73
 = [up to date]      upstream/py-lmdb_0.74 -> upstream/py-lmdb_0.74
 = [up to date]      upstream/py-lmdb_0.75 -> upstream/py-lmdb_0.75
 = [up to date]      upstream/py-lmdb_0.76 -> upstream/py-lmdb_0.76
 = [up to date]      upstream/py-lmdb_0.77 -> upstream/py-lmdb_0.77
 = [up to date]      upstream/py-lmdb_0.78 -> upstream/py-lmdb_0.78
 = [up to date]      upstream/py-lmdb_0.79 -> upstream/py-lmdb_0.79
 = [up to date]      upstream/py-lmdb_0.80 -> upstream/py-lmdb_0.80
 = [up to date]      upstream/py-lmdb_0.81 -> upstream/py-lmdb_0.81
 = [up to date]      upstream/py-lmdb_0.82 -> upstream/py-lmdb_0.82
 = [up to date]      upstream/py-lmdb_0.83 -> upstream/py-lmdb_0.83
 = [up to date]      upstream/py-lmdb_0.84 -> upstream/py-lmdb_0.84
 = [up to date]      upstream/py-lmdb_0.85 -> upstream/py-lmdb_0.85
 = [up to date]      upstream/py-lmdb_0.86 -> upstream/py-lmdb_0.86
$
We can check if anything needs to be pushed from our local repository to the git.debian.org repository with the following command: 
$ git push debian --dry-run -v
Pushing to ssh://git.debian.org/git/pkg-db/py-lmdb.git
To ssh://git.debian.org/git/pkg-db/py-lmdb.git
 = [up to date]      debian/sid -> debian/sid
 = [up to date]      pristine-tar -> pristine-tar
 = [up to date]      debian/0.84-1 -> debian/0.84-1
 = [up to date]      debian/0.86-1 -> debian/0.86-1
 = [up to date]      upstream/last-cython-version -> upstream/last-cython-version
 = [up to date]      upstream/py-lmdb_0.1 -> upstream/py-lmdb_0.1
 = [up to date]      upstream/py-lmdb_0.2 -> upstream/py-lmdb_0.2
 = [up to date]      upstream/py-lmdb_0.3 -> upstream/py-lmdb_0.3
 = [up to date]      upstream/py-lmdb_0.4 -> upstream/py-lmdb_0.4
 = [up to date]      upstream/py-lmdb_0.5 -> upstream/py-lmdb_0.5
 = [up to date]      upstream/py-lmdb_0.51 -> upstream/py-lmdb_0.51
 = [up to date]      upstream/py-lmdb_0.52 -> upstream/py-lmdb_0.52
 = [up to date]      upstream/py-lmdb_0.53 -> upstream/py-lmdb_0.53
 = [up to date]      upstream/py-lmdb_0.54 -> upstream/py-lmdb_0.54
 = [up to date]      upstream/py-lmdb_0.56 -> upstream/py-lmdb_0.56
 = [up to date]      upstream/py-lmdb_0.57 -> upstream/py-lmdb_0.57
 = [up to date]      upstream/py-lmdb_0.58 -> upstream/py-lmdb_0.58
 = [up to date]      upstream/py-lmdb_0.59 -> upstream/py-lmdb_0.59
 = [up to date]      upstream/py-lmdb_0.60 -> upstream/py-lmdb_0.60
 = [up to date]      upstream/py-lmdb_0.61 -> upstream/py-lmdb_0.61
 = [up to date]      upstream/py-lmdb_0.62 -> upstream/py-lmdb_0.62
 = [up to date]      upstream/py-lmdb_0.63 -> upstream/py-lmdb_0.63
 = [up to date]      upstream/py-lmdb_0.64 -> upstream/py-lmdb_0.64
 = [up to date]      upstream/py-lmdb_0.65 -> upstream/py-lmdb_0.65
 = [up to date]      upstream/py-lmdb_0.66 -> upstream/py-lmdb_0.66
 = [up to date]      upstream/py-lmdb_0.67 -> upstream/py-lmdb_0.67
 = [up to date]      upstream/py-lmdb_0.68 -> upstream/py-lmdb_0.68
 = [up to date]      upstream/py-lmdb_0.69 -> upstream/py-lmdb_0.69
 = [up to date]      upstream/py-lmdb_0.70 -> upstream/py-lmdb_0.70
 = [up to date]      upstream/py-lmdb_0.71 -> upstream/py-lmdb_0.71
 = [up to date]      upstream/py-lmdb_0.72 -> upstream/py-lmdb_0.72
 = [up to date]      upstream/py-lmdb_0.73 -> upstream/py-lmdb_0.73
 = [up to date]      upstream/py-lmdb_0.74 -> upstream/py-lmdb_0.74
 = [up to date]      upstream/py-lmdb_0.75 -> upstream/py-lmdb_0.75
 = [up to date]      upstream/py-lmdb_0.76 -> upstream/py-lmdb_0.76
 = [up to date]      upstream/py-lmdb_0.77 -> upstream/py-lmdb_0.77
 = [up to date]      upstream/py-lmdb_0.78 -> upstream/py-lmdb_0.78
 = [up to date]      upstream/py-lmdb_0.79 -> upstream/py-lmdb_0.79
 = [up to date]      upstream/py-lmdb_0.80 -> upstream/py-lmdb_0.80
 = [up to date]      upstream/py-lmdb_0.81 -> upstream/py-lmdb_0.81
 = [up to date]      upstream/py-lmdb_0.82 -> upstream/py-lmdb_0.82
 = [up to date]      upstream/py-lmdb_0.83 -> upstream/py-lmdb_0.83
 = [up to date]      upstream/py-lmdb_0.84 -> upstream/py-lmdb_0.84
 = [up to date]      upstream/py-lmdb_0.85 -> upstream/py-lmdb_0.85
 = [up to date]      upstream/py-lmdb_0.86 -> upstream/py-lmdb_0.86
Everything up-to-date
Finally, in order to set up a fresh local clone of the git.debian.org repository that's configured like the local repository created above, we have to do the following: 
$ git clone --origin debian ssh://git.debian.org/git/pkg-db/py-lmdb.git
Cloning into 'py-lmdb'...
remote: Counting objects: 3333, done.
remote: Compressing objects: 100% (1070/1070), done.
remote: Total 3333 (delta 2231), reused 3333 (delta 2231)
Receiving objects: 100% (3333/3333), 1.37 MiB   1.11 MiB/s, done.
Resolving deltas: 100% (2231/2231), done.
Checking connectivity... done.
$ cd py-lmdb
$ git remote add --no-tags upstream https://github.com/dw/py-lmdb
$ git config --add remote.upstream.fetch 'refs/tags/*:refs/tags/upstream/*'
$ git fetch upstream
remote: Counting objects: 56, done.
remote: Total 56 (delta 25), reused 25 (delta 25), pack-reused 31
Unpacking objects: 100% (56/56), done.
From https://github.com/dw/py-lmdb
 * [new branch]      master     -> upstream/master
 * [new branch]      release    -> upstream/release
 * [new branch]      win32-sparse-patch -> upstream/win32-sparse-patch
$ git branch --track pristine-tar debian/pristine-tar 
Branch pristine-tar set up to track remote branch pristine-tar from debian.
$ git config --add remote.debian.push 'refs/tags/debian/*'
$ git config --add remote.debian.push 'refs/tags/upstream/*'
$ git config --add remote.debian.push 'refs/heads/debian/*'
$ git config --add remote.debian.push 'refs/heads/pristine-tar'
$ git config --add remote.debian.fetch 'refs/tags/*:refs/tags/*'
$
This is a fair amount of effort beyond a simple git clone, though, so I wonder if anything can be done to optimize this.

2 July 2015

Michael Prokop: HAProxy with Debian/squeeze clients causing random Hash Sum mismatch

Update on 2015-07-02 22:15 UTC: as Petter Reinholdtsen noted in the comments:
Try adding /etc/apt/apt.conf.d/90squid with content like this: Acquire::http::Pipeline-Depth 0; It turn off the feature in apt confusing proxies.
this indeed avoids those Hash Sum mismatch failures with HAProxy as well. Thanks, Petter! Many of you might know apt s Hash Sum mismatch issue and there are plenty of bug reports about it (like #517874, #624122, #743298 + #762079). Recently I saw the Hash Sum mismatch usually only when using random mirrors with e.g. httpredir.debian.org in apt s sources.list, but with a static mirror such issues usually don t exist anymore. A customer of mine has a Debian mirror and this issue wasn t a problem there neither, until recently: Since the mirror also includes packages provided to customers and the mirror needs to be available 24/7 we decided to provide another instance of the mirror and put those systems behind HAProxy (version 1.5.8-3 as present in Debian/jessie). The HAProxy setup worked fine and we didn t notice any issues in our tests, until the daily Q/A builds randomly started to report failures: 
Failed to fetch http://example.org/foobar_amd64.deb Hash Sum mismatch
When repeating the download there was no problem though. This problem only appeared about once every 15-20 minutes with random package files and it affected only Debian/squeeze clients (wheezy and jessie aren t affected at all). The problem also didn t appear when directly accessing the mirrors behind HAproxy. We tried plenty of different options for apt (Acquire::http::No-Cache=true, Acquire::http::No-Partial=true, ) and also played with some HAProxy configurations, nothing really helped. With apt s Debug::Acquire::http=True we saw that there really was a checksum failure and HTTP status code 102 ( Processing , or in terms of apt: Waiting for headers ) seems to be involved. The actual problem between apt on Debian/squeeze and HAProxy is still unknown to us though. While digging deeper into this issue is on my todo list yet, I found a way to avoid those Hash Sum mismatch failures: switch from http to https in sources.list. As soon as https is used the problem doesn t appear anymore. I m documenting it here just in case anyone else should run into it.

Norbert Preining: An amusing lintian error Lenna

Well, there we are, trying to build another round of TeX Live packages for Debian, just to realize that the lintian error that should have been downgraded to warning (or removed) is still around, due to doubts about the license. Ok. Well, anyway, but what I found is even more funny: 
E: texlive-extra source: license-problem-non-free-img-lenna texmf-dist/doc/latex/reflectgraphics/lenna.jpg
which is about one of the most used images in images processing courses, Lenna: who-is-lenna
Without comments, I just quote the lintian error it is a whole lot of fun to read.
Ref: https://en.wikipedia.org/wiki/Lenna, https://www.debian.org/vote/2012/vote_002, #771191
Info: The given source file is cropped from playboy centerfold. This image is a picture of Lena S derberg, shot by photographer Dwight Hooker, cropped from the centerfold of the November 1972 issue of Playboy magazine. According to Hutchison, Jamie (May June 2001). Culture, Communication, and an Information Age Madonna (PDF). IEEE Professional Communication Society Newsletter 45 (3). (page 5 second column second paragraph), this image is distributable but not free. Moreover, Lenna photo has been pointed to as an example of sexism in the sciences, reinforcing gender stereotypes. Please use well known and free test image. Please also submit md5sum, sha1sum, and sha256 of this file as a bug report for lintian.
How fortunate our generation is that we don t have anything else to care about Anyway, back to rebuilding orig.tars, source packages, and binary packages!

29 June 2015

Lunar: Reproducible builds: week 9 in Stretch cycle

What happened about the reproducible builds effort this week: Toolchain fixes Norbert Preining uploaded texinfo/6.0.0.dfsg.1-2 which makes texinfo indices reproducible. Original patch by Chris Lamb. Lunar submitted recently rebased patches to make the file order of files inside .deb stable. akira filled #789843 to make tex4ht stop printing timestamps in its HTML output by default. Dhole wrote a patch for xutils-dev to prevent timestamps when creating gzip compresed files. Reiner Herrmann sent a follow-up patch for wheel to use UTC as timezone when outputing timestamps. Mattia Rizzolo started a discussion regarding the failure to build from source of subversion when -Wdate-time is added to CPPFLAGS which happens when asking dpkg-buildflags to use the reproducible profile. SWIG errors out because it doesn't recognize the aforementioned flag. Trying to get the .buildinfo specification to more definitive state, Lunar started a discussion on storing the checksums of the binary package used in dpkg status database. akira discovered while proposing a fix for simgrid that CMake internal command to create tarballs would record a timestamp in the gzip header. A way to prevent it is to use the GZIP environment variable to ask gzip not to store timestamps, but this will soon become unsupported. It's up for discussion if the best place to fix the problem would be to fix it for all CMake users at once. Infrastructure-related work Andreas Henriksson did a delayed NMU upload of pbuilder which adds minimal support for build profiles and includes several fixes from Mattia Rizzolo affecting reproducibility tests. Neils Thykier uploaded lintian which both raises the severity of package-contains-timestamped-gzip and avoids false positives for this tag (thanks to Tomasz Buchert). Petter Reinholdtsen filled #789761 suggesting that how-can-i-help should prompt its users about fixing reproducibility issues. Packages fixed The following packages became reproducible due to changes in their build dependencies: autorun4linuxcd, libwildmagic, lifelines, plexus-i18n, texlive-base, texlive-extra, texlive-lang. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues but not all of them: Untested uploaded as they are not in main: Patches submitted which have not made their way to the archive yet: debbindiff development debbindiff/23 includes a few bugfixes by Helmut Grohne that result in a significant speedup (especially on larger files). It used to exhibit the quadratic time string concatenation antipattern. Version 24 was released on June 23rd in a hurry to fix an undefined variable introduced in the previous version. (Reiner Herrmann) debbindiff now has a test suite! It is written using the PyTest framework (thanks Isis Lovecruft for the suggestion). The current focus has been on the comparators, and we are now at 93% of code coverage for these modules. Several problems were identified and fixed in the process: paths appearing in output of javap, readelf, objdump, zipinfo, unsqusahfs; useless MD5 checksum and last modified date in javap output; bad handling of charsets in PO files; the destination path for gzip compressed files not ending in .gz; only metadata of cpio archives were actually compared. stat output was further trimmed to make directory comparison more useful. Having the test suite enabled a refactoring of how comparators were written, switching from a forest of differences to a single tree. This helped removing dust from the oldest parts of the code. Together with some other small changes, version 25 was released on June 27th. A follow up release was made the next day to fix a hole in the test suite and the resulting unidentified leftover from the comparator refactoring. (Lunar) Documentation update Ximin Luo improved code examples for some proposed environment variables for reference timestamps. Dhole added an example on how to fix timestamps C pre-processor macros by adding a way to set the build date externally. akira documented her fix for tex4ht timestamps. Package reviews 94 obsolete reviews have been removed, 330 added and 153 updated this week. Hats off for Chris West (Faux) who investigated many fail to build from source issues and reported the relevant bugs. Slight improvements were made to the scripts for editing the review database, edit-notes and clean-notes. (Mattia Rizzolo) Meetings A meeting was held on June 23rd. Minutes are available. The next meeting will happen on Tuesday 2015-07-07 at 17:00 UTC. Misc. The Linux Foundation announced that it was funding the work of Lunar and h01ger on reproducible builds in Debian and other distributions. This was further relayed in a Bits from Debian blog post.

Next.

Previous.