In recent years, privacy issues have become a growing concern among free-software projects and users. As more and more software tasks become web-based, surveillance and tracking of users is also on the rise. While some software may use advertising as a source of revenue, which has the side effect of monitoring users, the Django community recently got into an interesting debate surrounding a proposal to add user tracking actually developer tracking to the popular Python web framework.

Tracking for funding A novel aspect of this debate is that the initiative comes from concerns of the Django Software Foundation (DSF) about funding. The proposal suggests that "relying on the free labor of volunteers is ineffective, unfair, and risky" and states that "the future of Django depends on our ability to fund its development". In fact, the DSF recently hired an engineer to help oversee Django's development, which has been quite successful in helping the project make timely releases with fewer bugs. Various fundraising efforts have resulted in major new Django features, but it is difficult to attract sponsors without some hard data on the usage of Django. The proposed feature tries to count the number of "unique developers" and gather some metrics of their environments by using Google Analytics (GA) in Django. The actual proposal (DEP 8) is done as a pull request, which is part of Django Enhancement Proposal (DEP) process that is similar in spirit to the Python Enhancement Proposal (PEP) process. DEP 8 was brought forward by a longtime Django developer, Jacob Kaplan-Moss. The rationale is that "if we had clear data on the extent of Django's usage, it would be much easier to approach organizations for funding". The proposal is essentially about adding code in Django to send a certain set of metrics when "developer" commands are run. The system would be "opt-out", enabled by default unless turned off, although the developer would be warned the first time the phone-home system is used. The proposal notes that an opt-in system "severely undercounts" and is therefore not considered "substantially better than a community survey" that the DSF is already doing.

Information gathered The pieces of information reported are specifically designed to run only in a developer's environment and not in production. The metrics identified are, at the time of writing:

• an event category (the developer commands: startproject, startapp, runserver)
• the HTTP User-Agent string identifying the Django, Python, and OS versions
• a user-specific unique identifier (a UUID generated on first run)

The proposal mentions the use of the GA aip flag which, according to GA documentation, makes "the IP address of the sender 'anonymized'". It is not quite clear how that is done at Google and, given that it is a proprietary platform, there is no way to verify that claim. The proposal says it means that "we can't see, and Google Analytics doesn't store, your actual IP". But that is not actually what Google does: GA stores IP addresses, the documentation just says they are anonymized, without explaining how. GA is presented as a trade-off, since "Google's track record indicates that they don't value privacy nearly as high" as the DSF does. The alternative, deploying its own analytics software, was presented as making sustainability problems worse. According to the proposal, Google "can't track Django users. [...] The only thing Google could do would be to lie about anonymizing IP addresses, and attempt to match users based on their IPs". The truth is that we don't actually know what Google means when it "anonymizes" data: Jannis Leidel, a Django team member, commented that "Google has previously been subjected to secret US court orders and was required to collaborate in mass surveillance conducted by US intelligence services" that limit even Google's capacity of ensuring its users' anonymity. Leidel also argued that the legal framework of the US may not apply elsewhere in the world: "for example the strict German (and by extension EU) privacy laws would exclude the automatic opt-in as a lawful option". Furthermore, the proposal claims that "if we discovered Google was lying about this, we'd obviously stop using them immediately", but it is unclear exactly how this could be implemented if the software was already deployed. There are also concerns that an implementation could block normal operation, especially in countries (like China) where Google itself may be blocked. Finally, some expressed concerns that the information could constitute a security problem, since it would unduly expose the version number of Django that is running.

In other projects Django is certainly not the first project to consider implementing analytics to get more information about its users. The proposal is largely inspired by a similar system implemented by the OS X Homebrew package manager, which has its own opt-out analytics. Other projects embed GA code directly in their web pages. This is apparently the option chosen by the Oscar Django-based ecommerce solution, but that was seen by the DSF as less useful since it would count Django administrators and wasn't seen as useful as counting developers. Wagtail, a Django-based content-management system, was incorrectly identified as using GA directly, as well. It actually uses referrer information to identify installed domains through the version updates checks, with opt-out. Wagtail didn't use GA because the project wanted only minimal data and it was worried about users' reactions. NPM, the JavaScript package manager, also considered similar tracking extensions. Laurie Voss, the co-founder of NPM, said it decided to completely avoid phoning home, because "users would absolutely hate it". But NPM users are constantly downloading packages to rebuild applications from scratch, so it has more complete usage metrics, which are aggregated and available via a public API. NPM users seem to find this is a "reasonable utility/privacy trade". Some NPM packages do phone home and have seen "very mixed" feedback from users, Voss said. Eric Holscher, co-founder of Read the Docs, said the project is considering using Sentry for centralized reporting, which is a different idea, but interesting considering Sentry is fully open source. So even though it is a commercial service (as opposed to the closed-source Google Analytics), it may be possible to verify any anonymity claims.

Debian's response Since Django is shipped with Debian, one concern was the reaction of the distribution to the change. Indeed, "major distros' positions would be very important for public reception" to the feature, another developer stated. One of the current maintainers of Django in Debian, Rapha l Hertzog, explicitly stated from the start that such a system would "likely be disabled by default in Debian". There were two short discussions on Debian mailing lists where the overall consensus seemed to be that any opt-out tracking code was undesirable in Debian, especially if it was aimed at Google servers. I have done some research to see what, exactly, was acceptable as a phone-home system in the Debian community. My research has revealed ten distinct bug reports against packages that would unexpectedly connect to the network, most of which were not directly about collecting statistics but more often about checking for new versions. In most cases I found, the feature was disabled. In the case of version checks, it seems right for Debian to disable the feature, because the package cannot upgrade itself: that task is delegated to the package manager. One of those issues was the infamous "OK Google" voice activation binary blog controversy that was previously reported here and has since then been fixed (although other issues remain in Chromium). I have also found out that there is no clearly defined policy in Debian regarding tracking software. What I have found, however, is that there seems to be a strong consensus in Debian that any tracking is unacceptable. This is, for example, an extract of a policy that was drafted (but never formally adopted) by Ian Jackson, a longtime Debian developer:

Software in Debian should not communicate over the network except: in order to, and as necessary to, perform their function[...]; or for other purposes with explicit permission from the user.

In other words, opt-in only, period. Jackson explained that "when we originally wrote the core of the policy documents, the DFSG [Debian Free Software Guidelines], the SC [Social Contract], and so on, no-one would have considered this behaviour acceptable", which explains why no explicit formal policy has been adopted yet in the Debian project. One of the concerns with opt-out systems (or even prompts that default to opt-in) was well explained back then by Debian developer Bas Wijnen:

It very much resembles having to click through a license for every package you install. One of the nice things about Debian is that the user doesn't need to worry about such things: Debian makes sure things are fine.

One could argue that Debian has its own tracking systems. For example, by default, Debian will "phone home" through the APT update system (though it only reports the packages requested). However, this is currently not automated by default, although there are plans to do so soon. Furthermore, Debian members do not consider APT as tracking, because it needs to connect to the network to accomplish its primary function. Since there are multiple distributed mirrors (which the user gets to choose when installing), the risk of surveillance and tracking is also greatly reduced. A better parallel could be drawn with Debian's popcon system, which actually tracks Debian installations, including package lists. But as Barry Warsaw pointed out in that discussion, "popcon is 'opt-in' and [...] the overwhelming majority in Debian is in favour of it in contrast to 'opt-out'". It should be noted that popcon, while opt-in, defaults to "yes" if users click through the install process. [Update: As pointed out in the comments, popcon actually defaults to "no" in Debian.] There are around 200,000 submissions at this time, which are tracked with machine-specific unique identifiers that are submitted daily. Ubuntu, which also uses the popcon software, gets around 2.8 million daily submissions, while Canonical estimates there are 40 million desktop users of Ubuntu. This would mean there is about an order of magnitude more installations than what is reported by popcon. Policy aside, Warsaw explained that "Debian has a reputation for taking privacy issues very serious and likes to keep it".

Next steps There are obviously disagreements within the Django project about how to handle this problem. It looks like the phone-home system may end up being implemented as a proxy system "which would allow us to strip IP addresses instead of relying on Google to anonymize them, or to anonymize them ourselves", another Django developer, Aymeric Augustin, said. Augustin also stated that the feature wouldn't "land before Django drops support for Python 2", which is currently estimated to be around 2020. It is unclear, then, how the proposal would resolve the funding issues, considering how long it would take to deploy the change and then collect the information so that it can be used to spur the funding efforts. It also seems the system may explicitly prompt the user, with an opt-out default, instead of just splashing a warning or privacy agreement without a prompt. As Shai Berger, another Django contributor, stated, "you do not get [those] kind of numbers in community surveys". Berger also made the argument that "we trust the community to give back without being forced to do so"; furthermore:

I don't believe the increase we might get in the number of reports by making it harder to opt-out, can be worth the ill-will generated for people who might feel the reporting was "sneaked" upon them, or even those who feel they were nagged into participation rather than choosing to participate.

Other options may also include gathering metrics in pip or PyPI, which was proposed by Donald Stufft. Leidel also proposed that the system could ask to opt-in only after a few times the commands are called. It is encouraging to see that a community can discuss such issues without heating up too much and shows great maturity for the Django project. Every free-software project may be confronted with funding and sustainability issues. Django seems to be trying to address this in a transparent way. The project is willing to engage with the whole spectrum of the community, from the top leaders to downstream distributors, including individual developers. This practice should serve as a model, if not of how to do funding or tracking, at least of how to discuss those issues productively. Everyone seems to agree the point is not to surveil users, but improve the software. As Lars Wirzenius, a Debian developer, commented: "it's a very sad situation if free software projects have to compromise on privacy to get funded". Hopefully, Django will be able to improve its funding without compromising its principles.

Note: this article first appeared in the Linux Weekly News.

This month s update falls onto a national holiday in Japan. My recent start as a normal company employee in Japan doesn t leave me enough time during normal days to work on Debian, so things have to wait for holidays. There have been a few notable changes in the current packages, and above all I wanted to fix an RC bug and on the way fixed also several other (sometimes rather old) bugs.
From the list of new packages I want to pick apxproof: I have written something myself for one of my rather long papers (with proofs about 60pp), where at times I had to factor out the proofs into an appendix. I did this my own way, but I would have preferred to have a nice package! Another interesting change is the upstream merge of collection-mathextra (which translated to the Debian package texlive-math-extra) and collection-science (Debian: texlive-science) into a new collection collection-mathscience. Since introducing new packages and phasing out old ones is generally a pain in Debian, I decided to digress from the upstream naming convention and use texlive-science for the new collection-mathscience. In the end Mathematics is the most important science of all Finally also a word about removals: Several ConTeXt packages have been removed due to the fact that they are outdated. These removals will find their way in an update of the Debian ConTeXt package in near future. The TeX Live packages lost voss-mathmode, which was retracted by the author due to various reasons. He is working on an updated version that will hopeful reappear in both TeX Live and Debian in near future. Well, that s it for now. Here now the full list with links. Enjoy. New packages apxproof, bangorexam, biblatex-gb7714-2015, biblatex-lni, biblatex-sbl, context-cmscbf, context-cmttbf, context-inifile, context-layout, delimset, latex2nemeth, latexbangla, latex-papersize, ling-macros, notex-bst, platex-tools, testidx, uppunctlm, wtref, xcolor-material. Removed packages voss-mathmode. Updated packages apa6, autoaligne, babel-german, biblatex-abnt, biblatex-anonymous, biblatex-apa, biblatex-manuscripts-philology, biblatex-nature, biblatex-realauthor, bibtex, bidi, boondox, bxcjkjatype, chickenize, churchslavonic, cjk-gs-integrate, context-filter, cooking-units, ctex, denisbdoc, dvips, europasscv, fixme, glossaries, gzt, handout, imakeidx, ipaex-type1, jsclasses, jslectureplanner, kpathsea, l3build, l3experimental, l3kernel, l3packages, latexindent, latexmk, listofitems, luatexja, marginnote, mcf2graph, minted, multirow, nameauth, newpx, newtx, noto, nucleardata, optidef, overlays, pdflatexpicscale, pst-eucl, reledmac, repere, scanpages, semantic-markup, tableaux, tcolorbox, tetex, texlive-scripts, ticket, todonotes, tracklang, tudscr, turabian-formatting, updmap-map, uspace, visualtikz, xassoccnt, xecjk, yathesis.

I ve been watching the June budget and pondering its effects on me, our co-op, co-ops in general and the wider community.

IT Taxes

I just want to flag this one up as a special interest: the telephone tax is killed off before it starts. Yippee! But now we get to wait and see how the government will support private investment to get universal provision of fast broadband. I ve no problem with tax relief for the video games industry being scrapped - why should any type of software get special treatment?

VAT up 2.5% to 20% from 4 January 2011

This seems bad for everyone. It s a bit less bad for our co-op because we have some international suppliers, whose sales tax rates won t change, but we buy a lot from the UK too and it hurts all our workers.

Personal Taxes

The basic income tax allowance rise of 1,000 is welcome, as is restoring the pension-earnings links. More widely, the freezes in various things, increase in Capital Gains Tax and drop in cider tax all seem broadly good ideas. The housing benefit 400 maximum is a bit mixed - how bad it is may depend how lenders react when borrowers get into difficulty.

Business Taxes

At a time when a VAT increase is called unavoidable , it stinks a bit to cut corporation tax, extend the 10% capital gains tax allowance and raise the employers National Insurance threshold. Of course, our co-op pays none of those, so it also hurts us by giving our competitors an advantage.

Business Incentives

George Osborne said he wants to tackle regional economic differences, but the big change in this budget is bad for all existing businesses outside London, the South-East and East: National Insurance exemptions for new businesses. Once again, existing co-ops take it in the neck from another government obsessed with capitalism and employment instead of businesses and work. Meanwhile, increasing the Enterprise Finance Guarantee props up debt-laden business and a new Growth Capital Fund encourages capitalist businesses, while both appear useless to good co-ops at first glance. I don t mind being ignored, but could we please elect a government which doesn t actively hinder co-ops? Where s the fabled commitment to fairness ? Promises around the Green Investment Bank and Green Deal sound good, but are in the future.

Council Tax

Council Tax will be frozen for a year. I wonder if that applies to parish councils, because our village planned a cut after a one-year project-based increase last year. The Budget Document has pretty much no detail.

So, how was it for you? Have I missed some friendly changes?

The Trade Union Congress s chief executive recently came out in the crazy position of sharing a platform with the British Phonographic Industry (and why are they still relevant? What is the market share of phonographs now?) and supporting the much-hated Digital Britain guilty-without-trial three strikes disconnect measures. Yes, you read that right: the TUC s chief exec wants digital economy workers to be arbitrarily disconnectable in response to allegations of file-sharing by large competitors. If I m being charitable, I think he was duped. That stupid article was in response to yet another set of filesharing-will-fankle-us scare figures which carefully only included the creative industries most impacted by piracy (see second paragraph of introduction). See pages 12 and 14 of the full report to note that businesses that don t depend on profit from copyright licensing were not even included! So, these are not the creative industries. They are the KEA and WIPO definitions of copyright industries. Of course they are hurt by copyright infringement and have little to lose from over-zealous copyright laws, but they are not the creative industries. I contacted the TUC to basically ask why they were so against digital economy workers in this way. I received an anonymous answer from TUC Information Services that was almost helpful:

The current policy of the TUC on this issue was decided at our annual congress last autumn. and directing me to Composite Motions 18 and 19 (PDF) and the verbatim report of the conference (PDF) pp. 107-9

Composite Motion 18 seems largely innocent. In fact, its call for social inclusion, a Universal Service Obligation and digital inclusion could be seen as an instruction to oppose non-judicial three-strikes, because allowing so-called Kangkaroo courts run by private-sector transnationals to decide who may use the internet is basically incompatible with inclusion and universal service. The NUJ-supported Composite Motion 19 is the dangerous one and I m alarmed that the claims around piracy are completely without evidence or justification in the verbatim report of the conference. Were references supplied, or does the TUC adopt positions based solely on reference-free argument? How does the TUC reconcile contradictory motions for inclusion and universal service and for exclusion through private courts? Their answer?

The TUC does not, as a rule, enter into correspondence with individual union members [...]

Oh great(!) You are in a maze of twisted union organisations and none of the ones which are screwing things up are accountable to you. Thankfully, I have happier Digital Britain news from another source which I should write about soon.

Amazon Kindle Un-selling Books was the most-viewed news item on my blog this year. Once again, a lock-out/bad-business story takes the top spot. I don t think that one got as much coverage in the mainstream media as they usually do, which probably contributed to the relatively high reader numbers. I feel like I get criticised often for commenting about business ethics, but it does seem to attract a fairly big audience. The repost of Tribute to Richard Rothwell came a fairly close second. Again, not something that the mainstream tech news covered. the world is a sadder and lonelier place without him, and the rest of the world has lost one of its brightest stars. Top-placed technical item was HOWTO Apache httpd 2.2 PAM Authentication Modules which was a distant third. It was posted in late 2008, but I suspect that will keep getting viewers until everyone s upgraded. It s currently the top recipient of search engine traffic for the site. Other technical stories in the top ten were Windows 7: Released with known critical bug in fifth, ssh security in sixth and Top 8 J2ME MIDP Applications in tenth place. The only completely non-technical topic was the Cooperatives-SW Board and Housing Enquiry in ninth place. Cross-over topics complete the ten with 2009 Software in the Public Interest Board Election in fourth place, New #ukgovOSS Action Plan in seventh and Meeting about forming a Koha Foundation in eighth. Finally, I think the strangest three search phrases ending at my site this year are spi bike week , request free t-shirt and will britannia building society demutualise . Happy New Year! Found any easter eggs in your web stats?

High-speed internet links are something that makes FOSS development much easier, but I m still having reliability problems with my ADSL. A fringe benefit of moving to a more open Linux-based router is that I can see what s going wrong in more detail. But now I have to gamble nearly two hundred pounds in order to get the service providers to continue trying to find the bug! Sorry for the length, but let me explain how bad broadband in rural England is just now Outside the big cities with their cable networks, Company A owns the lines, Company B provides the services and we buy those services from reseller Company C. Sometimes the same company is A and B or B and C, but never all three. Most of the companies who are B won t be C themselves - in other words, they don t sell retail, but they nearly always seem to have a main or preferred retail partner. Company A is nearly always OpenReach, part of BT Group, the telephone monopoly privatised back in the 1980s but still somehow keeping its landline monopoly. Why is that allowed? Company C has been The Phone Co-op for me for the last few years. Back in February, I changed how I bought my services, combining phone and broadband into one purchase. They d been two purchases mainly because I switched my ADSL from Pipex to the co-op at a different time to switching my line to them. As I understand it, this change had the side effect of changing Company B from OpenReach to Opal Telecom. Opal are the provider business behind TalkTalk, hardest to use Broadband ISP 2009. Since the change, I ve been suffering unexplained disconnections, especially in the early evenings and at weekends. I m pretty sure it s Opal s equipment to blame for two reasons:- Firstly, this wasn t happening until Opal became Company B, while Companies A and C are the same as before (when I had a stable but unspectacular 4Mbps connection). Secondly, since changing my router, I can see LCP TermReq being sent by Opal s PPP peers. Opal insist their equipment isn t sending TermReq messages. My Netgear router disagrees, but I can t seem to get anyone to take its pppd debug logs seriously. So, I ve complained to my supplier (Company C), they ve pretty straightforwardly passed it to Company B and I ve jumped through all the bloody silly hoops like using only wired connections to the router (yay, wires across the house), plugging the router into different phone sockets (yay, more wires across the house) and disconnecting everything from the phone sockets so Company A can run some voltage test (yay, uncontactability - just as well other software.coop workers can cover for me). Now the next step is for some phone engineer to call. For this to happen, I have to agree that I will pay GBP 164 if a fault is found on my phone network instead of theirs. If I don t, they will give up trying to find the fault. I don t really see why this is necessary: surely if there was a line fault, it would have shown itself on the old service? Also, the router says the line has no fault and the SNR and Attenuation are both fine but not great. Unlike most people, I ve already had my house phone network tested with professional equipment (yay for network engineers in the co-op trying to avoid the painful state of British broadband bugfixing), but I m still left betting hundreds of pounds on the phone engineers not pointing the finger at my network. What choice do I have? I ve read elsewhere about people giving up on ADSL and switching to mobile internet because then they have only one company to deal with. I can see some attraction in that, but I live in a village with poor mobile service and I suspect the forthcoming T-Mobile/Orange and T-Mobile/Three link-ups will soon make that less straightforward too. A confusopoly is probably more profitable to telephone companies. Do most people bet a couple of hundred pounds to get their broadband fixed? How many broadband faults get ignored at that point? Why don t the retail company Cs have access to enough Company B systems to lead the fault-finding instead of only being able to pass it on to people who seem to give replies that contradict the logs and won t talk directly to customers? Is fixing this A-B-C three-companies-pass-the-buck system a necessity for Digital Britain? Why isn t the government intervening to end the OpenReach monopoly outside the big cities?

Earlier this week, Tom Watson, Cabinet Office Parliamentary Secretary, published the Open Source, Open Standards and Re Use: Government Action Plan. It s had a pretty mixed reaction, with mild scepticism (niq s soapbox: Is gov.uk going open-source?) being the average reaction from what I ve read. I particularly liked the kind offer by Bristol Wireless to debianise Tom Watson s laptop. I think commentators are bang-on that the procurement process needs to change and that this sounds positive. I ll love it if I m wrong, but this looks like the lip service as usual which I ve seen in the last 10 years working on FOSS in the UK. I want to see the action that comes from this plan! When government actually starts buying FOSS from typical FOSS service providers and not just the IBMs of this world, then I ll believe it. My suspicion is re-ignited by some of the activites around this action plan. For example, does anyone know why the Cabinet Office didn t select FOSS to run their special public FOSS Aggregation page? (Actually, what s the best FOSS tag aggregator web service out there? I know we can set them up in Wordpress widgets, but what hosted services are there?) I m also a bit bemused that their page requires users to accept cookies until they expire, yet its privacy policy explicitly says You may suppress cookies after your visit or configure your Internet browser to prevent them. Yes, I can prevent them, but then it does nothing useful! (Based on a comment I made at the OSS Watch team blog and discussions with a few user groups.)