Search Results: "evo"

30 September 2012

Russell Coker: Links September 2012

Scott Adams has an interesting idea for political debates he calls Fact Bubbler [1]. It sounds implausible the first time you read it, but then so did Wikipedia. Arstechnica has an interesting article on software defined radio [2]. This could change many things. The Nieder Family has another update on the way patents threaten their daughter s ability to communicate [3]. Also Apple is making things worse by taking pre-emptive strikes against iPhone apps which are involved in legal disputes never use an iPhone (or other proprietary system) for anything important if you have a choice. In more positive news crowd-funded gene sequencing has found an explanation for Maya Nieder s developmental delays [4]. This will revolutionise medicine! Steven Cherry of IEEE Spectrum has an interesting interview with Peter Cappelli about the difficulties that computer and engineering companies have in hiring talented people [5]. The Guardian has an interesting article about the non-profit investigative reporting organisation ProPublica [6]. See the ProPublica.org site for some quality news reports [7]. Michael O.Church wrote an interesting and insightful article about the fate of a Just a Programmer in a startup funded by Venture Capitalists [8]. It doesn t sound good at all. EyeNetra is developing a smart-phone based system for testing eyes [9]. It s apparently possible to manufacture glasses for $0.75 so the cost and difficulty of performing eye tests is the main factor that prevents poor people in developing countries from getting glasses. So a cheap portable eye testing system is going to help many people get the glasses they need. Bryan Gardiner wrote an interesting article for Wired about the Gorilla glass that is used in most mobile phones and tablets [10]. Maco wrote an interesting article about crochet and reverse-engineering along with a Python program to print a crochet pattern [11]. I wonder whether anyone has tried to make a crochet robot, something like a 3D printer but which crochet s things rather than printing them. Holding the wool would be a real challenge, it s not nearly as easy as printing on something that s stuck down and incapable of movement.
  1. [1] http://dilbert.com/blog/entry/fact_bubbler/
  2. [2] http://arstechnica.com/tech-policy/2012/07/how-software-defined-radio-could-revolutionize-wireless/
  3. [3] http://niederfamily.blogspot.com.au/2012/06/silencing-of-maya.html
  4. [4] http://blog.ted.com/2012/07/17/newly-discovered-gene-may-explain-4-year-olds-rare-disease-thanks-to-ted-fellow-jimmy-lin/
  5. [5] http://spectrum.ieee.org/podcast/at-work/tech-careers/why-bad-jobsor-no-jobshappen-to-good-workers
  6. [6] http://www.guardian.co.uk/technology/2012/jul/02/propublica-investigative-reporting
  7. [7] http://www.propublica.org/
  8. [8] http://michaelochurch.wordpress.com/2012/07/08/dont-waste-your-time-in-crappy-startup-jobs/
  9. [9] http://eyenetra.com/
  10. [10] http://www.wired.com/wiredscience/2012/09/ff-corning-gorilla-glass/all/
  11. [11] http://ubuntulinuxtipstricks.blogspot.com.au/2010/10/algorithms-reverse-engineering-and.html
Related posts:
  1. Links April 2012 Karen Tse gave an interesting TED talk about how to...
  2. Links May 2012 Vijay Kumar gave an interesting TED talk about autonomous UAVs...
  3. Links July 2012 The New York Times has an interesting article about hacker...

20 September 2012

Bartosz Feński: Certificate of volunteering work for Debian

My current employer has some internal program that it allows employee to devote two days a year for volunteering work.
It doesn t matter what it is (work for hospice, planting trees, helping in some foundation) so I thought about devoting this time for the Debian project. Either fixing RC bugs or finally give my packages enough attention and take care of their updates. Anyway the employer needs the certificate that I took participation in such event.
So the question is if it is possible to get some official document/certificate that I devoted some time to work on Debian? Bureaucracy has to be fulfilled unfortunately.

18 September 2012

Francois Marier: Advice to newcomers and veterans of the Free and Open Source software communities

A few months ago, a collection of essays called Open Advice was released. It consists of the things that seasoned Free and Open Source developers wish they had known when they got started. The LWN book review is the best way to get a good feel for what it's about, but here are the notes I took while reading it:

15 September 2012

Eddy Petrișor: Why a lack of skepticism is dangerous...

Some of my Romanian readers might know that for the last two years I've got involved in the skeptical movement to such a degree that I am a co-producer of a bi-weekly podcast on science and skepticism (in Romanian) called Skeptics in Romania . Some might even be regular listeners of the show.

(There isn't much to see now visually on the site, but me and the other people behind the project have some ongoing plans to change that.)

In spite of our modest site, up until now we had some successes, one of them being the publication of an article on us in a known Romanian printed publication and another being the invitation to a live show face to face with Oreste Teodorescu, a well known Romanian mysticist and woo promoter.

During that live show we managed to show a demonstration (video below, in Romanian) of how astrology gives the impression of working, without actually working, and, taking into account we had no prior TV camera experience and that it was a live show, I think we managed an honourable presence.


<iframe allowfullscreen="allowfullscreen" frameborder="0" height="315" src="http://www.youtube.com/embed/y5OG1q8_3Ro" width="420"></iframe>

We also have a series of interviews in English with some really interesting people: Dr. Eugenie Scott, Prof. Christopher French, Prof. Edzard Ernst, Samantha Stein and others. We did these interviews at Denkfest 2011, in Zurich, and we integrated the translated (voice over) interviews in our podcast. The conlusion is that most of our activities revolve around the podcast, so let me tell you more about that.

The podcast has a somewhat fixed structure, it starts with a conversation between ourselves, then we have a segment on the history of science, technology, skepticism and woo, and then we have a segment called The dangers of not being skeptical . In this segment we present cases of people who lost their lives, their health, their money or any combination of the former because they were duped into some scam, science-y sounding non-science, unfounded claim or some other woo.

Having lost recently my brother-in-law to a form of cancer known as Hodgkin lymphoma, I became especially sensitive about miracle-cure claims for cancer, and this section of the show has lately seen its fair share of such cases. Honestly, if there could be a way to prosecute the irresponsible, ignorant and/or cynical people promoting all sorts of quack "therapies", especially for cancer*, I would really like to see it happen. But there isn't, and we're trying the best accessible approach, informing the public.

During my brother-in-law's last two years of his life, he went through lots of chemotherapy and radiotherapy sessions, repeated periods of hospitalisation, and lots of drugs. This is the best of what we currently have for treating and curing most forms of cancer, and too many times this isn't enough. I can't even imagine how stressful and discouraging it must feel when the best of what we have doesn't help.

Here is where the desperation and hopes of patients and their families meet the purely irresponsible cynical or ignorant promoters of woo and quack therapies. Because it takes either an ignorant or a really cynical (I really feel this word isn't enough) person to prey on the suffering of other people to make easy money under the false pretence of offering a cure.

It almost happened to my brother-in-law and his family, because they almost went for some herbal concoction promoted as a cancer cure on some forum, blog or page of a seller of this fake therapy. It was really hard for me to make them understand why using such a product it not advisable, not even in parallel with the medical treatment due its possible counter effect or interactions with the real medical treatment, without them getting the wrong idea that I wasn't trying to help. While trying to be brief and informative not to lose their attention, I told them how "natural" doesn't necessarily mean "good" (uranium, lead and Irukandji's venom are all natural), and how plants are drugs because they all contain chemical substances (and no, "chemical" does not mean "human made" or "artificial") which could interact with the medical treatment.


But most people don't even have the chance of having close by a person with a more science-leaning thought process and a skeptical mind. Those unfortunate people are the most vulnerable people and constitute the biggest chunk of the victims of baseless pseudo-cures or pseudo-treatments.

On our last show, I presented the case of Yvonne Main, a cancer suffering patient who mistook an invasive carcinoma for a cyst, and irdologist Ruth Nelson for a real healthcare giver.

Yvonne Main, died from an invasive carcinoma
after seeking help from a iridologist,
and delaying real medical treament for 18 months


Yvonne, after seeking medical advice from a person that essentially promotes the dead idea of guessing diseases by looking at the eyes**, used natural treatments for about 18 months and, after all this time, her carcinoma grew to a size of 10 to 11 cm, eating through her skull and causing damage which was later attempted to be countered through bone transplant from her ribs.

Ruth Nelson wasn't prosecuted in any way and continues her practice of quackery unharmed.

This is not the only case, nor even one case from a select few where woo and quackery lead to grave consequences for patients. There are many, many more; they're so many that even after splitting them in categories they seem too many per category, especially when you realise these are only the findings of, essentially, a single man:


http://www.whatstheharm.net/


This is part of what I have been doing in the last few years, instead of working on Debian. Is it a good thing? Is it a bad thing? Maybe it's good. I want to know what do you think?


* you will, most likely, never hear such a promoter of non-therapies say that there isn't just one cancer, and that, in fact, cancer is a name for a certain family of diseases which are all called cancer - that's a first sign that you might be dealing with quack
** probably in the line of thought that the eyes are the gates to the soul so they must tell something significant about health

13 September 2012

Gunnar Wolf: Great online course available: Securing Digital Democracy , by J. Alex Halderman

I was pointed at a great online course If you are into e-voting analysis (or, more broadly, into democratic processes' history, evolution and future), I strongly suggest you to take a look at Securing Digital Democracy . Just the name of the teacher should be enough to make it interesting: University of Michigan professor J. Alex Halderman, the guy who has analized/hacked several electronic booths, and one of the clearest, smartest voices to explain what should we require of a voting system and how electronic booths are the worst fit for any purpose. The course is delivered through Coursera; I have found Coursera to be an effective, usable, unobtrusive platform So much I even signed up for another course. I am not so happy with online courses requiring to wait so much between lessons, but after all, it tries to mimic what we see at "regular" (i.e. classroom) teaching settings. And, after all, we autodidacts are still a minority. The course in question started ten days ago, but you can still perfectly join. Each week has two lessons, worth of approximately 40 minutes of video each, and are "graded" through a quiz. Lets see how this evolves.

10 September 2012

Vincent Sanders: I thought San Diego must be Heaven on earth...It seemed to me the best spot for building a city I ever saw.

I think perhaps Alonzo Horton overstated the case but San Diego was a perfectly pleasant place to spend a week. Recently I was fortunate and attended the Linux Plumbers conference (and bits of Kernel Summit) and as on previous occasions there were some interesting sessions and some less helpful diversions.

Melodie went along with me, our first trip away together without the kids in a long time. We did the tourist thing for a few days including the USS Midway aircraft carrier tour (recommended) and the harbour boat tour (also recommended).

The conference was co-located with several other events and there seemed to be a lot of people around not connected with KS or LPC. For me the hallway track was, as usual, much more rewarding and I got to catch up with several old friends and make some new acquaintances. A lot of my colleagues were about attending the Gstreamer and Linuxcon conferences so there was the obligatory work evening event (which was pizza and beer, just done brilliantly).

A track that did stand out for me was the Clang / LLVM presentations. They gave an excellent overview of their progress towards making the Kernel compile with their tools, such innovation appears to be making both projects stronger.

One thing that occurred to me was the blandness of KS and LPC this time, usually there are at least some loud disagreements but I failed to attend even one session where there was more than small differences which were quickly resolved.

Perhaps such conferences, like the majority of people attending them, are becoming middle aged and a little complacent. An observation like that does make me wonder where the next change will come from and what it will look like. Just as open source software (including Linux) has disrupted the proprietary software industry in the last decade what will come along and disrupt us? Or is open source the end of the line and we will just continue to evolve?



4 September 2012

Gunnar Wolf: Electronic voting in Panama: Slower, more expensive, more uncertainty... Goodbye!

Panama just underwent a nasty e-voting exercise: Electronic-mediated elections were held for the committee of the PRD party. It sounds simple - Even trivial! There were only 4100 authorized voters, it was geographically trivial (all set inside a stadium)... But it blew up in smoke. I won't reiterate all what happened, I'll rather direct you to our project's (the e-voting observatorium) page: News regarding Panama (for those coming from the future, search starting at 2012-08-27 and yes, it's all in Spanish, but there are free-as-in-beer translation services. Many e-vote proponents/sellers/pushers were very eagerly waiting for this election to brag about one more success... So much that they could not just ignore it, and started rationalizing it away. Anyway, while feeding the observatorium, I came across this opinion-article in the Voto Digital website, which makes quite a bit of pro-e-voting noise. I replied to it, and I think my analysis is worth sharing also with you:
So, lets make some simple numbers, rounding the numbers: The PRD vote in Panama was done for a universe of 4100 voters. It took 10 hours (instead of the planned 4), so 410 people were processed every hour. There were 40 voting (electronic) booths, so each processed 10 people per hour. This means, each person spent 6 minutes by the booth. A manual vote in this fashion is highly parallelizable: Each of the voters can be given ballots with anticipation, or many of them cna be allowed in to be given the ballots in situ (depending on the electoral scheme employed). The contention time is the time it takes to each voter to get near the booth and deposit his ballot (either folded or in an envelope) - And it will very rarely be more than a couple of seconds. So, given that using electronic booths parallelism cannot grow (there is a fixed number of machines) and the queues grew wildly, with traditional voting it would have surely fitted in the expected four hours (they were expected, also, based on their past experiences). As for counting, that's the slowest part of manual voting, it's also highly parallelizable: If each of the 40 booths has slightly over 100 ballots, the party personnel can easily count them in under 30 minutes. Capture and aggregation for the 40 partial results would take an extra 10 minutes, even being generous. Manual voting would have saved them around five hours, without demanding additional resources (and being thus much more economical than having to buy 40 specific-purpose computers). And as an additional advantage, the physical and tangible vote proofs would remain, in case they were ever again needed.

Pietro Abate: Package Managers Comparison - take 2

On year ago, we (the mancoosi team) published a comparison study regarding the state of the art of dependency solving in debian. As few noticed, the data presented had few glitches that I promised to fix. So we've repeated our tests using exactly the same data we used one year ago, but now using the latest available versions of all package managers as available in debian unstable. During the last year, three out of four solver that we evaluated release a major upgrade so I expected many improvements in performances and accuracy. Mpm, our test-bench for new technologies, changed quite a bit under the wood as a consequence of the evolution of apt-cudf and the recent work done in apt-get to integrate external cudf dependency solvers. Overall the results of our study are not changed. All solvers but mpm, that is based on aspcud, are not scalable as the number of packages (and alternatives) grows. It seems that Smart is the solver that does not give up, incurring in a timeout (fixed at 60 seconds) most of the time. Aptitude is the solver that tried to give you a solution, doesn't matter what and as result providing solutions that do not satisfy the user request for one reason or the other. Apt-get does surprisingly well, but it gives up pretty often showing the incomplete nature of it's internal solver. Cupt sometimes timeout, sometimes gives up, but when it is able to provide an answer it is usually optimal and it is very fast ... Mpm consistently finds an optimal solution, but sometimes it takes really a long time to do it. Since mpm is written in python and not optimized for speed this is not a big problem for us. The technology used by mpm is now integrated in apt-get and I hope this will alleviate this problem. All the details of our study can be found one the Mancoosi Website as usual with a lot of details. For example here you can find the results when mixing four major releases : sarge-etch-lenny-squeeze. Comments are more then welcome.

2 September 2012

Benjamin Mako Hill: Open Brands

In late July, the Awesome Foundations invited me to participate in an interesting conversation about open brands at their conference. Awesome is a young collection of organizations struggling with the idea of if, and how, they want to try to control who gets call themselves Awesome. I was asked to talk about how the free software community approaches the issue. Guidance from free software is surprisingly unclear. I have watched and participated in struggles over issues of branding in every successful free software project I've worked in. Many years ago, Greg Pomerantz and I wrote a draft trademark policy for the Debian distribution over a couple beers. Over the last year, I've been working with Debian Project Leader Stefano Zacchiroli and lawyers at the Software Freedom Law Center to help draft a trademark policy for the Debian project. Through that process, I've come up with three principles which I think lead to more clear discussion about whether a free culture or free software should register a trademark and, if they do, how they should think about licensing it. I've listed those principles below in order of importance. 1. We want people to use our brands. Conversation about trademarks seem to turn into an exercise in imagining all the horrible ways in which a brand might be misused. This is silly and wrong. It is worth being extremely clear on this point: Our problem is not that people will misuse our brands. Our problem is that not enough people will use them at all. The most important goal of a trademark policy should be to make legitimate use possible and easy. We want people to make t-shirts with our logos. We want people to write books about our products. We want people to create user groups and hold conferences. We want people to use, talk about, and promote our projects both commercially and non-commercially. Trademarks will limit the diffusion of our brand and, in that way, will hurt our projects. Sometimes, after carefully considering these drawbacks, we think the trade-off is worth making. And sometimes it is. However, projects are generally overly risk averse and, as a result, almost always err on the side of too much control. I am confident that free software and free culture projects' desire to control their brands has done more damage than all brand misuse put together. 2. We want our projects to be able to evolve. The creation of a trademark puts legal power to control a brand in the hands of an individual, firm, or a non-profit. Although it might not seem like such a big deal, this power is, fundamentally, the ability to determine what a project is and is not. By doing this, it creates a single point of failure and a new position of authority and, in that process, limits projects' ability to shift and grow organically over time. I've heard that in US politics, there is no trademark for the terms Republican or Democrat and that you do not need permission to create an organization that claims to be part of either party. And that does not mean that everybody is confused. Through social and organizational structures, it is clear who is in, who is out, and who is on the fringes. More importantly, this structure allows for new branches and groups outside of the orthodoxy to grow and develop on the margins. Both parties have been around since the nineteenth century, have swapped places on the political spectrum on a large number of issues, and have played host to major internal ideological disagreements. Almost any organization should aspire to such longevity, internal debate, and flexibility. 3. We should not confuse our communities. Although they are often abused, trademarks are fundamentally pro-consumer. The point of legally protected brands is to help consumers from being confused as the source of a product or service. Users might love software from the Debian project, or might hate it, but it's nice for them to be able to know that they're getting "Debian Quality" when they download a distribution. Of course, legally protected trademarks aren't the only way to ensure this. Domains names, internal policies, and laws against fraud and misrepresentation all serve this purpose as well. The Open Source Initiative applied for a trademark on the term open source and had their application rejected. The lack of a registered trademark has not kept folks from policing use of the term. Folks try to call their stuff "open source" when it is not and are kept in line by a community of folks who know better. And since lawyers are rarely involved, it is hardly clear that a registered trademark would help in the vast majority of these these situations. It is also the case that most free software/culture organizations lack the money, lawyers, or time, to enforce trademarks in any case. Keeping your communities of users and developers clear on what is, and what isn't, your product and your project is deeply important. But how we choose to do this is something we should never take for granted.

27 August 2012

Robert Collins: Why platform specific package systems exist and won t go away

A while back mdz blogged about challenges facing Ubuntu and other Linux distributions. He raises the point that runtime libraries for Python / Ruby etc have a unique set of issues because they tend to have their own packaging systems. Merely a month later he attended Debconf 2010 where a presentation was given on the issues that Java packages have on Dpkg based systems. Since then the conversation seems to have dried up. I ve been reminded of it recently in discussions within Canonical looking at how we deploy web services. Matt suggested some ways forward, including: I think its time we revisit and expand on those points. Nothing much has changed in how Ubuntu or other distributions approach integration with other packaging systems but the world has kept evolving. Internet access is growing ever more ubiquitous, more platforms are building packaging systems clojure, scala, node.js, to name but three, and a substantial and ever growing number of products expect to operate in a hybrid fashion with an evolving web service plus a local client which is kept up to date via package updates. Twitter, Facebook and Google Plus are three such products. Android has demonstrated a large scale app store on top of Linux, with its own custom packaging format. In order to expand them, we need some background context on the use cases that these different packaging systems need to support. Platforms such as antivirus scanners, node.js, Python, Clojure and so forth care a great deal about getting their software out to their users. They care about making it extremely easy to get the latest and greatest versions of their libraries. I say this because the evidence is all around us: every successful development community / product has built a targeted package management system which layers on top of Windows, and Mac OSX, and *nux. The only rational explanation I can come up for this behaviour is that the lower level operating system package management tools don t deliver what they need. E.g. this isn t as shallow as wanting a packaging system written in their own language, which would be easy to write off as parochialism rather than a thoughtful solution to their problems. In general packaging systems provide a language for shipping source or binary form, from one or more repositories, to users machines. They may support replications, and they may support multiple operating systems. They generally end up as graph traversal engines, pulling in dependencies of various sorts you can see the DOAP specification for an attempt at generic modelling of this. One problem that turns up rapidly when dealing with Linux distribution package managers is that the versions upstream packages have, and the versions a package has in e.g. Debian, differ. They differ because at some stage, someone will need to do a new package for the distribution when no upstream change has been made. This might be to apply a local patch, or it might be to correct a defect caused by a broken build server. Whatever the cause, there is a many to one relationship between the package versions that end users see via dpkg / rpm etc, and those that upstream ship. It is a near certainty that once this happens to a library package, that comparing package versions across different distribution packages becomes hard. You cannot reliably infer whether a given package version is sufficient as a dependency or not, when comparing binary packages between Red Hat and Debian. Or Debian and Ubuntu. The result of this is that even when the software (e.g. rpm) is available on multiple distributions (say Ubuntu and RHEL), or even on multiple operating systems (say Ubuntu and Windows), that many packages will /have/ to be targeted specifically to build and execute properly. (Obviously, compilation has to proceed separately for different architectures, its more the depedency metadata that says and build with version X of dependency Y that has to be customised). The result of this is that there is to the best of my knowledge no distribution of binary packages that targets Debian/Ubuntu and RHEL and Suse and Windows and Mac OS X, although there are vibrant communities building distributions of and for each in isolation. Some of the ports systems come close, but they are still focused on delivering to a small number of platforms. There s nothing that gives 99% coverage of users. And that means that to reach all their users, they have to write or adopt a new system. For any platform X, there is a strong pressure to have the platform be maintainable by folk that primarily work with X itself, or with the language that X is written in. Consider Python there is strong pressure to use C, or Python, and nothing else, for any tools that is somewhat parochial, but also just good engineering reducing variables and making the system more likely to be well maintained. The closest system I know of Steam is just now porting to Ubuntu (and perhaps Linux in general), and has reached its massive popularity by focusing entirely on applications for Windows, with Mac OSX a recent addition. Systems like pypi which have multi platform eggs do target the wide range of platforms I listed above, but they do so both narrowly and haphazardly: whether a binary or source package is available for a given platform is up to the maintainer of the package, and the packages themselves are dealing with a very narrow subset of the platforms complexity: Python provides compilation logic, they don t create generic C libraries with stable ABI s for use by other programs, they don t have turing complete scripts for dealing with configuration file management and so forth. Anti virus updaters similarly narrow the problem they deal with, and add constraints on latency- updates of anti virus signatures are time sensitive when a new rapidly spreading threat is detected. A minor point, but it adds to the friction of considering a single packaging tool for all needs is the different use cases of low level package management tools like dpkg or rpm vs the use cases that e.g. pypi has. A primary use case for packages on pypi is for them to be used by people that are not machine administrators. They don t have root, and don t want it. Contrast that with dpkg or rpm where the primary use case (to date) is the installation of system wide libraries and tools. Things like man page installation don t make any sense for non-system-wide package systems, whereas they are a primary feature for e.g. dpkg. In short, the per-platform/language tools are (generally):
  1. Written in languages that are familiar to the consumers of the tools.
  2. Targeted at use on top of existing platforms, by non-privileged users, and where temporary breakage is fine.
  3. Intended to get the software packaged in them onto widely disparate operating systems.
  4. Very narrow they make huge assumptions about how things can fit together, which their specific language/toolchain permits, and don t generalise beyond that.
  5. Don t provide for security updates in any specific form: that is left up to folk that ship individual things within the manager.
operating system package managers:
  1. Are written in languages which are very easy to bootstrap onto an architecture, and to deploy onto bare metal (as part of installation).
  2. Designed for delivering system components, and to avoid be able to upgrade the toolchain itself safely.
  3. Originally built to install onto one operating system, ports to other operating systems are usually fragile and only adopted in niche.
  4. Are hugely broad they install data, scripts, binaries, and need to know about late binding, system caches etc for every binary and runtime format the operating system supports
  5. Make special provision to allow security updates to be installed in a low latency fashion, without requiring anything consuming the package that is updated to change [but usually force-uninstalling anything that is super-tightly coupled to a library version].
Anti virus package managers:
  1. Exist to update daemons that run with system wide escalated privileges, or even file system layer drivers.
  2. Update datasets in realtime.
  3. Without permitting updates that are produced by third parties.
Given that, lets look at the routes Matt suggested Decoupling applications from the core as a strategy makes an assumption that the core and applications are partitionable. If they are not, then applications and the core will share common elements that need to be updated together. Consider, for instance, a Python application. If you run with a system installed Python, and it is built without zlib for some reason, but the Python application requires zlib, you have a problem. A classic example of this problem is facing Ubuntu today, with all the system provided tools moving to Python 3, but vast swathes of Python applications still being unported to Python 3 at all. Currently, the Python packaging system virtualenv/buildout + distribute don t provide a way to install the Python runtime itself, but will happily install their own components for everything up the stack from the runtime. Ubuntu makes extensive use of Python for its own tools, so the system Python has a lot of packages installed which buildout etc cannot ignore this often leads to issues with e.g. buildout, when the bootstrap environment has (say) zope.interfaces, but its then not accessible from the built-out environment that disables the standard sys.path (to achieve more robust separation). If we want to pursue decoupling, whether we build a new package manager or use e.g. virtualenv (or gem or npm or ), we ll need to be aware of this issue and perhaps offer, for an extended time, a dedicated no-frills, no-distro-packages install, to avoid it, and to allow an extended supported period for application authors without committing to a massive, distro sponsored porting effort. While its tempting to say we should install pip/npm/lein/maven and other external package systems, this is actually risky: they often evolve sufficiently fast that Ubuntu will be delivering an old, incompatible version of the tool to users well before Ubuntu goes out of support, or even befor the next release of Ubuntu. Treating data as a service. All the cases I ve seen so far of applications grabbing datasets from the web have depended on web infrastructure for validating the dataset. E.g. SSL certificates, or SSL + content checksums. Basically, small self-rolled distribution systems. I m likely ignorant of details here, and I depend on you, dear reader, to edumacate me. There is potential value in having data repackaged, when our packaging system has behind-firewall support, and the adhoc system that (for instance) a virus scanner system has does not. In this case, I specifically mean the problem of updated a machine which has no internet access, not even via a proxy. The challenge I see it is again the cross platform issue: The vendor will be supporting Ubuntu + Debian + RHEL + Suse, and from their perspective its probably cheaper to roll their own solution than to directly support dpkg + rpm + whatever Apple offer + Windows the skills to roll an adhoc distribution tool are more common than the skills to integrate closely with dpkg or rpm What about creating a set of interfaces for talking to dpkg / rpm / the system packagers on Windows and Mac OSX ? Here I think there is some promise, but it needs as Matt said careful thought. PackageKit isn t sufficient, at least today. There are, I think, two specific cases to cater to:
  1. The anti-virus / fresh data set case.
  2. The egg/gem/npm/ specific case.
For the egg/gem/npm case, we would need to support a pretty large set of common functionality, on Windows/Mac OSX / *nux (because otherwise upstream won t adopt what we create: losing 90% of their users (windows) or 5% (mac) isn t going to be well accepted :) . We d need to support multiple installations (because of mutually incompatible dependencies between applications), and we d need to support multiple language bindings in some fashion some approachable fashion where the upstream will feel capable of fixing and tweaking what we offer. We re going to need to support offline updates, replication, local builds, local repositories, and various signing strategies to match the various tradeoffs made by the upstream tools. For the anti-virus / fresh data case, we d need to support a similar set of operating systems, though I strongly suspect that there would be more tolerance for limited support in that most things in that space either have very platform specific code, or they are just a large-scale form of the egg/gem/npm problem, which also wants easy updates. What next? We should validate this discussion with at least two or three upstreams. Find out whats missing I suspect a lot and whats wrong I hope not much :) . Then we ll be in a position to decide if there is a tractable, widespread solution *possible*. Separately, we should stop fighting with upstreams that have their own packaging systems. They are satisfying different use cases than our core distro packaging systems are designed to solve. We should stop mindlessly repackaging things from e.g. eggs to debs, unless we need that specific thing as part of the transitive runtime or buildtime dependencies for the distribution itself. In particular, if us folk that build system packaging tools adopt and use the upstream application packaging tools, we can learn in a deep way the (real) advantages they have, and become more able to reason about how to unify the various engineering efforts going into them and perhaps even eventually satisfy them using dpkg/rpm on our machines.

22 August 2012

Gunnar Wolf: An industry commits suicide and blames us

[ once again, I am translating somebody else's material In this case, my good Costa Rican friend Carolina Flores. Please excuse my stylistic mistakes My English is far from native as you well know. But this material is worth sharing, and worth investing some tens of minutes doing a quick translation. If you can read Spanish, go read Caro's original entry ] Have you been to a music record store lately? I did so last Saturday, as a mere excercise. I was not planning on buying anything but I wanted to monitor things and confirm my suspicions. What was I suspicious of? First, that I would only find old records. And so it was: The only recent record I found was ...little broken hearts by Norah Jones. the second, that I would only find music for over 50 year old people. so it was: Were I there to look for a present for my father, I would have walked out with 10 good records. Third, that in the store nothing worth commenting would happen. About that last point, I should point out it was around 10 AM and the store had just opened its doors. Lets concede the benefit of doubt. I don't think many of you will remember, but in Barrio La California (where there is now a beauty parlour, almost in front of AM.PM) there was Auco Disco. In Auco Disco there was a guy specialized in rock (Mauricio Alice) and another one specialized in jazz (I don't remember his name). In that record you could always find rare records, but if they were not there, at least you were sure to find somebody to say: "No, we don't have that, but that's an excellent record, it's the best that [insert group here] have ever recorded because just afterwards they switched their guitar player, they had gone a bit south but with that record they are flying. But no, we don't have it; I can recommend you this record by [insert another group] because it has a guitar solo in track six that is amazing". It would happen more or less like that, which means, one would arrive to Auco Disco at 10 AM and leave around 5 PM with three new records, after having listened to a spectacular music selection. What happened to those stores? Were they killed by The Pirate Bay? That's the simplistic answer from the recording industry! The answer is that those stores never got anything from the industry but an invoice. The industry specially in prescindible markets such as ours was limited to hiring artists, taking care of them recording a sellable product, producing the object called record, and that's it. The more commercial radio stations were paid to program those songs as it cannot be casual that "Mosa, mosa" is the summer hit in all of Latin America, can it? but, record stores? Nothing. Lets carry on with that idea: Radio stations are paid to program said music. This idea should not lead us to believe that recording companies are to blame for bad taste. I won't reveal my sources, but I know the success of the "Locura autom tica" song by La Secta was a real example. Nobody paid for it. That song got to the number one because of its own merits(?) (you don't know the effort it took to find that thing, I cannot recommend it to you). Same thing happens with other stations that don't program reggaet n, that try to save the species, and where they play what we do like. But the thing is, everything we like is not available in any record store in this country. Then, even if we wanted to buy a record or give it as a gift to somebody, it is plain impossible. And don't tell me it's the same to present as a gift a link or a CD full of downloaded MP3 as it is to give a record with cover and booklet, wrapped in gift paper. I might be old-school, but the fetish object record still exists, not only because of its cover, but because of its sound. A 3MB MP3 is akin to drinking coffee dripping from a bag that has been used eight times with the same coffee beans. That format is the worst that has ever happened to music, and if we had any bit dignity we would never purchase digital files in Amazon or iTunes safe for MP3 with an acceptable compression level. That, if we could buy them, because not only that is allowed to us. As the musical industry has no interest in resolving ITS problem (that is not our problem, it is those companies') it has not even been resolved how to charge for a MP3 download that includes import fees (well, if downloading from here a MP3 from a USA-based file server can be considered importing goods into Costa Rica!!!) so we don't have to get dizzy entering into the nineties to Titi Online to discover there is nothing by Muse, Andrew Bird, The Killers, Death Cab for Cutie, Paramore, Bj rk... (believe me, I looked them all up, even Norah jones and La Secta. They also were not there). This all leads me to the question, which I present with all due respect (NOT): What the fuck do they expect us to do??? It is outrageous; above all because in the best case they will sell us a watery coffee download that won't allow us to get all the details a vinyl or less compression would give us. In the worst case, post-MP3 groups will end up recording music with no harmonics or hidden sounds, because, what for? Nobody will hear it. They even admit it: "Some musicians and audio engineers say the MP3 format is changing the way studios mix their recordings. They say the MP3 format "flattens" dynamics differences in tone and volume in a song. As a result, a great deal of new music sounds very much alike, and there is nothing as focusing to create a dynamic listening experience. Why working so hard in creating complex sound if nobody can detect it?" (Rolling Stone, The Death of High fidelity, December 26 2007, taken from here). That's why I am not surprised by Adri n's post regarding the sales of old records. The price has nothing to do with it. The causes are related to the fetish object record and what it means or does not mean for people that have never purchased one. Adri n also asks if somebody here keeps buying records. I answered that I would if the stores sold anything I like. I do it even after the nausea I feel while reading "This phonogram is an intellectual work protected in favor of its producer COPYING IT IN WHOLE OR PARTIALLY IS FORBIDDEN " (like that, uppercased, yelling to whoever is only guilty of having bought a record and defending the producer, not the artist). But I am sure that almost nobody buys records because doing so is no longer a gratifying experience; because if buying a record is clicking to wait 15 days for it to reach the mailbox, we prefer clicking on the download link. But there is another reason for people not to buy records anymore. In one of my talks regarding the dictatorship of the all rights reserved, I asked the 30 twenty-something-year-old students if any of them had ever bought a record. One answered he had, because he is an author and performer (cantautor in Spanish) and understands the effort that producing a record entails. The rest of them had never done so. Is it possible that said young people have never listened to real music? Is it possible that, were it not for concerts, what they consider music is a set of washed-out MP3 that are about to fill up 1TB of their computer? Does people no longer buy records because they cannot differentiate one sound from the other? It is not very clear for me where do I want to get to. The recording industry is despicable. An industry that instead of innovating sets its energy on suing adolescents for downloading songs, trying to pass laws restricting our freedoms in Internet, putting up DRMs making us hostages to our devices* and forcing us to listen just the aroma of music, deserves my whole contempt. If we add ot this that said industry won't allow us to legally download their breadcrumbs because it has not understood that Internet does not need a van crossing borders, besides my contempt they deserve my pity and my heartfelt condolence. But the condolence is also for music, real music, that which is not compressed under the shoe using a terrible format. It is also for independent musicians that have not realized that begging for a bit of space to that industry they just add to themselves the "despicable" tag, given they deserve the fruits of their work to enter their bank account. However, there are good things that have come out of this absurdity. Be it for those that have joined projects such as Aut mata (even if it is in MP3) and for dreams come true such as Musopen (that have achieved that the music that's in theory Public Domain becomes so in practice as well). Good for the Electronic Frontier Foundation and the list of lawyers willing to defend people accused of ilegally downloading music in the USA. Good for the Creative Commons licenses that allow free sharing. All those are growing solutions, although none of them allows me to buy the Panamanian Carlos M ndez's record. Thankfully, a friend of mine who knows I will never give a dime to Apple, bought the files for me in iTunes. I thank him deeply, although I would have prefered to go to Auco Disco and have Mauricio tell me that the 2007 EP I have from Carlos is better than the record he did on 2009. * My devices don't have DRM because I use free software. I also use the ogg file format. Image by verbeeldingskr8

20 August 2012

Lars Wirzenius: Baserock, or what I've been doing at work last year

I was hired last year ago by Codethink to work on Baserock, a new method and toolset for developing embedded Linux systems. While I don't have a lot of experience developing embedded Linux systems specifically, I have a fair bit of experience devoloping Linux distributions, from Debian and Ubuntu, and that's quite similar. I had already written up some ideas for doing test driven distro development, and system level branching and merging. Then I got hired to do that stuff for real. Baserock is still a work in progress, but we've just made a public development release of the free software version, and I can now talk about it. This blog post is my own; I am not talking for my employer. I won't write about all the aspects and features of Baserock, since that would make this blog post really long. You can read that stuff in the wiki, anyway, and ask questions on the Baserock mailing list and IRC channel if you're curious. I just thought I'd mention what I've been doing this past year at work. However, there's one thing I want to especially mention: I'm excited. Getting to re-invent workflows and tools for system development is fun. I can get up in the morning, decide to try a new version of gcc, make the one line change, and run the one shell command to rebuild everything, and play with the new system pretty soon after. And if it doesn't work, no worries, I'll just not merge that branch into the mainstream. And then I still have time for other things that workday. Baserock is getting rid of the unnecessary friction in system development, all the stupid little things you have to do just to keep working. That is going to make a lot of developers lives better in this corner of the software development universe.

Steve Kemp: Failing to debug a crash with epiphany-browser and webkit

Today I'm in bed, because I have le sniffles, and a painful headache. I'm using epiphany to write this post, via VNC to my main desktop, but I'm hating it as I've somehow evolved into a state where the following crashes my browser: Running under GDB shows nothing terribly helpful: 
Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007ffff51a0a46 in ?? () from /usr/lib/libwebkit-1.0.so.2
#2  0x00007ffff3d8f79d in ?? () from /usr/lib/libsoup-2.4.so.1
#3  0x00007ffff2a4947e in g_closure_invoke () from /usr/lib/libgobject-2.0.so.0
#4  0x00007ffff2a5f7f4 in ?? () from /usr/lib/libgobject-2.0.so.0
...
To get more detail I ran "apt-get install epiphany-browser-dbg" - this narrows down the crash, but not in a useful way: 
Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007ffff51a0a46 in finishedCallback (session=<value optimized out>, msg=0x7fffd801d9c0, data=<value optimized="optimized" out="out">) at ../WebCore/platform/network/soup/ResourceHandleSoup.cpp:329
#2  0x00007ffff3d8f79d in ?? () from /usr/lib/libsoup-2.4.so.1
#3  0x00007ffff2a4947e in g_closure_invoke () from /usr/lib/libgobject-2.0.so.0
#4  0x00007ffff2a5f7f4 in ?? () from /usr/lib/libgobject-2.0.so.0
..
So this crash happens in ResourceHandleSoup.cpp. Slowly I realized that this came from the webkit package. We see that the last call by name is to the function in line ResourceHandleSoup.cpp:329, that puts us at the last line of this function: 
// Called at the end of the message, with all the necessary about the last informations.
// Doesn't get called for redirects.
static void finishedCallback(SoupSession *session, SoupMessage* msg, gpointer data)
 
    RefPtr<ResourceHandle> handle = adoptRef(static_cast<ResourceHandle*>(data));
    // TODO: maybe we should run this code even if there's no client?
    if (!handle)
        return;
    ResourceHandleInternal* d = handle->getInternal();
    ResourceHandleClient* client = handle->client();
    if (!client)
       return;
..
..
    client->didFinishLoading(handle.get());
So we see there is some validation that happens, then a call to "didFinishLoading" and somewhere shortly after that it dies. didFinishLoading looks trivial: 
void WebCoreSynchronousLoader::didFinishLoading(ResourceHandle*)
 
      g_main_loop_quit(m_mainLoop);
      m_finished = true;
So my mental-debugging is stymied. I blame my headache. It looks like there is no obvious NULL-pointer deference, if we pretend client cannot be NULL. So the next step is to get the source, the build-dependencies and then build a debug version of webkit. I ran "apt-get source webkit", then editted the file ./debian/rules to add --enable-debug and rebuilt it: 
skx@precious:~/Debian/epiphany/webkit-1.2.7$ DEB_BUILD_OPTIONS="nostrip noopt" debuild -sa
*time passes* The build fails: 
  CXX    WebCore/svg/libwebkit_1_0_la-SVGUseElement.lo
../WebCore/svg/SVGUseElement.cpp: In member function  virtual void WebCore::SVGUseElement::insertedIntoDocument() :
../WebCore/svg/SVGUseElement.cpp:125: error:  class WebCore::Document  has no member named  isXHTMLDocument 
../WebCore/svg/SVGUseElement.cpp:125: error:  class WebCore::Document  has no member named  parser 
make[2]: *** [WebCore/svg/libwebkit_1_0_la-SVGUseElement.lo] Error 1
Ugh. So I guess we disable that "--enable-debug", and hope that "nostrip noopt" helps instead. *Thorin sits down and starts singing about gold* Finally the debugging build has finished and I've woken up again. Let us do this thing. I'd looked over the webkit tracker and the crashing bugs list in the meantime, but nothing jumped out at me as being similar to my issue. Anyway without the --enable-debug flag present in the call to ../configure the Debian webkit packages were built, eventually, and installed: 
skx@precious:~/Debian/epiphany$ mv libwebkit-dev_1.2.7-0+squeeze2_amd64.deb x.deb
skx@precious:~/Debian/epiphany$ sudo dpkg --install libweb*deb
[sudo] password for skx:
(Reading database ... 173767 files and directories currently installed.)
Preparing to replace libwebkit-1.0-2 1.2.7-0+squeeze2 (using libwebkit-1.0-2_1.2.7-0+squeeze2_amd64.deb) ...
Unpacking replacement libwebkit-1.0-2 ...
Preparing to replace libwebkit-1.0-2-dbg 1.2.7-0+squeeze2 (using libwebkit-1.0-2-dbg_1.2.7-0+squeeze2_amd64.deb) ...
Unpacking replacement libwebkit-1.0-2-dbg ...
Preparing to replace libwebkit-1.0-common 1.2.7-0+squeeze2 (using libwebkit-1.0-common_1.2.7-0+squeeze2_all.deb) ...
Unpacking replacement libwebkit-1.0-common ...
Setting up libwebkit-1.0-common (1.2.7-0+squeeze2) ...
Setting up libwebkit-1.0-2 (1.2.7-0+squeeze2) ...
Setting up libwebkit-1.0-2-dbg (1.2.7-0+squeeze2) ...
skx@precious:~/Debian/epiphany$
Good news everybody: The crash still happens! Firing up GDB should hopefully reveal more details - but sadly it didn't. 
Program received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00007ffff51a0a46 in finishedCallback (session=<value optimized="optimized" out="out">, msg=0xb03420, data=<value optimized="optimized" out="out">) at ../WebCore/platform/network/soup/ResourceHandleSoup.cpp:329
..
(gdb) up
#1  0x00007ffff51a0a46 in finishedCallback (session=<value optimized="optimized" out="out">, msg=0xb03420, data=<value optimized="optimized" out="out">) at ../WebCore/platform/network/soup/ResourceHandleSoup.cpp:329
329     client->didFinishLoading(handle.get());
(gdb) p client
$1 = <value optimized out>
At this point my head hurts too much, and I'm stuck. No quote today, I cannot be bothered.

17 August 2012

Dirk Eddelbuettel: Follow-up to Counting CRAN Package Depends, Imports and LinkingTo

A few days ago, I blogged about visualizing CRAN dependency ranks which turned out to be a somewhat popular post. David Smith followed-up at the REvo blog suggesting to exclude packages already shipping with R (which is indicated by their 'Recommended' priority). Good idea! So here is an updated version, where we limit the display to the top twenty packages counted by reverse 'Depends:', and excluding those already shipping with R such as MASS, lattice, survival, Matrix, or nlme. CRAN package chart of Reverse Depends relations excluding Recommended packages The mvtnorm package is still out by a wide margin, but we can note that (cough, cough) our Rcpp package for seamless R and C++ is now tied for second with the coda package for MCMC analysis. Also of note is the fact that CRAN keeps growing relentlessly and moved from 3969 packages to 3981 packages in the space of these few days... Lastly, I have been asked about the code and/or data behind this. It is really pretty simply as the main data.frame can be had from CRAN (where I also found the initial few lines to load it). After that, one only needs a little bit of subsetting as shown below. I look forward to seeing other people riff on this data set. 
#!/usr/bin/r
##
## Initial db downloand from http://developer.r-project.org/CRAN/Scripts/depends.R and adapted
require("tools")
## this function is essentially the same as R Core's from the URL
## http://developer.r-project.org/CRAN/Scripts/depends.R
getDB <- function()  
    contrib.url(getOption("repos")["CRAN"], "source") # trigger chooseCRANmirror() if required
    description <- sprintf("%s/web/packages/packages.rds", getOption("repos")["CRAN"])
    con <- if(substring(description, 1L, 7L) == "file://")  
        file(description, "rb")
      else  
        url(description, "rb")
     
    on.exit(close(con))
    db <- readRDS(gzcon(con))
    rownames(db) <- db[,"Package"]
    db
 
db <- getDB()
## count packages
getCounts <- function(db, col)  
    foo <- sapply(db[,col],
                  function(s)   if (is.na(s)) NA else length(strsplit(s, ",")[[1]])   )
 
## build a data.frame with the number of entries for reverse depends, reverse imports,
## reverse linkingto and reverse suggests; also keep Recommended status
ddall <- data.frame(pkg=db[,1],
                    RDepends=getCounts(db, "Reverse depends"),
                    RImports=getCounts(db, "Reverse imports"),
                    RLinkingTo=getCounts(db, "Reverse linking to"),
                    RSuggests=getCounts(db, "Reverse suggests"),
                    Recommended=db[,"Priority"]=="recommended"
                    )
## Subset to non-Recommended packages as in David Smith's follow-up post
dd <- subset(ddall, is.na(ddall[,"Recommended"])   ddall[,"Recommended"] != TRUE)
labeltxt <- paste("Analysis as of", format(Sys.Date(), "%d %b %Y"),
                  "covering", nrow(db), "total CRAN packages")
cutOff <- 20
doPNG <- TRUE
if (doPNG) png("/tmp/CRAN_ReverseDepends.png", width=600, heigh=600)
z <- dd[head(order(dd[,2], decreasing=TRUE), cutOff),c(1,2)]
dotchart(z[,2], labels=z[,1], cex=1, pch=19,
         main="CRAN Packages sorted by Reverse Depends:",
         sub=paste("Limited to top", cutOff, "packages, excluding 'Recommended' ones shipped with R"),
         xlab=labeltxt)
if (doPNG) dev.off()
if (doPNG) png("/tmp/CRAN_ReverseImports.png", width=600, heigh=600)
z <- dd[head(order(dd[,3], decreasing=TRUE), cutOff),c(1,3)]
dotchart(z[,2], labels=z[,1], cex=1, pch=19,
         main="CRAN Packages sorted by Reverse Imports:",
         sub=paste("Limited to top", cutOff, "packages, excluding 'Recommended' ones shipped with R"),
         xlab=labeltxt)
if (doPNG) dev.off()
# no cutOff but rather a na.omit
if (doPNG) png("/tmp/CRAN_ReverseLinkingTo.png", width=600, heigh=600)
z <- na.omit(dd[head(order(dd[,4], decreasing=TRUE), 30),c(1,4)])
dotchart(z[,2], labels=z[,1], pch=19,
         main="CRAN Packages sorted by Reverse LinkingTo:",
         xlab=labeltxt)
if (doPNG) dev.off()

Gunnar Wolf: More on hacker conferences and sexual harassment

At the beginning of this year, I blogged about a Mexican security-minded hacker conference scaring away its female audience by advertising in a sexist way. I don't know if it is the need to be l33t or kewl, to show off that hackers are really socially inept, or what... but this seems to carry on. I know many are familiar with the red/yellow card project (and followup) by kdotcdot. I am (rightfully? naively?) proud that at DebConf we have achieved a clean conference environment, without such problems... Yes, I know that, during the last ~year at DebConf11 we discussed an anti-harassment policy (look at the thread, it was quite interesting!), came up with standards of respect And Debian as a whole voted on a GR that ratifies a diversity statement. The fact that we had those very positive discussions, documents and events shows we needed to have them. But, again, this shows that being a hacker does not necessarily mean being a jerk. And I'm very proud to be part of this community. I recently stumbled across a very nice, insightful post by Valerie Aurora on The Ada Initiative: Supporting women in open technology and culture DEFCON: Why conference harassment matters. Take a good read at it. I hope it helps shape other hacker groups in a less-aggressive, more welcoming way. Oh! And before closing: Be sure to at least skim through both Valerie Aurora's and kdotcdot's comments. LOTS of insight in them.

31 July 2012

Martin Pitt: My impressions from GUADEC

I have had the pleasure of attending GUADEC in full this year. TL;DR: A lot of great presentations + lots of hall conversations about QA stuff + the obligatory be er,ach = . Last year I just went to the hackfest, and I never made it to any previous one, so GUADEC 2012 was a kind of first-time experience for me. It was great to put some faces and personal impressions to a lot of the people I have worked with for many years, as well as catching up with others that I did meet before. I discussed various hardware/software testing stuff with Colin Walters, Matthias Clasen, Lennart Poettering, Bertrand Lorentz, and others, so I have a better idea how to proceed with automated testing in plumbing and GNOME now. It was also really nice to meet my fellow PyGObject co-maintainer Paolo Borelli, as well as seeing Simon Schampier and Ignacio Casal Quinteiro again. No need to replicate the whole schedule here (see for yourself on the interwebs), so I just want to point out some personal highlights in the presentations: There were a lot of other good ones, some of them technical and some amusing and enlightening, such as Frederico s review of the history of GNOME. On Monday I prepared and participated in a PyGObject hackfest, but I ll blog about this separately. I want to thank all presenters for the excellent program, as well as the tireless GUADEC organizer team for making everything so smooth and working flawlessly! Great Job, and see you again in Strasbourg or Brno!

23 July 2012

Enrico Zini: Evolution's old odd mail folders to mbox

Evolution's old odd mail folders to mbox Something wrong happened in my dad's Evolution. It just would get stuck checking mail forever, with no useful diagnostic that I could find. Fun. Not. Anyway, I solved by resetting everything to factory defaults, moving away all gconf entries and .evolution/ files. Then it started to work again, of course then I needed to reconfigure it from scratch. It turned out however that some old mail was only archived locally, and in a kind of weird format that looks like this: 
$ ls -la Enrico/
total 336
drwx------ 2 enrico enrico   4096 Jul 23 03:05 .
drwxr-xr-x 7 enrico enrico   4096 Jul 23 03:12 ..
-rw------- 1 enrico enrico   3230 Dec  4  2010 113.HEADER
-rw------- 1 enrico enrico  14521 Dec  4  2010 113.TEXT
-rw------- 1 enrico enrico   3209 Oct 22  2010 134.HEADER
-rw------- 1 enrico enrico   2937 Oct 22  2010 134.TEXT
-rw------- 1 enrico enrico   3116 Jun 27  2011 15.
-rw------- 1 enrico enrico   3678 Jun 27  2011 168.
-rw------- 1 enrico enrico     73 Apr 27  2009 22.1.MIME
-rw------- 1 enrico enrico   3199 Apr 27  2009 22.2
-rw------- 1 enrico enrico     88 Apr 27  2009 22.2.MIME
[...]
I couldn't even find the name of that mail folder layout, let alone conversion tools. So I had to sit down and waste my sunday break writing software to convert that to a mbox file. Here's the tool, may it save you the awful time I had today: http://anonscm.debian.org/gitweb/?p=users/enrico/evo2mbox.git Note: feel free to fork it, or send patches, but don't bother with feature requests. Evolution isn't and won't be a personal interest of mine. Anything that makes an afternoon at my parents more tiresome than a whole busy month of paid work, doesn't deserve to be. Luckily they now seem to have changed the local folder format to Maildir.

21 July 2012

Enrico Zini: Giving away distromatch

Giving away distromatch at last year's Fosdem I tried to inject a lot of energy into distromatch but shortly afterwards I've had to urgently rewrite the nm.debian.org website. After Lars Wirzenius GTDFH talks in Bologna and Varese I wrote a tool which, among other things, is able to scan my home dir and list how many projects I'm working on. The output was scary. Like, they are too many. Like, I couldn't even recite the list out of memory. And since I couldn't do that, I had no idea there were so many. And I kept being stressful because I couldn't manage to take care of them all properly. Now that I became conscious of the situation, it's time to deal with it like a grown up, and politely back off from some of my irresponsible responsibilities. Distromatch is one of them. It had just started as a proof of concept prototype, and I had the vision that it could be the basis for a fantastic culture of sharing and exchange of information across distributions. I need to distinguish the vision from the responsibility. I still have that vision for distromatch, but I cannot take responsibility for making it happen. So I am giving it up to anyone who has the time and resources to pick up that responsibility. Current status It works well enough as a prototype. I believe it can successfully map a large enough slice of packages, that one can prototype stuff based on it. I have for example used it to export the Debtags categories for other distros, and the resulting file looked big enough to be used for prototyping category-based features on distributions that don't have them yet. I think it also works well enough to support a few common use cases, like sharing screenshots, or doing most of the work of converting dependency lists from a distro to another. And finally, anyone can deploy it, and work on it. Existing data sources Everything I index in the Debian distromatch deployment is available at http://dde.debian.net/exports/distromatch/. The rpm-based data in there comes from an export script I wrote that runs on Sophie, but which I cannot maintain properly. This is an experimental export of Fedora and OpenSUSE data: http://tmp.vuntz.net/misc/distromatch/distromatch-opensuse-fedora.tar All existing export scripts are found in distromatch git repo on gitorious. Contacts I gathered at Fosdem At Fosdem I devoted quite some work to get contacts from all possible distributions and software repositories, so that distromatch could be hooked into them. Here is a dump of what I have collected: Some of those contacts may have "expired" in the meantime: I wouldn't assume all of them still remember talking with me, although most probably still do. My commitment for the time being I am happy to commit, at the moment, to maintaining a working data export for Debian data. I can take responsibility for making it so that the Debian data for it stays up to date, and to fix it asap if it isn't the case. I hope that now someone can take distromatch over from me, and make it grow to achieve its great potential.

16 July 2012

Thorsten Glaser: Apache 2, https clients linked against GnuTLS, connection errors

I ve been debugging a weird problem at work after upgrading a complex system from lenny to wheezy, some https clients failed to connect: GNU wget and Debian s version of lynx(1) which is linked against libgnutls26 fail. NSS applications continue to work, as does cURL; wget and lynx on MirBSD (linked with OpenSSL of course) work. Even Debian s gnutls-cli tools from both gnutls26 and gnutls28 work. Huh. The error_log shows renegotiation problems, yet setting the new Apache 2 configuration option to use insecure renegotiation doesn t help either. (The option is a total #FAIL: its only other value is use secure TLSv1.x renegotiation , but I don t want/need SSL renegortiation at all, anyway.) Natureshadow told me this was a hot issue on Debianforum at the moment, yet, nobody had a clue or enough information to file a formal bugreport against (initially) apache2, as that s what changed. I tracked it down on a new VM with no configuration otherwise, and here are my findings so others don t run into it.

Tracking down the problem, this can be reduced to the following configuration (minimised, to show the problem) in /etc/apache2/sites-enabled/1one:

	<VirtualHost *:443>
		ServerName wiki-70.lan.tarent.de
		RedirectMatch permanent . https://evolvis-70.lan.tarent.de/
		SSLEngine on
		SSLCertificateFile /etc/ssl/W_lan_tarent_de.cer
		SSLCertificateKeyFile /etc/ssl/private/W_lan_tarent_de.key
		SSLCertificateChainFile /etc/ssl/godaddy.ca
	</VirtualHost>

Do not mind the actual content, this is a very stripped-down demo on a not-actually-set-up-yet box.

Same is valid for the companion configuration file /etc/apache2/sites-enabled/2two:

	NameVirtualHost *:443
	<VirtualHost *:443>
		ServerName evolvis-70.lan.tarent.de
		SSLEngine on
		# workaround for BEAST (CVE-2011-3389), short-term
		SSLCipherSuite RC4-SHA
		SSLCertificateFile /etc/ssl/W_lan_tarent_de.cer
		SSLCertificateKeyFile /etc/ssl/private/W_lan_tarent_de.key
		SSLCertificateChainFile /etc/ssl/godaddy.ca
		SSLProtocol TLSv1
	</VirtualHost>

Turns out the BEAST workaround was at fault here: the differing SSLCipherSuites between the vhosts (on the same Legacy IP / TCP Port tuple, as we use Wildcard SSL Certificates) made Apache 2 want to renegotiate, so either commenting it on 2two or, better, adding it to 1one helped. Interestingly enough, the SSLProtocol directive did not matter (in my tests). So, keep SSL settings synchronised between vhosts. In fact, those were already from include files, but 2two was from the Evolvis 5 generation, whereas we added to 1one an Include of the httpd.ssl1.inc file generated by the previous releases of EvolvisForge and had not switched those legacy vhosts to the new configuration, as everything worked on lenny. This wlog entry brought to you by the system administrators of tarent solutions GmbH and the Evolvis Project, based on FusionForge.

13 July 2012

Jonathan McDowell: Revoked: 0xF1BD4BE45B430367 (1024D key)

I generated 0x94FA372B2DA8B985 (my 4096R GPG key) back in 2008, and revoked my old 1024R v3 key around the same time. However I left my main 1024D key alone, figuring I'd get round to revoking it at some point once the new one was sufficiently trusted. This probably happened some time ago, so today I have finally revoked this key: 
pub   1024D/0xF1BD4BE45B430367 1999-10-26 [revoked: 2012-07-13]
uid                            Jonathan McDowell <noodles@earth.li>
If you're not in the habit of refreshing your GPG keyring regularly now might be a good time to (gpg --refresh-keys) or at the very least pull my updated key (gpg --recv-key 0xF1BD4BE45B430367) to make sure you don't accidentally continue to use it. (If I haven't signed your key with my stronger one please do wave a fingerprint/ID at me next time we meet, and ask me for the same in return.)

Next.

Previous.