Search Results: "pm"

28 September 2010

Guido G nther: What maps the shared objects of a security upgrade?

For a security update of a shared library to take effect processes need to be restarted in order to use the libraries from the newly installed package. The openssl and freetype updates got me thinking again if I caught all the affected services on servers and desktops/laptop systems I'm running. So I put together a small script that takes a package name and scans /proc/pid/maps for processes that map libraries from this package, it then looks for init scripts in these packages. It can also hook into apt and restart the services after the upgrade. Since the mapping to the init scripts is quiet dumb at the moment this should be used with a bit of caution though. The script whatmaps can be fetched form here. Debs are also available. Besides Debian packages it also handles RPMs and already proved useful since it pointed me to clamav being possibly affected by the recent bzip2 advisory. Flattr this

Lior Kaplan: Welcome The Document Foundation and LibreOffice

For those who haven t heard: OpenOffice.org Community announces The Document Foundation
The community of volunteers developing and promoting OpenOffice.org sets up an independent Foundation to drive the further growth of the project The Internet, September 28, 2010 The community of volunteers who develop and promote OpenOffice.org, the leading free office software, announce a major change in the project s structure. After ten years successful growth with Sun Microsystems as founding and principle sponsor, the project launches an independent foundation called The Document Foundation , to fulfil the promise of independence written in the original charter. The Foundation will be the cornerstone of a new ecosystem where individuals and organisations can contribute to and benefit from the availability of a truly free office suite. It will generate increased competition and choice for the benefit of customers and drive innovation in the office suite market. From now on, the OpenOffice.org community will be known as The Document Foundation . Oracle, who acquired OpenOffice.org assets as a result of its acquisition of Sun Microsystems, has been invited to become a member of the new Foundation, and donate the brand the community has grown during the past ten years. Pending this decision, the brand LibreOffice has been chosen for the software going forward. The Document Foundation is the result of a collective effort by leading independent members of the OpenOffice.org community, including several project leads and key members of the Community Council. It will be led initially by a Steering Committee of developers and national language projects managers. The Foundation aims to lower the barrier of adoption for both users and developers, to make LibreOffice the most accessible office suite ever. The Foundation will coordinate and oversee the development of LibreOffice, which is available in beta version at the placeholder site: http://www.libreoffice.org. Developers are invited to join the project and contribute to the code in the new friendly and open environment, to shape the future of office productivity suites alongside contributors who translate, test, document, support, and promote the software. Speaking for the group of volunteers, Sophie Gautier a veteran of the community and the former maintainer of the French speaking language project has declared: We believe that the Foundation is a key step for the evolution of the free office suite, as it liberates the development of the code and the evolution of the project from the constraints represented by the commercial interests of a single company. Free software advocates around the world have the extraordinary opportunity of joining the group of founding members today, to write a completely new chapter in the history of FLOSS . [...]
So welcome The Document Foundation and LibreOffice. I already can see how does the community benefits from this due to the merge with go-oo:
Q: What does this announcement mean to other derivatives of OpenOffice.org?
A: We want The Document Foundation to be open to code contributions from as many people as possible. We are delighted to announce that the enhancements produced by the Go-OOo team will be merged into LibreOffice, effective immediately. We hope that others will follow suit.
(from http://www.documentfoundation.org/faq/) So, will Oracle contribute the OpenOffice.org trademark to the community ? Time will tell
Filed under: LibreOffice, Openoffice.org, Proud to use free software

27 September 2010

Steve McIntyre: Armel buildds and porter box hosted at ARM

One of the nice things that I've been involved with since starting to work at ARM in Cambridge is setting up newer, faster machines to help with the armel port. We have six machines hosted in the machine room here now: arne All of these machines are Marvell DB-78x00-BP development boards, each configured with a 1GHz Feroceon processor (ARM v5t), 1.5GB of RAM and a 250GB drive attached via SATA. They're nice machines, reasonably powerful yet (as with many ARM-based machines) they draw very very little electrical power even when working hard. These very boards were used for a while by the folks at Canonical to help build the Ubuntu armel port, but now we've got them. In terms of configuration, these machines are not quite fully supported in Debian yet, though. The kernels we're using are locally-built, based on the Debian linux-source-2.6.32 package but with a .config (marvell.config) that's tweaked slightly to add the support for these boards. There aren't any source changes needed, so I'm hoping to get support added directly in Debian, either as a new kernel flavour or (preferred) as a patch to an existing flavour. I've had conflicting advice about whether the latter is possible, so I'm going to have to experiment and find out for myself. UPDATE 2010-09-28: I've tested, and it seems that the boards will need a new flavour after all, as the config is incompatible with the closest other config (kirkwood). Ah well... I had no end of trouble trying to get make-kpkg do the right thing, so on advice from Ben I built the kernel using "make deb-pkg", a standard target in the Linux kernel's build system: fakeroot make -j2 deb-pkg DEBEMAIL=93sam@debian.org DEBFULLNAME="Steve McIntyre" KDEB_PKGVERSION=buildd23 Annoyingly, that wouldn't work when cross-compiling either so I had to build the kernel natively. To make the resulting kernel image package install properly (and, just as importantly, allow for future easy upgrades for the DSA folks), I also needed the following tweaks to the Debian system: Finally, I've tweaked the uboot config on the machines to use the uImage and uInitrd files that are generated:
Marvell>> setenv IDE ide reset
Marvell>> setenv loadkernel ext2load ide 0:1 0x2000000 /uImage
Marvell>> setenv loadinitramfs ext2load ide 0:1 0x3000000 /uInitrd
Marvell>> setenv bootboth bootm 0x2000000 0x3000000
Marvell>> setenv bootcmd setenv bootargs \$\(bootargs\)\;$(IDE)\;$(loadkernel)\;$(loadinitramfs)\;$(bootboth)
Marvell>> saveenv
And that's it, as far as I can see. I'll now wait for people to tell me what I've got wrong above... :-)

25 September 2010

Siegfried Gevatter: Language Identification and it s state in Free Software

Working on a new feature for eSpeak GUI I started looking into language identification. Forcing users to manually choose the text s language is a botheration, so trying to guess it by checking which system dictionary contains the most words from the text or some other method would surely be beneficial. After a quick search I learned that it s much easier than this: it s possible to reliably determine the language based on statistic n-gram information. Ignoring the fact that now I officially hate Firefox, Chromium, OpenOffice.org and everyone else there for not implementing this and having me spend the day changing the spell-checker s language, I was left with the choice on how to use this in eSpeak GUI. The first option I found was TextCat, which is also the only library I ve found to be packaged for Debian. However, ignoring the fact that upstream isn t maintaining it any more (such a library shouldn t need too much maintainance, after all), the package declares incorrect dependencies (bug filled a month ago, no response yet) and the API is also pretty crappy (it requires a physical file indicating the location of the statistic models). Unrelated to that, I ve also found that the Catalan text samples it includes are incorrect, so the same may be true for other languages. I guess it d make sense to work on a new (and completely Unicode) language samples collection. I ve thought of using something like the Universal Declaration of Human Rights since this way all languages can have the same text, but being more of a legal thing it may be biased by some words being too repetitive. Looking for other alternatives to the TextCat library I ve only found the following: So it looks like I ll have to start by getting a good collection of text samples I can use to generate the statistic data. Then I have several options on how to actually use it. As I see it, those are my possibilities:
  1. Fixing libtextcat s packaging and just using that.
  2. Taking it over as new upstream maintainer. Not my preferred option as I don t really feel like maintaining a C library at this point.
  3. Trying to convince the maintainer of the new TextCat (with last commit January this year and a more sane API) to re-license it in a GPL-compatible way, packaging that and seeing how that one works (haven t tried it out yet).
  4. Writing my own implementation in Python, maybe based upon this example or TextCat.pm.
Any other ideas, pointers to some library I may have missed or offers to collaborate are very welcome. Please also note that my intention in writing this post is not only to rant about there being no well-maintained ready-to-use library being available, but especially raising awareness on the topic of language identification. I d love to see this feature all around the desktop, just like (and in combination with) spell-checking, which is already omnipresent. Related posts:
  1. espeak-gui 0.2
  2. Introducing espeak-gui

11 comments
Siegfried-Angel Gevatter Pujals, 2010. Permalink License Post tags: , ,

23 September 2010

Joerg Jaspert: FTPMaster meeting, IV - still without Baklava. But with minutes now.

Wellwell, the meeting is over since Sunday evening, but for some reasons I only was able to send the minutes today. Go get them over at the project list while they are still fresh, before another one snatches them away from you. They are basically a short summary of what we did or intend to do soon. I do like such kind of meetings, they usually help a lot, both in getting things done but also in lifting up my spirit and motivation to do something. And so I am actually looking forward to the springtime plan we have and hope it will work out nicely, and that we get all of the team together. The more, the better. :)

Thijs Kinkhorst: Grid authentication made easy: the TCS eScience project

As I'm currently attending the EUgridPMA meeting in Zagreb I thought I'd share a bit of this project I've been working on for the past year: the TCS eScience project. In the scientific world lots of calculations are performed on distributed computing platforms known as grid. Because users of other institutions will be using your hardware, authentication is needed and this problem has been solved with x.509 personal certificates. The problem however, is that these certificates of course have to be issued by some CA. Currently in Europe alone there are over 40 active CA's, even multiple per country, dedicated to this job. They are accredited through the EUgridPMA which meets regularly. For scientists, it's often cumbersome to obtain a certificate: find your local CA, present an ID (probably in person), and sometime later receive your certificate. The process can take days or even weeks. Scientists are not interested in CA's but just want to practice science. Our solution is a central web portal where users can request a certificate and have it delivered in minutes. This leverages the fact that identities of scientists normally have already been vetted at their home institution: users log in to the portal via federated login. Their home institution passes a special attribute that declares "Yes, we have really seen photo ID of this person and the name is correct". This attribute must of course not be passed for guests or test users or role accounts. However, it may still be easy to mass-provision it. In the Netherlands for example, the employer is required by law to verify the identity of each employee, so all employees can be automatically assigned the attribute. After logging in and uploading (or generating) a csr, the request is passed in the back end to the Comodo API. This also means that we do not need to perform the complex operations of running an online CA (with hardware crypto devices, crl's, etc.). The use of Comodo is part of the same deal as the TERENA Certificate Service for host SSL certificates. The Comodo API responds within two minutes with the certificate which the user can download. Currently 10 European countries are involved with the project (nl no se fi dk at cz it fr be), and more have shown interest. The certificates we issue have been accredited by the EUgridPMA so can be used on the grid. A separate but similar service is being set up for 'regular' personal certificates for the academic community, e.g. for s/mime usage. More details are in the presentation and paper by portal software developers Henrik and Thomas at the most recent TNC.

22 September 2010

Luca Bruno: Report from the Debian/Ubuntu Community Conference, ITA 2010

From the 17th to the 19th of September in Perugia, Italy, it took place the 5th edition of the Italian Debian Community Conference, which has been attended by many contributors and users. For the first time, the event has been organized in collaboration with the Italian Ubuntu community, as to create a new joint conference in order to foster shared contributions and emphasizing the large common ground of our projects. This new experimental kind of mini-conference was then labeled DUCC-IT, to reflect both the local profile and the mutual collaboration. After the initial social night, spent discussing of several ongoing free software efforts and having dinner all together, the conference official opening started on Saturday, temporarily housed at University of Perugia, with a series of talks and hands-on session aimed at recruiting new contributors to work on development, translations, documentation writing and marketing. It has been a good opportunity to celebrate the Software Freedom Day too, in collaboration with FSUG Italia and the participation of some local schools.

DUCC-IT '10 Group Photo The event been also attended by some members of the Debian Women and Ubuntu Women teams (whose goal is to promote women participation in both projects), who organized a round-table debate taking the Italian panorama as a study case. The discussion embraced different topics, ranging from the wide difference in numbers, to the deep causes of this phenomenon and how to improve the situation. With the help of the hacklab staff (hosting the debate), an audio/video streaming has been made available in real-time, and many remote participants joined us with comments and questions. This new kind of collaboration between our communities was found to be really positive and more events has already been drafted for the next year, including a translation sprint and a contributors meeting. We encourage worldwide local communities to try and engage in a similar experiment: organizing and joining a DUCC event will be pure fun. A detailed report of the conference will be soon available, completed by photos, participants comments, video records and slides for the talks.
We d really like to thank the Math Department of University of Perugia, the Projectz On Island hacklab, FSUG Italia, the Ubuntu-it community and everyone who contribute to this event.

Michal Čihař: Fixing phpMyAdmin bugs in Launchpad

Once in a time when looking at my Debian Developer's Packages Overview, I just start to think there is way too much bug reports for phpMyAdmin package in Launchpad. Today the threshold was 30 :-). Managing them is usually not that hard, because vast majority of them are just duplicates of same issue. Basically all problems come with the fact that dbconfig-common really does not behave nicely when database server is not configured and user tries to configure it. I know people should just answer no to first question, whether to use dbconfig-common, but they don't and it then horribly fails. Today I finally decided to ignore all errors from dbconfig-common in our packaging scripts, so you will get an error, but the package will be installed. As it seems that this is what most Ubuntu users expect after all. This also shows huge difference between Debian and Ubuntu users - such bug has never appeared in Debian bug tracker, but has zillions of duplicates in Launchpad in various incarnations (LP#618852, LP#621569 and the most favorite LP#456674).

21 September 2010

Russ Allbery: Lightweight DNS servers

Since I got a lot of responses to my previous journal entry about tinydns, I wanted to share the answers with everyone else (and with search engines) in case anyone else was wondering the same thing. The general consensus choice for a simple authoritative DNS server was NSD (which is also packaged for Debian, currently as nsd3). I took a brief look and indeed it would do what I want. It's still a bit more complicated than I would prefer, but nowhere near as bad as BIND, and it handles all modern DNS features (IPv6, DNSSEC, etc.). Several people mentioned unbound, which looks like a great solution for a different problem than the problem that I have. It's a caching DNS server rather than an authoritative DNS server, although it supports some authoritative overrides. There was also one recommendation of PowerDNS (Debian package pdns-server and friends), which I'd heard of before and which I think I'd turn to if I was looking for a full-featured DNS server. I think it's overkill for my tiny problem, but it has the neat feature that you can run an arbitrary command to provide DNS responses. That means that it could potentially replace lbnamed, should we need something with more features for some of the DNS tricks that we do with Stanford. Finally, tinydns development hasn't completely stopped since djb stopped working on it, and there are maintained forks that have patches to support new record types. Still no DNSSEC support that I'm aware of, but continuing to use it with patches to support SRV and AAAA records is quite appealing, since I much prefer the zone file format (the "standard" zone file format is a horrible bodge) and, of course, I'm already familiar with it. If a new version were packaged for Debian, I'd probably just keep using it. And as a bonus follow-up from my original post, on the topic of the divergent handling of iptables for IPv4 and IPv6, Martin Krafft recommended ferm, which looks very interesting but which I haven't yet investigated further.

Joey Hess: solar charge controller adventure

Xantrex controller, temporarily badly installed
As planned I have installed the Xantrex C-35 charge controller in the offgrid house. This was a bit of an adventure.. First I had to wait around in town all day for the UPS man to deliver the thing. Of course he didn't until 4, so by the time I got over to the house it was almost dusk. Further delayed by necessary visiting with the nearest neighbor (a country thing), who claims there are "panthers" up this holler.. So by the time I was ready to work, it was nearly too dark. First I checked the battery bank which I had left charging for a week without a controller or backflow diode. It had gained only half a volt (to 10.5v), so now I knew I really did need the controller in order for the sun to work for me while I am away. Then, barely able to see, I swapped that bank to provide house power. By 10 pm, using just my laptop, I'd drained 1.5 volts from it and it was as low as it should go. I had 4 hours of laptop battery. This was a bit of a problem since I tend to stay up until 3 am on it. Instead, I spent 2 hours, by kerosine lantern light, exposing, tracing, and testing wiring. I was able to find 2 wires that were clearly going to the battery bank and had current, and 2 others that were labeled as going to the panels, and, in the dead of night, had no current. Great, I thought, that's the wires I need, let's hook up the controller.. And I did, although only to the battery, since after all I couldn't be certain the other two wires were for the panels, without seeing voltage on them in the daytime. (This turned out to be probably wise, because in the morning, when I tried it, the controller reported no PV voltage. They turned out to be mislabled, and actually connected the battery to the breaker box.)
The problem with doing all that stuff at midnight is it took me quite a long time to wind down. I normally start winding down at 10 am and get to sleep at 3. Last night I got to sleep at 5. My four hours of battery for my laptop were nursed all hours through the night. Mostly thanks to powertop and the improved power saving of the kernel these days. And to not having wifi or anything enabled. I do wonder if my recent use of redshift to redden the screen causes the it to use less power at night, as I have never gotten five hours out of this laptop before.
Anyway, to get the controller hooked up to the PV array and actually charging the batteries, I ended up having to rip lots of wiring out of the wall, until I simplified the mess to something I could understand. And at the moment I've bypassed the PV array's breaker, which is certainly a safety hazard. So more wiring to do, but I did get it working. Finally, I hooked the house wiring up to the same battery bank that is being charged from the panel. This house had been using a 2 battery bank setup, because without a charge controller to moderate the PV voltage (17-25 volts), you can't also connect a load that expects 12 volts. The charge controller fixes that problem. The other battery bank is sitting unused for now. Later, I will equalize and connect the two, in order to get one large(ish) battery bank. Now, during the sunny fall daytime, I can use all the power I want; it's free from the sun. Battery bank shouldn't drain at all when I use power during the day, and the controller should charge them up and keep them topped off, so they'll be there for me at night. Now to verify that it's working like it's supposed to!
(I posted some documents and more thoughts from this here).

20 September 2010

Steve Kemp: A new Blog::Spam release.

I've just released a new Blog::Spam module to CPAN. The Blog::Spam module is 99% of the code behind the blog & forum spam detection service. On the whole I'm pleased with the way that development has gone, and although I'm going to keep tinkering it is essentially "complete". Now I need to go back and fix bugs in chronicle, asql, and rinse. ObSubject: "You are just an ordinary man in a cape!" - Batman Begins.

Richard Darst: How DebCamp relates to DebConf

The week before DebConf has been traditionally been DebCamp. This has been advertised as a time for core teams to come early to work on their projects together, and this post will talk more about what that means.At DebCamp, there is not much infrastructure set up yet. We have (unfurnished) hacklabs, accommodations, and usually food from the very beginning. We'll have Internet access, but the network and power distribution isn't set up yet. People find what they can and make their own space to work. The DebConf team works on setting up hacklabs starting on day 0 or 1, and the DebCampers serve as the test subjects to make sure that everything works.There intentionally aren't many scheduled activities during DebCamp. The organizers have too much else to do, and events aren't the main purpose of DebCamp. If some people wanted a place to meet, they could probably find a place, though. Some years, there have been one or two workshop type events on the DebCamp schedule.DebCamp has frequently been explained by "Development teams like debian-installer and debian-edu also have the possibility of meeting beforehand for undisturbed collaborative work. ... Interested teams should contact the organizers upfront to make special arrangements." (From the DC10 website) Also included in DebCamp are the DebConf organizers - we desperately need that time to pull organizers from over the world together to get us on the same page locally. Furthermore, anyone without a workplan can attend DebCamp by paying their actual costs.However, saying that DebCamp is limited to these groups of people doesn't reflect reality: Anyone can attend. Thus the DebCamp description doesn't necessarily reflect the actual limits for attending. This has the appearance of resulting in (and some people believe it has) a DebCamp which is biased towards those who well-connected with organizers, since they know that basically anyone can go and get a free extended holiday with Debian people. In fact, getting anyone to review the submitted workplans has met resistance in the last few years.For DebConf10, we did manage to reach agreement to filter out possible tourist-like people, but it was a similar (or only slightly more strict) criteria than DebConf itself. This was done very late, too late to be fair to force adjustments in travel plans of too many people. The difference between the DebCamp listed criteria and who has gotten sponsored for DebCamp in the past caused contention among some new organizers this year. Why didn't they know that they would have qualified for DebCamp past years? The moral of the story is that we need to be more clear on the purpose of DebCamp and make the actual qualifications for attending concrete and well-known so that it remains fair.Of course, it's also true that non-core developers and other interested people to attending DebCamp is a good thing, since it is a strong motivator to get more involved in Debian. Local people are encouraged to attend and learn/hack/volunteer. DebCamp isn't meant to be exclusive, just to reflect the realities of a limited budget.
DebCamp is a very important part of DebConf, both for Debian development purposes and for DebConf set-up purposes. However, since it is a week of limited attendance with mostly the same people attending year after year, we need to ensure that it remains transparent to the rest of Debian.Tags: debconf

Michal Čihař: New default theme for phpMyAdmin

There seems to be increasing demand for changing default theme in phpMyAdmin. Well the obvious question is which one to choose :-). There are currently 3 themes compatible with 3.3 branch, what is still far away from current master, however I don't see that they would mean big improvement over current ones. Maybe we should just make some contest for new theme? Anyway if we decided to change the theme, we will still get complains from users that we did the change. At least this is what I remember from last attempt to do so several years ago :-).

19 September 2010

Joerg Jaspert: FTPMaster meeting, III - discovering new Restaurants for fun and profit

And there, another day come and passed, still no real breakage, only some small things here and there. We are doing something wrong, definitely. Ok, granted, providing everyone who has experimental in their sources.list with a free upgrade path due to us suddenly forgetting that apt actually really wants this NotAutomatic: yes flag in the release instead of just looking it up in our database, later on doing the same to all of the backports user (accidently, happens when you have identical codebases), does count enough for a small icecream. But we didn t have anything serious, which is great. Alex finished patching dak rm in a way that we can now actually close all bugs attached to a (source) package we remove and was thinking of enhancing it to actually also be able to close wnpp bugs attached to it (think of O: bugs for example). Mark had design work on g-p-s, which actually is how do we generate packages and sources and how to move the config sanely into a database . Sounds easy, but trust me, it isn t, when you consider all the special cases Debian has. He also jumped in helpfully whenever there was a question around some coding and stuff, at one time turning up with the insanity/genius/younameit code that allows us to transfer even more of dak.conf into the database. This gets us a good way along the way that should end up with a dak.conf consisting of only the database details and nothing else. My job today looked similar like yesterday. Merge code I got, eat, remove lots of values from dak.conf. This weekend besides totally missing out on Baklava, had us change our dak code quite a bit: 46 files changed, 982 insertions(+), 1060 deletions(-). Besides that we do have about 18 points on our meeting notes, a mail going off to one debian lists sometime the next days summarizing it a bit more.

Obey Arthur Liu: Google Summer of Code 2010 Debian Report

Hello fellow developers, The summer is over :( but I m happy to announce that this year s Summer of Code at Debian has been better than ever! :) This is indeed the 4th time we had the privilege of participating in the Google Summer of Code and each year has been a little different. This year, 8 of our 10 students succeeded in our (very strict!) final evaluations, but we have reasons to believe that they will translate into more long-term developers than ever, all thank to you. The highlight this year has been getting almost all of our students at DebConf10. Thanks again this year to generous Travel Grants from the Google Open Source Team, we managed to fly in 7 of our students (up from 3!). You certainly saw them, presenting during DebianDay, hacking on the grass of Columbia, hacking^Wcheering our Debian Project Leader throwing the inaugural pitch of a professional baseball game or hacking^Wsun-tanning on the tr s kitsch Coney Island beach. Before I give the keyboard to our Students, I d like to tell you that it will be the pleasure and honor of Obey Arthur Liu (yours truly, as Administrator) and Bastian Venthur (as Mentor) to represent Debian at the Summer of Code 2010 Mentors Summit on 23-24 October 2010, at the Google Headquarters in Mountain View. Like last year, we expect many other DDs to be present under other hats. We will be having 2 days of unconference on GSoC and free software related topics. We all look forward to reporting from California on Planet and soc-coordination@l.a.d.o! All of our students had a wonderful experience, even if they couldn t come to DebConf, that is best shared in their own voice, so without further ado, our successful projects: Multi-Arch support in APT by David Kalnischkies, mentored by Michael Vogt apt-get install MultiArch does mostly work now as most code is already merged in squeeze, but if not complain about us at deity@l.d.o! Still, a lot left on the todo list not only in APT so let us all add MultiArch again to the Release Goals and work hard on squeezing it into wheezy. :) Debbugs Bug Reporting and Manipulation API by David Wendt Jr., mentored by Bastian Venthur Hello, I m David Wendt, and I went to Debconf10 to learn more about the development side of Debian. Having used it since the 9th grade, I ve been intimately familiar with many of Debian s internals. However, I wanted to see the developers and other Debian users. At DebConf, I was able to see a variety of talks from Debian and Ubuntu developers. I also got to meet with my mentor as well as the maintainer of Debbugs. Content-aware Config Files Upgrading by Krzysztof Tyszecki, mentored by Dominique Dumont Config::Model is now capable of manipulating files using shorter and easier to write models. Thanks to that, packagers may start experiment with creating upgrade models. Further work is needed to support more complicated config files Dominique Dumont is working on DEP-5 parser, I ll shortly start working on a cupsd config file parser.
The best thing about DebConf10 is that every person I talked with knew what I was doing. I had a mission to get some feedback on my project. Everybody liked the idea of making upgrades less cumbersome. On the other side, it was my first visit to United States, so I decided to go on a daytrip on my own (instead of staying inside the building, despite heat warnings). I had a chance to visit many interesting places like Ground Zero, the UN headquarters, Grand Central Terminal, Times square and Rockefeller Center that was a great experience. Hurd port and de-Linux-ization of Debian-Installer by J r mie Koenig, mentored by Samuel Thibault Debconf10 was great! Among other people working on the installer, I met Aur lien Jarno from the Debian/kFreeBSD team and we worked together on a cross-platform busybox package. Besides, the talks were very interesting and I ve filled my TODO-list for the year.
For instance I learned about the Jigsaw project of OpenJDK, and how Debian would be the ideal platform to experiment with it. More generally, some people think Debian could push Java 7 forward and I d like to see this happen. Smart Upload Server for FTP Master by Petr Jasek, mentored by Joerg Jaspert I must say that it was great time for me in NY, I ve met and talked and coded with people from ftp-master team like Torsten Werner who helped me to push the project a bit further and with some other people who were looking forward to release of the tool which I hope they will use quite soon. Everybody interested, everybody excited, really cool place and time. And I can t forget the Coney Island beach and stuff, lot of fun, lot of sun;) Aptitude Qt by Piotr Galiszewski, mentored by Sune Vuorela Currently, development branches support full features searching, viewing extended package s informations, performing cache and packages operations. Code and GUI still require a lot of work which will be continued. Informations about further progress could be found on aptitude mailing list and repository rss channel. Debian-Installer on Neo FreeRunner and Handheld Devices by Thibaut Girka, mentored by Gaudenz Steinlin For me, DebConf 10 started at the airport, where Sylvestre Ledru (whom I didn t know of before) was wearing a GSoC 2007 t-shirt, that is, given the circumstances, almost equivalent to say I m a hacker, I m going to DebConf 10 .
I ve spent my time at the conference attending various talks, hacking, meeting DDs and other hackers (amongst others, my co-mentor Per Andersson, Paul Wise, Julien Cristau, Christian Perrier, Cyril Brulebois, Martin Michlmayr, Colin Watson and Otavio Salvadores who I have to thank for his patience while dealing with my questions), chatting, cross-signing keys, rushing to finish eating before 7pm, getting sunburnt, sightseeing (thanks, Arthur, for the lightning-fast tour of Manhattan!), and so on. Debian Developers and community, we count on you. See you next year! (cross-posted to debian-devel-announce@l.d.o and soc-coordination@l.a.d.o)

Joerg Jaspert: FTPMaster meeting, II - NO baklava

Wanted to post this yesterday, but somehow didnt work out, so here we go now Another meeting day passed, another lot of commits into our repository, another day without entirely breaking the system. But wait, one more chance to break it today, maybe we finally manage. :) Yesterday we got volatile sorted out for Squeeze (Lenny stays like it is, seperate). Its now a suite on ftp-master, and we have all the neccessary information and workflow around it defined. At least so far that we can operate the suite, we don t define what gets into it, we just take that from the volatile team (or, I think, in future, from the SRMs) in a kind of textfile similar to how testing works. Besides having Alex unbreak our bts categorize script, Mark fixing up our BYHAND processing and implementing the volatile stuff I mentioned above, we also merged a patchset from Luca that allows us to provide the packages.debian.org service with a tree containing changelogs, copyright files, NEWS.Debian and README.Debian. Metadata basically. Which they extracted in the past, and that was taking up quite a lot of resources. Much easier if we provide it and they just use it. This is now done for both, the main archive as well as the backports archive. As soon as I upgrade the security archive it will also be done there, which should additionally enable packages.debian.org to display those files for all archives (run by FTPMaster). This update is scheduled for the upcoming week, DSA already gave me the needed bits and pieces on a host so I can do it without breaking security during the process. We currently export this metadata set only to the packages.debian.org host. We do want to provide it to more, but I am waiting for a reply from DSA as to which machine I should use for the open access to those files. When I have that reply it should be available for you all. As a sidenote from that: If you need to compute any data that has to do with the Debian archive (information out of source/binary packages or something around it) and if you do that with your own scripts - you are most likely doing it wrong. Talk to us. We might be able to help you extracting the data you need and keep it updated. Recent examples for that are the above changelogs, but also things like the package-file-mapping snapshot uses to easily determine which file belongs to which source/version, but also information like the Maintainers index, the Uploaders index and whatnot. The simple way is - ask us. We are processing a shitload of data, it might not be too complicated to let your set fall out somewhere at the side, but if you never ask you will never find out Today we start working on some internals of dak and will have some more discussion on agenda points like data.debian.org and the git source format. We already had some words on other points, but those are ones which will take a long time more to fix, like generally reworking the internals of our database to enable us having multiple archives with pretty different access levels inside one machine. (Think of security archive on ftpmaster host. You want to ensure embargoed security issues actually stay embargoed until they are released. Means a good set of neccessary changes inside the database to handle this. We have ideas, but getting them implemented will take a bit).
And of course we had been out for food too, helps to stay alive. Turkish restaurant. All nice and well - until they told us they have NO BAKLAVA. Args. Suckers. They ruined this year! WAAAAAAAAAAAAAAAAH. I wish them a windows OS for their live! If not worse! (Can it be worse than windows?) ;-)

18 September 2010

Robert Collins: Maintainable pyunit test suites fixtures

So a while back I blogged about maintainable test suites. One of the things I ve been doing since is fiddling with the heart of the fixtures concept. To refresh your memory, I m defining fixture as some basic state you want to reach as part of doing a test. For instance, when you ve mocked out 2 system calls in preparation for some test code that represent a state you want to reach. When you ve loaded sample data into a database before running the actual code you want to make assertions about that also represents a state you want to reach. So does simply combining three or four objects so you can run some code. Now, there are existing frameworks in python for this sort of thing. testresources and testscenarios both go some way towards this (and I and to blame for them :) ), so does the zope testrunner with layers, and the testfixtures project has some lovely stuff as well. And this is without even mentioning py.test! There are a few things that you need from the point of view of running a test and establishing this state: Large test suites or test suites dealing with various sorts of external facilities will also often want to optimise this process and put the same state into place for many tests. The (and I m not exaggerating) terrible setUpClass and setUpModule and other similar helpers are often abused for this. Why are they terrible? They are terrible because they are fragile; there is no (defined in the contract) way to check that the state is valid for the next test, and its common to see false passes and false failures in tests using setUpClass and similar. So we also need some way to reuse such expensive things while still having a way to check that test isolation hasn t been compromised. Having looked around, I ve come to the conclusion we ll all benefit if there is a single core protocol for doing these things, something that can be used and built on in many different ways for many different purposes. There was nothing (that I found) that actually met all these requires and was also tasteful enough that folk might really like using it. I give you fixtures . Or on Launchpad. This small API is intended to be a common contract that all sorts of different higher level test libraries can build on. As such it has little to no policy or syntatic sugar. It does have a nice core, integration with pyunit.TestCase, and I m going to add a library of useful generic fixtures (like temporary directories, environment isolators and so on) to it. I d be delighted to add more committers to the project, and intend to have it be both Python 2.x and 3.x compatible (if its not already my CI machine isn t back online after the move yet, I m short of round tuits). Now, if you re writing some code like:
class MyTest(TestCase):
    def setUp(self):
        foo = Foo()
        bar = Bar()
        self.quux = Quux(Foo(), Bar())
        self.addCleanup(self.quux.done)
You can make it reusable across your code base simply by moving it into a fixture like this:
class QuuxFixture(fixtures.Fixture):
    def setUp(self):
        foo = Foo()
        bar = Bar()
        self.quux = Quux(Foo(), Bar())
        self.addCleanup(self.quux.done)
class MyTest(TestCase, fixtures.TestWithFixtures):
    def setUp(self):
        self.useFixture(QuuxFixture)
I do hope that the major frameworks (nose, py.test, unittest2, twisted) will include the useFixture glue themselves shortly; I will offer it as a patch to the code after giving it some time to settle. Further possibilities include declared fixtures for tests, and we should be able to make setUpClass better by letting fixtures installed during it get reset between tests.

17 September 2010

Jo Shields: Mono mythbusting, September 2010 edition

There are corners of the Internest where foolish people congregate, and invent stories. These foolish stories are then read as gospel by trusting people, and reposted, until the original made-up source is concealed from view. As an attempt to stem this flow of disinformation, here are some commonly held but incorrect beliefs about the Mono framework, and an explanation of the reality of the situation, as far as I understand it. The next Mono version is co-developed with Microsoft There is a grain of truth behind this one, but it s a gross mischaracterisation. Mono 2.8, when it ships, will bundle, for convenience, a number of Free Software libraries, which are released by Microsoft under a license considered Free Software by the Free Software Foundation, the Ms-PL. These are: So, to summarise, there are five Free Software libraries written by Microsoft under the Ms-PL included in Mono 2.8 but only two of those five are new, and none of them were co-developed in any meaningful sense. Mono development is dead There have been reports that Mono development has ended, on the basis that no commits have been made to Novell s Subversion server for a few weeks. However, these reports miss one minor detail Mono has moved to Github.com. There were 35 commits from 9 different people to the main mono.git repository (of dozens of repositories under the main Mono project) in the last 2 days, at time of writing far from dead. Mono lacks features found in Microsoft.NET Mono lacks libraries found in Microsoft.NET, such as the Windows Presentation Framework GUI toolkit. In terms of features, i.e. things implemented at a compiler or runtime level, Mono is typically more advanced. This is helped in no small part by Mono s Free Software development model, allowing experimentation and what if changes to the core runtime which a sluggish corporate behemoth like Microsoft cannot accommodate. To give three real-world examples, Mono allows embedding of its compiler as a service, and provides a REPL shell this is a planned feature for .NET 5.0, but has been available for years in Mono; Mono.Simd provides a number of data structures which will run on any version of Mono or Microsoft.NET, but will use optimized CPU extensions like SSE when run on a sufficiently new version of Mono, on an appropriate architecture as far as I m aware, there is nothing like this available or planned for Microsoft.NET. Mono is able to produce fully compiled, static executables which do not need to JIT anything at runtime this is used for iPhone compilation, for example, where JITters are not permitted. There is no comparable feature in Microsoft.NET. Clearly, Microsoft.NET can only be thought of as more featureful if one defines features in terms of does it have lots of libraries? in terms of functionality, Mono is ahead. Mono can sneak onto your system without your knowledge If you don t have the mono-runtime package installed, you can t run Mono apps. It is possible to install some Mono apps alongside the awful-yet-popular mononono equivs package, since the popular equivs script fails to place Conflicts on the correct packages (F-Spot will be blocked, Tomboy will not). No package in Debian or Ubuntu embeds its own copy of the Mono runtime, and we have no plans to make any changes to packaging which would allow execution of any C# application without mono-runtime installed. If one is using a different OS, then things may be different e.g. The Sims 3 for Mac & PC uses embedded Mono, which you wouldn t know about without looking. Canonical are pursuing a pro-Mono agenda, and are responsible for it being pushed in Debian Mono development has been happening in Debian for longer than Canonical has existed the first upload was made in April 2002. Ubuntu is made primarily from software already available in Debian deemed of sufficient quality, and when F-Spot and Tomboy became parts of the default Ubuntu desktop system in 2006, both pieces of software were already available in Debian and deemed of sufficient quality. Nobody working on Mono in Debian is paid by Canonical actually, that s not entirely true, three packages related to accessibility are officially maintained by someone who was hired by Canonical after doing the initial packaging work when he worked for Novell. But the Mono runtime itself, Canonical have no influence over its direction in Debian. As for a pro-Mono agenda , they ve always taken an extremely pragmatic approach to language choice, never showing any real preference for one language or another when it comes to app selection. They don t exhibit any overt anti-Mono policies, which is not the same thing. The names of people contributing to Mono in Debian are not secret check the pkg-mono team page on Alioth. The System.Windows.Forms namespace is protected by Microsoft patents The truth is, nobody knows for sure if SWF, or any other part of Mono, or any other framework such as Vala or Python, is covered by Microsoft patents. The way the US patent system is contrived, it is actively dangerous to check whether something contains any patents when you write it, as you are liable for triple damages should it later emerge that there WAS a patent, even if your searching missed the fact. You cannot patent an API or namespace only a specific implementation of a software concept, in those countries where software patents are permitted. There has never been any evidence shown that Mono s implementation of SWF, or indeed any part of Mono, infringes any Microsoft-held patents, because if that were the case, then the code would be rewritten to avoid the issue much like the approach taken by Linux kernel developers when a patent becomes apparent. The belief within the Mono community is that the core parts of Mono as defined in ECMA334/335 are safe (they are covered by the Microsoft Community Promise patent grant); any of the Ms-PL libraries mentioned above is definitely safe (Ms-PL includes an explicit patent grant); and the rest of the package is likely safe too (on the basis that it is a named component of the Open Innovation Network s list of protected software and on the basis that there s unlikely to be anything patentable in one of many implementations of basic ideas like database connectors). Nobody knows for sure, because that s how the system works. But, again, nobody knows for sure that Microsoft patents don t apply to other frameworks such as Python there is simply a belief and an assumption that they do not. MonoDevelop contributors removed GPL code from MonoDevelop, in an attack on the GPL This is somewhat disingenuous, given MonoDevelop is LGPL. The MonoDevelop team excised the remaining GPL code (and there wasn t much of it) in order to grant greater Freedom to developers. Previously, the entire MonoDevelop IDE was a GPL combination, meaning any add-ins for the IDE also needed to be GPL, regardless of developer choice. Now, any developer can write an add-in for MonoDevelop, using the license of their choice, whether another Free license like the Mozilla Public License, or a proprietary closed-source add-in. They are still welcome to produce GPL add-ins, if they want to, as well. Mono won t run random Microsoft.NET apps anyway, so isn t really cross-platform Actually, this one is often true. When developers write apps for Windows primarily, they rarely take the time to think is this the correct way to do it? and will often plough on with an assumption about the Windows way of doing things. It might be simple things like filesystem assumptions (assuming a case-insensitive filesystem, or assuming a backslash is used to separate directories, not a forward slash). It might be more involved, such as using P/Invokes into Windows-specific C libraries, when a more cross-platform alternative is possible. So, often, random applications for Microsoft.NET won t run on Mono. The reverse is also true F-Spot or GNOME Do are fairly heavily tied to Linux (or to UNIX-like OSes with X11, at any rate), through the libraries they invoke being fairly platform-specific. You can write platform-specific Java, with one quick piece of JNI, too. It should be noted, however, that Mono makes the chance of .NET applications being ported to Linux (and/or Mac) much more likely, since even in the worst case scenario, a company only needs to fix a portion of their source to make it cross-platform. The Mono team have a tool called MoMA, which will scan an application and its libraries, and give you detailed reports on the app s portability. This info is used at both ends by app vendors who want to become more portable, and by the Mono team who want to fill in the most frequently encountered blanks. And, it should be stressed, writing cross-platform apps is entirely possible if one desires it e.g. the IRC client Smuxi is pure cross-platform C#, and the executables compiled on Mono on Linux will run fine on Microsoft.NET on Windows. Portability in this direction is important too consider how many people have been introduced to Free Software thanks to availability of Free apps on Windows like OpenOffice.org, Firefox, Pidgin or GIMP. You can download Tomboy for Windows, and work is ongoing on fixing Banshee portability issues (which are mostly caused by gStreamer). Mono on Android is a terrible idea which will be super slow There are two efforts to enable developers to write Android applications with Mono a paid proprietary product from Novell called MonoDroid, and an untitled porting effort by big-name Android hacker Koushik Dutta. I have no insight into MonoDroid since it hasn t been released yet, but Koush did some benchmarks of his work which exhibited some remarkable figures compared to Dalvik (and even compared to Sun Java for ARM). Those numbers haven t been updated to reflect the Froyo Dalvik JITter, but Mono on Android is still very exciting from a performance perspective. Mono isn t really Free Software The source code is all available under Free Software licenses. So, um, yes it is. We don t need Mono because we have $LANG By the same logic, we don t need $LANG because we have Mono. By all means, use the language of your choice other developers will use the language of their choice too. If you want to use Vala or Python or Java, then by all means, go ahead. It doesn t mean there s no room for more languages, better suited to other usage patterns. Haskell is great for some types of development, so is Fortran, and so is C#. You might not need Mono because of $LANG, but there are others in the world with different needs. Mono apps are all slow, and crash your computer, and stuff No userland app can crash your Linux system, unless there s a bug in the kernel, or you have some severe problems with your hardware. If you ve ever observed a crash as a result of running a Mono app, then it s really coincidence that Mono is tickling whichever part of your system is busted. As for slow startup time may well be slower than for apps written in C or Vala. There s a delay due to the JITter needing to compile all the libraries used by the application (we may AOT the most common libraries in a future Mono package version, at upstream s recommendation). Once an app is running, it should be fast compared to a Python app, and memory-light compared to a Java app. The new garbage collector in Mono 2.8 should also offer significant improvements to performance, especially under heavy workloads. Ubuntu s default GTK+ themes require Mono This is my favourite recent nonsense to emerge from the Internest. There are reports mostly restricted to IRC and blog comments, that (paraphrasing here) removing Mono removes the Ubuntu themes . Here s the reality: in Ubuntu 10.04, a new visual style was implemented throughout the distro. As part of this, small icons (e.g. notification area icons) were set to be black-and-white by default, and colourised when attention is needed (e.g. the power-off icon turns red when a reboot is needed, the messaging indicator turns green when there s a new message). All of these new monochrome icons are in a package named ubuntu-mono . Removing the ubuntu-mono icon package will also remove the new Ubuntu GTK+ themes, in the light-themes package. So there s your explanation: the themes have a dependence on some monochrome icons, not on the Mono framework.

Joerg Jaspert: FTPMaster meeting, I

I announced it some while ago on debian-project, we have a FTPMaster meeting this weekend. It started out with a little workout session View image before we started discussing some things from our agenda. In the middle of the discussion we took a little sidestep, looking for a victim to promote to ftpmaster. So we selected someone not attending, he can t run away screaming. And for that: Send your condolences over to Torsten Werner. Or maybe congratulations, your call :) Right now Mark is fixing up our byhand processing and Alex is doing some other Debian work, while I m merging patches from Luca. Thats it for now, sometime soon we are going to eat, after which we put some more discussions on our agenda. Still two more days to go :)

Russell Coker: Preferring Low Quality and Microsoft Software

Is Low Quality in Italian Academia related to the choice of Low Quality Software? Diego Gambetta and Gloria Origgi wrote an interesting paper titled L-worlds: The curious preference for low quality and its norms [1]. The paper describes how in Italian universities (and large portions of Italian life) there are covert agreements that both parties in a transaction (the employee and the employer or the buyer and the seller) will deliver less than agreed while pretending that they are offering the agreed exchange. People who offer high quality in exchanges are discriminated against because they make people who offer low quality in exchange feel guilty. Nathan suggests that this is the explanation for people choosing to pay for inferior software from Microsoft instead of getting superior software for free [2]. Now it does seem quite plausible that someone who is offering low quality goods in the manner of Italian academia would refuse to consider software from any company other than Microsoft, after all the easiest way of selecting software is to phone a MS representative and be told exactly what to buy. But I don t think that this explains even a significant fraction of the people who refuse free software. There is no direct analogy between bilateral agreements to produce low quality and the choice of MS Software because MS Software is quite expensive (they demand what would be considered a high quality trade in the jargon of the paper). If someone was to buy one of the cheaper laptops in Australia (around $650 new) and upgrade to the full version of Windows 7 along with purchasing the home version of MS Office then the cost of software would be almost as much as the cost of hardware. If they wanted one other MS product then the cost of MS software would probably be greater than the cost of hardware. Hardware costs are steadily falling and MS prices are only increasing, for people who use MS software we should expect that soon the MS tax will be the majority of the costs of running a typical PC. Good Reasons for Choosing MS One thing we have to consider is that there are people who have good reasons for using MS software. One example is the companies that depend on proprietary applications which are central to their business. When the entire company s data is stored in an undocumented proprietary database it s really not easy to change to a different application even when everyone in the company knows the software to be of amazingly low quality. If the vendor of the proprietary application in question decides to only support MS Windows then it s customers (victims?) have no choice about which OS to use. One interesting thing to note about such companies that are locked in to proprietary software is that the amount that they spend per year on license and support fees is usually greater than the cost of hiring one good programmer. If a few such companies formed a consortium to develop free software to manage their business where each company paid the salary of one programmer then after a couple of years of development they could move to the free software and reduce their operating expenses. Another category of users who have a good reason to choose MS is the people who play games seriously. If you want to play games then MS Windows does offer some real advantages. The price of games will usually be a fraction of the hardware cost (the serious gamers spend a lot more on hardware than most people) and MS Windows is apparently the best PC OS for commercial games. Personally I ve found that there are more than enough free games on Linux to waste my time, Warzone 2100 [3] is one that I currently play, and I ve tried Battle for Wesnoth [4] in the past and found it too time consuming and addictive. How a Preference for Low Quality could lead to Microsoft I think that everyone who has any significant experience in the computer industry has encountered companies that have large areas of low quality. This generally tends to be in large corporations as small companies can t afford the waste. In some large corporations Linux on the desktop is never considered, even when people are hired as Linux sysadmins and there are obvious productivity benefits to having the same OS on the desktop as on the servers (even if two desktop PCs are required so that proprietary software can be run on MS-Windows). Major wrote a good satire of the corporate IT non-working culture with a comparison to medical work [5], it illustrates the principle of a coalition to ensure low quality. He later documented how he was sacked by the low quality coalition at a company that uses a lot of Microsoft software [6]. So it does seem that when customers don t care at all about the quality of the result it does help drive some sales for Microsoft. But that doesn t explain the market share that they have. It takes a lot of work to get Market Share without Quality Microsoft has spared no effort in gaining market share. Every possible effort including buying out small competitors, aggressively FUDing competition, using all manner of legal attacks (including the threat of patent suits), and deliberately breaking standards has been used. I think it s reasonable to assume that the MS senior management are not entirely stupid, they do so many things that are unethical and possibly illegal because they know that they need to do so to maintain their market share. The result is that the market capitalisation of MS is almost as high as that of Apple and Apple makes vastly superior products. Given the amount of effort that MS uses to keep market share it seems apparent that they aren t just relying on customers not caring about quality. Should users have to Understand Computers? My observation is that most users don t want to know much about how their computers work. The desire to understand computers seems to be about as common as the desire to understand cars, people just want to buy one that looks good and have it work. The difference is that cars are very compatible while computers aren t. Cars have the same controls in the same places and large parts of the design are specified by law so that they can t differ between models. Proprietary software is usually intentionally incompatible with other software (both open and proprietary) to try and gain a competitive advantage. Hardware is often incompatible due to the rapid developments in technology and the requirements for new interfaces to take advantage of new features. In concept it seems reasonable for someone who is about to spend $30,000 on a car and $1000 on a computer (for hardware and software) to spend 30 times longer considering which brand of car to buy. One could argue that more than 30 times as much consideration should be given to the car as most people can t afford to discard a car that they don t like. As people spend a few minutes considering which brand of car to buy they can be excused for spending a few seconds considering which type of computer to buy. But once a choice has been made about which software to use it s very difficult to change to something else, while in comparison it s easy to drive a car that was manufactured by a different company. So a poorly informed choice made at an early stage can have costly long-term affects when buying software. If we had mandated open standards for file formats and data interchange then users would be able to make choices that don t result in their data being locked in to some proprietary format. Such standards could be set through government tender processes, if every government agency was to only buy software that complies with open standards then the proprietary software vendors would scramble to make their products less incompatible. The result would be that bad choices in purchasing software could become learning experiences that result in better purchases in future instead of being a lock on users that forces them to keep using the same software that doesn t satisfy their needs. Conclusion I think that the best thing about the paper by Diego Gambetta and Gloria Origgi is that it highlights the issue of low quality. No-one wants to be considered a loser, so maybe this can encourage people to strive for high quality (or at least try to make their work suck a little less). Regardless of the conclusion they eventually reach, it s probably good for people to occasionally wonder do I suck? .

Next.

Previous.