Search Results: "will"

11 June 2011

Nathan Handler: Google Summer of Code Update #3

Last week was a rather slow week. I did some work on allowing the dashboard to store and process information that does not come from the BTS. This exposed a few issues with the way we were storing and presenting information. After talking to Matt, we decided that the dashboard should deal with task objects. These tasks would contain basic information such as a name, person assigned to it, status, and possibly a link to a bug. This means that we would be able to start up a new DEX project by simply choosing a project name and creating a basic text file containing a list of tasks (package names, patches, bugs, etc). I also have been trying to get a hold of some people who participated in the last DEX project. I am interested in getting some more feedback about what went well, problems they encountered, and tools that might have made things easier. I am also talking to some people who are currently working on starting up new DEX projects to try and make dextools align with as many workflows as possible. If you fall into either of these categories, I would really appreciate it if you contact me. There is also a new wiki page that I created so that it is easier for people who are not interested in me or the Google Summer of Code to quickly see what dextools is about and monitor its progress. I will be adding more information over the next few days. Currently, I am continuing to work on the dashboard and talking to people about their DEX experiences. I have also begun working on some scripts to generate graphs that will go on the dashboard. These graphs will allow us to quickly check our progress on different projects. They will also allow us to see and acknowledge individuals working on the tasks. I will add more information about these graphs to this blog and to the dextools wiki page.

Nathan Handler: Google Summer of Code Update #3

Last week was a rather slow week. I did some work on allowing the dashboard to store and process information that does not come from the BTS. This exposed a few issues with the way we were storing and presenting information. After talking to Matt, we decided that the dashboard should deal with task objects. These tasks would contain basic information such as a name, person assigned to it, status, and possibly a link to a bug. This means that we would be able to start up a new DEX project by simply choosing a project name and creating a basic text file containing a list of tasks (package names, patches, bugs, etc). I also have been trying to get a hold of some people who participated in the last DEX project. I am interested in getting some more feedback about what went well, problems they encountered, and tools that might have made things easier. I am also talking to some people who are currently working on starting up new DEX projects to try and make dextools align with as many workflows as possible. If you fall into either of these categories, I would really appreciate it if you contact me. There is also a new wiki page that I created so that it is easier for people who are not interested in me or the Google Summer of Code to quickly see what dextools is about and monitor its progress. I will be adding more information over the next few days. Currently, I am continuing to work on the dashboard and talking to people about their DEX experiences. I have also begun working on some scripts to generate graphs that will go on the dashboard. These graphs will allow us to quickly check our progress on different projects. They will also allow us to see and acknowledge individuals working on the tasks. I will add more information about these graphs to this blog and to the dextools wiki page.

10 June 2011

Neil Williams: Putting the magic smoke back into the ring main

Haven't been in my new home that long and through a long chain of contacts, my neighbour was finally able to contact me. (Just goes to show that having Debian friends nearby who like board games turns out to make some things JustWork.)
So, when I got the message and got home the flat was fine - just with no electrical power. Murphy's Law ensured that my laptop was also low on power, as was my mobile phone and my landline is cordless and the router was, naturally, not working. My communication options were suddenly rather limited.

The 50yr old wiring in this block finally surrendered the magic smoke and caused a fire downstairs (mostly melted insulation) before it disconnected. I got home around 1pm, electricians to fix the problem arrived about 5pm and haven't finished quite yet. (It's now 12:11am). A relatively large hole in the lawn was dug (a hard task in itself seeing as this area is officially in a drought and the ground is like iron), two junction boxes rebuilt, new connection made from downstairs to upstairs, the work just went on and on.

I'll get some accurate details when I manage to speak to my brother - I do a little bit of hardware but I deal in 1W or a few hundred mA, when a 80A cable melts I'm just a bit out of my depth. I've not seen a fuse that size since I left school.

Thanks to those Debian and non-Debian people who helped me actually contact the relevant people today. I was due to get so many things done today, instead I got through the worry to end up incredibly bored - it turns out to be surprisingly difficult to find things to do when there is no mains power, all the batteries in mobile devices are flat and it's too dark to read.
:-(

Nathan Handler: irssi bansearch.pl script

As a freenode staffer and Ubuntu Operator, a frequent question that I am asked is, Why can t I join #channel? or Why can t I talk in #chanel? . To answer this question, I normally check the ban and quiet lists for #channel, and then I check what channel modes are applied. For small channels, this is not too bad, but for channels with very large ban/quiet lists, it can take some time. This is why I have created bansearch.pl. This irssi script will attempt to determine why a user is unable to join/talk in a channel. Currently, the script is maintained in a Launchpad bzr branch. To get and install the script, run the following commands from your home directory. bzr branch lp:~nhandler/+junk/bansearch
mkdir -p ~/.irssi/scripts/autorun
ln -s ~/bansearch/bansearch.pl ~/.irssi/scripts/bansearch.pl
ln -s ~/.irssi/scripts/bansearch.pl ~/.irssi/scripts/autorun/bansearch.pl
You can leave out the last command if you do not want the script to run automatically when you start irssi. Using the script is quite simple. You can use it like this: /script load bansearch.pl
/bansearch nick [#channel]
If you do not specify a channel, it will use the current channel. The script will currently look for bans and quiets matching the user. It has some limited extban support, but it does not support CIDR bans. It also supports looking at channel modes that might prevent the user from joining/talking. The script knows whether the user is identified, so $a extbans, $~a, and cmode +r are all supported, and the script will only warn you about them if they match the user. The script was written for freenode, so while basic ban/quiet searching might work on other IRC networks, the channel mode checks will probably not work properly. The script is still fairly new, so there are still bugs in it. If you find any, please let me know, and I will try and patch them as quickly as possible. I am also open to any feedback about how to improve the script.

Nathan Handler: Google Summer of Code Update #2

I ve been spending the past few days working on a dashboard for DEX. I ve been using Python, javascript, and jquery to create this dashboard (partially as a learning exercise). Earlier today, I started thinking that a visual mockup of the dashboard might be helpful, as it will allow me to create and revise a design based on feedback without having to touch the code. At this point, I had already written a fair bit of code and had a rough but working early version of a dashboard coded. So for the first mockup, I simply created it to match my code. DEX Dashboard Thanks to the generosity of the folks at Balsamiq, I was able to get myself a free key to create the mockup above. As I mentioned earlier, this is what the code currently looks like, not what it will look like. Next, I m going to be spending some time revising that mockup so I have a visual goal to work towards. I also will be cleaning up the code that I currently have and making it available in a public VCS.

Nathan Handler: Google Summer of Code Report #1

Here is my first official Google Summer of Code report. As I still had classes, the first week of coding was rather slow. However, after my first phone call with my mentor, Matt Zimmerman, things started to pick up. I spent most of my time working on a dashboard/portal for the Debian Derivatives Exchange (DEX). We decided that the best approach to this would be to store as little information ourselves as possible. Instead, we will try to pull most of our information from other sources, such as Debian s BTS. The main advantage to this is that package maintainers do not need to change any of their workflows, but they still get to benefit from DEX s work. Currently, I have a python script that gets a list of usertags and bugs for debian-derivatives@lists.debian.org from the BTS. This script will end up getting run by cron. It stores the information it gets in static HTML files. Finally, JQuery and Javascript are used to present the final dashboard to the user. I created a mockup showing how this looks. I also have a functioning website that looks and acts like the mockup, but the code is not publicly available yet (see below). DEX uses Alioth to host its git branches and website. I have spent some time trying to familiarize myself with the recent changes that have taken place on Alioth, as well as trying to figure out the best way to store and execute my script to generate the website. So far, it looks like people are still deciding on the best way to handle having websites that are updated from VCS hooks (due to things being split between vasks and wagner). I will continue to look into this, and might try using an alternate host until things settle down. Next, I plan to work on updating the dashboard to provide a way to edit, store, and display certain extra fields of information. This is necessary for certain pieces of information that are not appropriate for adding to bugs and in a few other cases. I will also be taking some time to talk to a few people who were active contributors to the last DEX project. This will allow me to learn more about their experience, find out what changes they feel should be made to make things run smoother, and get feedback on my current Summer of Code work. Finally, although I have been posting updates on my Summer of Code work to my blog [2][3], Matt thinks that having all of this information in one place will be useful. As a result, I will be creating another wiki page about the project that will contain all of the information people need to know. This page will be updated as the summer progresses. I am also hoping to have my code available in a public VCS by the time of my next progress update so that people can check it out and provide feedback. In addition to the dashboard, I will be working on creating a few other tools to assist with the creation and tracking of new DEX projects. I will also be coming up with a way to generate graphs to visually allow us to track our progress on projects versus time as well as show the work done by people at the individual level. This last task is part of a goal to get more people involved in the DEX project. While we may only be a few weeks into the actual Google Summer of Code, I have already learned a lot of new things. I am looking forward to working more with Matt and other members of the community over the course of the summer.

Gunnar Wolf: Reading revolutions: Online digital text and implications for reading in academe A (very informal) review

It's been a long time since I last took some time to read First Monday A great online publication, if you are not familiar with it, that I would categorize (and no, I'm not probably well-informed in it to be authoritative) as dealing with social, psychological aspects of the cultural shifts the online world has brought upon us (often dealing with topics related to Free Software communities, the reason I first met the publication). Firstmonday is an Open Access champion from early on. It follows an approachable but academic format (this means, it is peer-reviewed, its articles give extensive lists of references, and the articles are not the short reads we have got used to finding on the net, but, quoting from their audience profile, English is not the first language of many First Monday readers; A large percentage of First Monday readers are not a part of academia; Cultures, educational backgrounds, and fields of study vary greatly among First Monday readers.) This means, it's at least a great publication for me to follow :) Anyway After a long time not following it, I have just read Reading revolutions: Online digital text and implications for reading in academe, by Barry W. Cull; First Monday, Volume 16, Number 6 - 6 June 2011 Nice, interesting read. As I was planning on telling about the article to a couple of friends more into the subjects than myself, I'll comment+quote some bits on it. Before going any further: The article makes several references to Maryanne Wolf. No relation to her I'm not lulling my (two? are you both still reading?) readers towards her work ;-) The article talks about the differences social, even dips into some physiological aspects that the activity of reading is sustaining due to the shift from an activity done mainly from books (or other similar printed material) to the computer screen. Of course, we all know from our own experience many of the basic traits Shorter attention spans, a different reading pattern (skimming instead of reading; browsing through several related items instead of in-depth reading a single text as a knowledge unit). The article begins with an overview of reading and humankind. Cull quotes Maryanne Wolf's phrase, despite the fact that it took our ancestors about 2,000 years to develop an alphabetic code, children are regularly expected to crack this code in about 2,000 days . An interesting point I never thaught of is the start of reading as a purely mental activity, detaching reason from verbalization, ~1200 years ago:
Interestingly, these early scribes first did their work by reading out loud to themselves. Not until the ninth century did monastic regulations begin requiring silent reading. By the thirteenth century the practice of men reading silently and alone became commonplace. This shift to silent reading was a profound change, one that Darnton suggested involved a greater mental adjustment than the shift to printed text
. One last important point in this older history, that I'm quoting because I know I'll need the reference later on for one of my texts is about reading as a social activity Yes, also related to the quietness I just mentioned:
Interestingly, with the democratization of the printed text, there was a return to reading aloud. Reading was a solitary silent process only for the educated elite who could afford to buy books. For the rest of the population, as Darnton pointed out, reading was a social activity which took place in workshops, barns, and taverns [and] while children played, women sewed, and men repaired tools.
After this introduction (obviously one of the most interesting parts to me), Cull gives numbers showing how reading is evolving (in the USA and Canada), and quotes some prediction on how the future will end up adapting. Of course, I live in a place with a very different society, so I cannot comment much. Then he confronts some studies regarding specifically leisure reading, as it is a much more trustable factor than just literacy (in a world as highly literate as ours is, many people only read when they have to and have never or very seldom experienced the pleasure of reading just for the sake of it), bringing into the discussion the Internet (and computers in general) usage patterns. I found also very interesting the next section, regarding the pattern changes many libraries are facing now, specially academic/research-oriented libraries:
For several millennia, right up until just two decades ago, the central role of a library was to collect and house physical texts: from clay tablets, to scrolls, to printed books (Battles, 2003; Manguel, 2006). While printed text remains essential to most academic libraries, today s libraries have also become a core conduit via which researchers access scholarly texts online. Just within the last few years, Canadian academic libraries, in a situation similar to libraries throughout the Western world, have reached an interesting tipping point librarians now spend the majority of their collections budgets on electronic instead of printed texts.
( )
Libraries are convinced that digital text, now in its infancy, is likely to have a long future. Not only do they purchase electronic texts, but most academic libraries have also become publishers of electronic texts, whether they are digitizing large portions of their book holdings, or focusing on scanning a relatively small number of archival documents from their unique special collections.
And yes, doing some work with our Institute's library, I can confirm this trend. About e-books: I have got quite into that topic since the Kindle won my heart (and my money!) half a year ago. The little device completly changed my reading habits, I have read lots more since I carry it. And yes, I have never considered a full tablet-like device The article talks long about the difference, about the disadvantage that multitasking means to the human brain (oh, do I suffer from it!) I liked this snippet, quoting Steve Jobs a couple of years ago, regarding an Apple e-book reader question:
When Apple was rumoured to be working on an e book reader a few years ago, CEO Steve Jobs expressed his lack of interest: It doesn t matter how good or bad the product is, the fact is that people don t read anymore, he reportedly said, continuing by stating that 40 percent of the people in the U.S. read one book or less last year. The whole conception is flawed at the top because people don t read anymore (Markoff, 2008). Nicholas Carr (2010) has summed up Apple s involvement in the tablet phenomenon this way: Jobs is no dummy. As a text delivery system, the iPad is perfectly suited to readers who don t read anymore .
The issue for me is, I do enjoy reading, but I am an information addict. I know that if I have parallel information flows, my attention will surely dilute between them. Of course, as I read this article on-screen, it was hard for me to take the needed discipline not to be distracted by IMs or IRC highlights during the whole reading (which was also as an excercise for myself ;-) ). I am surprised to see this on student preferences (and even more surprised to see this data comes from my university):
In a study of students at the Universidad Nacional Aut noma de M xico (UNAM), the majority of students preferred print, and 63 percent reported that they could bear reading a document on a computer screen for no more than one hour (Ram rez Leyva, 2003). When it comes to course textbooks, a marked student preference for paper over e books has recently been found (Woody, 2010).
As we approach the end, it talks about another important topics I have often tried (and often failed) to communicate to my users: That of paratext, the meaning of the different texts, covers, items, layouts, etc. that are not part of the text itself but do shape the way we face it. To some of us, this seems obvious. To others, it is so hard to understand It closes with two more topics I will refer to. One is the permanent connectivity. All the time, more people are connected virtually all of their waking time. This affects not only learning habits but priorities. Will this near constant access to information interfere with students desire to comprehend and remember information, necessary to the educational process of turning it into knowledge? Author and university business school lecturer Don Tapscott recently suggested that students might not have to stress about the details those you can check Finally, regarding the continuity and ellaboration found in texts that are each time more common He quotes Maryanne Wolf:
I am worried about kids who are immersed in digital culture. They will get to college and they will have been Twittering so much that they won t have the patience to read those really long cognitively convoluted and complex sentences. They may not have developed those rich networks which are required in order to read at a high level of sophistication. The effort is what we are going to lose. They are becoming not so much a lazy reader, but an atrophied reader
As you can see, not only this post is meant to tell my couple-of-interested-friends to read the article, but it's mainly meant as a mental placeholder for myself. I will be surely refering to some of these items.

8 June 2011

Lars Wirzenius: Living off freedom

I was laid off a couple of months ago, and have been thinking about what I want to do next. Actually, I've mostly been writing code for my pet projects, but earning a living again is occasionally on my mind. I would, very much, like to earn a living while deveoping interesting free software that I would like to use myself. There's a bunch of options available: In this blog entry, I want to think about that last option. Here's I imagine it would work: I would post a list of things I would like to do, estimates of how much work they'd be, and a hourly price. People would tell me how many hours of work they'd like to pay for. I would then do the work, send everyone their bill, and they'd pay it. In the ideal world, this would be an awesome way to fund development. I could, for example, write some automated tools for system testing of Debian, to help with release management, or I could finish Obnam. And that's just two things I'm already working on: there's no end of things I would like to develop, if I didn't need to worry about my next meal. However, this is not without problems. To start with, it might look like I'm trying to blackmail the world: "pay me, or this software won't be written". That's not a problem in the general case, but it'd easily look like that if the work would be to, say, maintain Debian packages. As a Debian developer, I should keep my packages in good shape whether someone is paying me or not. I would need to be careful to keep things ethical here. Another problem would be the issue of being realistic: would there be enough people willing to pay me, and how long would that last? That partly depends on the pricing. Although my needs are fairly modest, I do live in a country that is fairly expensive to live in. If I started doing this, it'd be easy for someone in a country it is cheaper to live in to compete on price (and they might well be better than I am, too). Then again, since there's no end of things to do, perhaps we could both make a living. Would you pay someone to develop free software by the hour? How much? How often? What should the software do?

Neil Williams: TDeb processing...

Just added a TDeb test build branch for multistrap to test out the changes in the next version of the emdebian-tdeb binary package - specifically:

Move em_installtdeb into the emdebian-grip-server package so that the emdebian-tdeb package can be slimmed down to only contain what will become useful for other maintainers to use in their debian/rules files and for translators when the time comes to implement DEP-4. CDBS rules / patches to follow... bug reports against dpkg and debhelper to follow ...

DEP-4 itself is being restarted and refreshed but with some problems on alioth currently, I'm going to (try and) keep the document up to date via SGML and people.debian.org.

The changes are now in Emdebian SVN and I'm starting testing with my own packages first. Multistrap for PO and po4a-build testing, dpkg-cross for debconf testing, some others for testing with build systems other than CDBS, then the discussions will restart on debian-devel and I'll upload the TDeb support scripts to unstable.

There are a variety of things which need to happen before these scripts become useful, including:

Why is the package called emdebian-tdeb? Simple - that's where the scripts have lived until now and the hope is that dpkg-gentdeb can migrate into dpkg-dev and dh_gentdeb into debhelper - eventually.

By all means play with the scripts and follow the example multistrap branch above to see what kind of changes might be needed in your own packages. I'll post again when I've worked out the changes needed in packages like dpkg-cross which only need TDebs for debconf support. (This is the easiest mode of TDeb usage but it's also the most useful during a release cycle - maintainers make a single change and then a nominated i18n boss-person - take a guess who - can update the all your debconf translations in one upload without a binary build - eventually possibly not even needing any bug reports, just a simple email advance warning.

What I've got to work on still is how the generation of an updated TDeb creates a new debian diff / debian.tar.gz to accompany the upload. For that I need XC-Package-Type: tdeb support in dpkg....

Sample output from the viewpoint of a translator updating a package already converted to support a TDeb:


neil@sylvester:tdeb-test$ dpkg-gentdeb -t
Set the changelog message to version 2.1.15+t0 first, closing any relevant bugs!
dch -v 2.1.15+t0
neil@sylvester:tdeb-test$ dch -v 2.1.15+t0 TDeb test
neil@sylvester:tdeb-test$ dpkg-gentdeb -t
Building TDeb update: 2.1.15+t0
(192 entries)
Discard doc/pod/1/da/multistrap (65 of 177 strings; only 36.72% translated; need 50%).
Discard doc/pod/1/da/device-table.pl (5 of 19 strings; only 26.31% translated; need 50%).
Processing untranslated files for pod/multistrap (1) . . .
Processing untranslated files for device-table.pl (1) . . .
Processing de translations for pod/multistrap (1). . .
Processing de translations for device-table.pl (1). . .
Processing fr translations for pod/multistrap (1). . .
Processing fr translations for device-table.pl (1). . .
Processing pt translations for pod/multistrap (1). . .
Processing pt translations for device-table.pl (1). . .
dpkg-gencontrol -pmultistrap-tdeb -Pdebian/multistrap-tdeb -cdebian/control
dpkg --build debian/multistrap-tdeb ../multistrap-tdeb_2.1.15+t0_all.tdeb
dpkg-deb: building package multistrap-tdeb' in ../multistrap-tdeb_2.1.15+t0_all.tdeb'.
neil@sylvester:tdeb-test$ dpkg -c ../multistrap-tdeb_2.1.15+t0_all.tdeb
drwxr-xr-x root/root 0 2011-06-08 21:06 ./
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/locale/
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/locale/pt/
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/locale/pt/LC_MESSAGES/
-rw-r--r-- root/root 6427 2011-06-08 21:06 ./usr/share/locale/pt/LC_MESSAGES/multistrap.mo
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/locale/vi/
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/locale/vi/LC_MESSAGES/
-rw-r--r-- root/root 6963 2011-06-08 21:06 ./usr/share/locale/vi/LC_MESSAGES/multistrap.mo
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/locale/fr/
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/locale/fr/LC_MESSAGES/
-rw-r--r-- root/root 6789 2011-06-08 21:06 ./usr/share/locale/fr/LC_MESSAGES/multistrap.mo
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/locale/da/
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/locale/da/LC_MESSAGES/
-rw-r--r-- root/root 16311 2011-06-08 21:06 ./usr/share/locale/da/LC_MESSAGES/multistrap.mo
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/man/
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/man/pt/
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/man/pt/man1/
-rw-r--r-- root/root 33963 2011-06-08 21:06 ./usr/share/man/pt/man1/multistrap.1
-rw-r--r-- root/root 4471 2011-06-08 21:06 ./usr/share/man/pt/man1/device-table.pl.1
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/man/vi/
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/man/fr/
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/man/fr/man1/
-rw-r--r-- root/root 33969 2011-06-08 21:06 ./usr/share/man/fr/man1/multistrap.1
-rw-r--r-- root/root 4708 2011-06-08 21:06 ./usr/share/man/fr/man1/device-table.pl.1
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/man/da/
drwxr-xr-x root/root 0 2011-06-08 21:06 ./usr/share/man/da/man1/


I'm also going to DebConf11



On the RoadTrip! all the way hrough France, Belgium, Netherlands, Germany, Austria, Slovenia, Croatia, Bosnia-Herzegovina (Banja Luka) in an MX5.

I've got a lot of things to talk to people about @ DebConf this year... you won't be able to tell if I'm just buying you a beer or trying to badger you about your packages!! Just note that TDebs have been already agreed (in terms of how TDebs work in the archive) with ftp-master at a previous meeting in Extremadura and I'm being badgered by a Release Manager and an ex-DPL to get TDebs implemented real soon now, so it is going to happen and hopefully in time for Wheezy - it's just a matter of fixing the bugs. The quicker everyone has a look at the new scripts (which have been completed re-written and immensely simplified since I first tried a dumb conversion of em_installtdeb) and the likely changes needed in packages, the quicker I can get the bugs out of the scripts.

More documentation to follow, I'll be updating the SGML whenever practical and the actual DEP just as soon as I get commit access back.

Matt Zimmerman: DEX finishes first batch of derivative patches for Debian

It s been a few months since Zack and I announced the DEX project, which aims to improve the Debian ecosystem by working jointly with derivative distributions. Our first milestone The goal of our first project, nicknamed ancient-patches, was to clear out an old batch of a few hundred Ubuntu patches whose status was unclear. We couldn t tell which ones had been merged into Debian, which were waiting in the BTS, and which had yet to be submitted to Debian. All of them were several years old. I m pleased to announce that this project is now complete. Thanks to help from David Paleino, Colin Watson, Nathan Handler and Steve Langasek, we were able to clear over 95% of the patches in a matter of days. These were the easy ones: patches which were obsolete, or had already been applied. We discussed the remainder, and resolved all of the patches whose status was still unclear. This left the harder ones: patches stalled in the BTS, and patches where there was no consensus about what to do with them. One of the stalled patches was merged into Debian via an NMU, eliminating the delta between Debian and Ubuntu. Another had been submitted to Debian by a third party, but was no longer shipping in Ubuntu, so we considered it obsolete for purposes of this project. This has left only two patches out of the original list of 277. Both of them are filed in the BTS and have been discussed with the relevant maintainer team. One of them is expected to be obsoleted when a new upstream version is packaged, which implements similar functionality. The other is being discussed with the upstream developers, but there is no conclusion yet about whether it can be merged upstream or in Debian. Conclusions Although we weren t quite able to clear the whole list, we still consider the project to be a success because: What s next In the most recent DEX update on debian-derivatives, I highlighted a few important events for DEX: We re looking forward to seeing DEX develop further. If you d like to get involved, come and join us on the debian-derivatives mailing list or IRC (#debian-derivatives on freenodeOFTC). Matt Zimmerman and Stefano Zacchiroli

Cosimo Alfarano: OpenNI in Debian

The world of augmented reality is quite fashinating.

I recently got interested into Natural Interaction due to the popularity of Microsoft's Kinect and decided to package OpenNI for Debian.

OpenNI is a framework aiming to making natural interaction transparent to the user/developer, publishing a unified API and abstracting the access to different sensors and modules. Currently it supports some Asus sensors and there is a branch on github supporting also Kinect (which is the one we're packaging).

Some cool projects are already using OpenNI: ROS, PointClouds and OpenCV come to my mind immediately.

We'are working on OpenNI package in the Multimedia Maintainers group on Alioth, the package is still not ready to be uploaded to unstable and upstream has some major problems like the lack of SONAME in its produced library, so some communication with upstream is needed but for preliminary use the packages are in a decent state: OpenNI, PrimeSense Sensor, NITE (non-free).

NITE is a closed source module set for advanced body and scene recognition, unfortunately PrimeSense didn't opensource them, thus I decided to use the same approach of flashplugin-nonfree and write a downloader.

For the sake of completeness, unstable already has libfreenect which purpose is similar but specifically oriented to Kinect.

I am studying both and really hope that in the next future some open source project will make (at least) skeleton and hand recognition available under an open license. People are already working on it, according to Google.

Martin F. Krafft: World IPv6 Day: ask your provider now

Today is World IPv6 Day. Please take a moment to test your connectivity, and if you are not IPv6-enabled yet, then send an e-mail to your provider or hoster and ask them for native IPv6 connectivity on your uplink. Do it even if you do not know what I am talking about or you don t care. The reason is quite simply that we re already too late and hence should act without further delay. If IPv6 network effects do not pick up and adoption rate increases, the big players will drive up the prices for everyone. Then you will find yourself locked in and paying. Or you simply won t be able to address individual computers anymore but always be forced to proxy via commercial providers and forced to say how high when they ask you to jump. Remember that they are commercial entities who might claim to act in the interest of their customers, but you are actually second to their profits. Here are some answers to frequently asked questions if you want to know more. PS: Google, having been so vocal about World IPv6 Day, I would have at least expected you to change your logo today! NP: Monkey3: 39 Laps

MJ Ray: Software in the Public Interest June 2011

Software in the Public Interest is one of the oldest and best non-profit associations (and not a foundation!) that supports free and open source software development and it will hold a public board of directors meeting on Wednesday at 20:30 UTC (21:30 UK). SPI meetings are held on irc.spi-inc.org (the OFTC network) in channel #spi. See the OFTC site for more information about connecting to OFTC. The agenda for the meeting is available on the web. A single resolution regarding ankur.org.in joining as an SPI associated project has been received. Please, come along and support SPI and thank its volunteers for their efforts.

7 June 2011

Vincent Bernat: Mobile browsing & content optimization

Since my laptop features WWAN connectivity, I purchased a 3G mobile broadband plan from SFR, a French mobile phone company. I took the cheapest one available: 250 MB per month for 10 .

Optimizing transparent proxy by SFR While browsing, I was curious about the fact that there were multiple accesses to non-public IP, like 10.141.0.4. I was browsing the mobile version of Le Monde, a French daily newspaper. I enabled Chromium developer tools and watched what happened in the Network tab. All images were served from IP like 10.141.0.4. Looking at the HTML source code, I discovered that <img> tags were rewritten to point to those images. The smaller one were identical to the original one but the larger ones (more than 50 KB) were of reduced quality (and therefore reduced size). <img> tags were not the only one to be rewritten. <style> and <script> tags that were pointing to external resources (with a src attribute) were also rewritten to include the CSS or the javascript inline. Before:
<script src="http://s1.lemde.fr/js/lib/jquery/1.5.1/jquery.js"></script>
After:
<script oldSrc="http://s1.lemde.fr/js/lib/jquery/1.5.1/jquery.js" NG="replaced">
/*!
 * jQuery JavaScript Library v1.5.1
 *
 * Copyright 2011, John Resig
 * Dual licensed under the MIT or GPL Version 2 licenses.
 * http://jquery.org/license
 * [...] */
</script>
This means that each page that includes jQuery, a popular javascript library, as an external resource, will be turned into a page that includes the whole jQuery code which is about 85 KB. Each page will weight more than 85 KB instead of relying on an external and cacheable resource. Nowadays, most browsers support HTTP keep-alive which allows to serve multiple resources into a single request. There are two cases:
  1. If the resource is served from the same server as the HTML page, keep-alive would enable to get almost the same result without hindering the cache: the browser would first request the HTML page, then the javascript resource into the same TCP connection. There is a small penalty of about 100 ms to let the browser request the second resource. The browser may decide to open another connection because the initial document is not loaded yet. Some browsers may even negotiate this additional connection before needing it. In this case, it can use this idle connection at no cost. Otherwise, we may fall in the second case.
  2. If the resource is served from another server (like in the example above) or if the browser does not have an idle connection ready for reuse, it would download the resource in parallel with the help of another connection. It will only costs about 300 ms of additional time due to the establishment of a new TCP connection with the server. Those 300 ms are a very small price to pay compared to the 2 seconds needed to download 85 KB for each page. Moreover, other static resources will likely be downloaded from the same server and therefore, this is 300 ms the first time and about 100 ms for additional resources (round-trip time for a new HTTP request into the same TCP connection).
UPDATED: In a previous version of this article, I confused pipelining and keep-alive mechanisms. While keep-alive allows to reuse an existing connection to download a new resource, only one request at a time by connection could be issued. Pipelining allows to push additional requests in the same TCP connection without waiting for answers. As noted by ric Daspet in a comment, pipelining is disabled for most browsers. However, keep-alive is well supported. Searching on the web for oldSrc and NG="replaced", I find very little information about what is used. It seems that such a feature may also break some scripts, like TinyMCE, trying to get additional resources relative to the path where the script is installed. It appears that SFR is using a product from Flash Networks called Harmony Gateway. I have asked SFR more info about this and if it was possible to disable the use of this feature but got no answer so far. I have retried a bit later and there was no rewriting for Javascript and CSS. Images were still degraded and served by a special server. The next day, my requests did not travel through a transparent proxy anymore (no more Via header in answers). Each try was done at a different location. The proxies may be installed at some locations only and configured differently. Another explanation is that SFR silently disabled this functionality for my account. If it was done correctly, such an accelerator service could be be proposed as an opt-out option since most users would appreciate faster page downloads. If you care about network neutrality, you can disable this option. Of course, it needs to be properly advertised, for example as a free option in your mobile plan.

Setting up our own optimizing proxy Google provides mod_pagespeed as a solution to optimize a website on the fly. This is an Apache module. It is targeted at optimizing your own website but since Apache can act as a regular proxy, it seems possible to enable mod_pagespeed for any website. That s pretty easy to setup if you don t mind using a precompiled version of mod_pagespeed. Download mod_pagespeed. I have downloaded the 64bit version for Debian.
# touch /etc/default/mod-pagespeed
# dpkg -i ~/download/mod-pagespeed-beta_current_amd64.deb
Selecting previously deselected package mod-pagespeed-beta.
(Reading database ... 277699 files and directories currently installed.)
Unpacking mod-pagespeed-beta (from .../mod-pagespeed-beta_current_amd64.deb) ...
dpkg: dependency problems prevent configuration of mod-pagespeed-beta:
 mod-pagespeed-beta depends on apache2.2-common; however:
  Package apache2.2-common is not installed.
dpkg: error processing mod-pagespeed-beta (--install):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 mod-pagespeed-beta
# apt-get install -f
[...]
Unpacking apache2.2-common (from .../apache2.2-common_2.2.19-1_amd64.deb) ...
Processing triggers for man-db ...
Setting up apache2.2-common (2.2.19-1) ...
[...]
# apt-get install apache2-mpm-worker
[...]
Unpacking apache2-mpm-worker (from .../apache2-mpm-worker_2.2.19-1_amd64.deb) ...
Setting up apache2-mpm-worker (2.2.19-1) ...
Starting web server: apache2.
# a2enmod proxy_http
Considering dependency proxy for proxy_http:
Enabling module proxy.
Enabling module proxy_http.
To to activate the new configuration, you need to run:
  /etc/init.d/apache2 restart
Then, we modify /etc/apache2/mods-enabled/proxy.conf:
ProxyRequests On
<Proxy *>
    AddDefaultCharset off
    Order deny,allow
    Deny from all
    Allow from 127.0.0.1
</Proxy>
Of course, this makes little sense to run the proxy on your laptop since you don t want to download the non-optimized content over your WWAN link. You need to run it on another server and therefore allow your laptop to access it. Also, we need to modify /etc/apache2/mods-enabled/pagespeed.conf to add the directive ModPagespeedDomain * to let mod_pagespeed rewrite any page. You should restart Apache after that. Now, check if the proxy is working:
$ curl -six http://localhost:80 http://www.lemonde.fr/   grep ^X-Mod-Pagespeed
X-Mod-Pagespeed: 0.9.17.7-716
All is working! Let s try a very simple site, like Debian Backports. It features one HTML file and two CSS files: Debian Backports website without pagespeed We downloaded a total of 8 KB (compression included). Let s use our proxy and try again: Debian Backports website with pagespeed mod_pagespeed succeeds in reducing the size of the HTML file and combined the CSS files. Moreover, it minified the resulting CSS file. There seems to be a bug in Chromium that display incorrect transfert sizes (including in the timeline representation) when using a proxy. I have checked with Wireshark that the overall quantity transferred was about 8 KB. The gain is very thin here but you can imagine that you will get better results on larger sites. mod_pagespeed seems to handle well most sites. However, it only optimizes pages that already are in its cache. The first time you grab a page, you will get the unoptimized version. I don t know if it is the expected behaviour.

Vincent Fourmond: Record sound currently playing to file using ALSA

I've been wondering for a long time about how to record sound which is currently being played on the computer. I know the ALSA framework is powerful enough to do that, and today just proved me so (at least to some extent). It turns out that it's pretty trivial using the file PCM plugin; all you need is a stanza like this in your $HOME/.asoundrc file:
pcm.save  
    type file
    format wav
    slave.pcm front 
    # my main PCM is called front
    file output.wav
 
Afterwards, you can use this save PCM with the tools that understand it, such as aplay or mplayer:
~ aplay -Dsave stuff.wav
~ mplayer -ao alsa:device=save stuff.wav
(for xine users, you may want to have a look at the audio.device.alsa_front_device configuration element). Funnier, though, is the possibility to use it with programs that don't provide such an easy way to change that, (say, the flash player in a browser ?) via the use of the default PCM specification in $HOME/.asoundrc:
pcm.!default save
After that, all programs using the default PCM will save the sound they play as a save.wav file in their current directory - but nothing stops you from specifiying an absolute directory. Better yet, you may apparently use pipes in stead of the file name, la popen.

Lars Wirzenius: TDDD systest tool proof of concept

Continuing my earlier musings about test driven distro development, and the tools that would require. I imagine something like this: We might also want this scenario: There might be other scenarios that would be useful to test as well. Even from these two, it's clear there's a need for at least three separate tools: I've written a rudimentary version of the first tool: vmdebootstrap. I've since learned there's a bunch of others that might also work. There's room for more: someone should write (or find) a tool to make a snapshot of a real system and create a VM image that mimicks it, for example. Anyway, for now, I'll assume that one of the existing tools is good enough to get started. For the second tool, I wrote a quick-and-dirty proof-of-concept thing, see systest.py. Here's a sample of how it might be used:
liw@havelock$ ./systest -v --target 192.168.122.139 --user tomjon
test 1/6: cat
test 2/6: only-ssh-port
ERROR: Assertion failed: ['22/tcp', '139/tcp', '445/tcp'] != ['22/tcp']
[status 1]
liw@havelock$ ./systest -v --target 192.168.122.139 --user tomjon \
    cat ping6-localhost ping-localhost simple-dns-lookup ssh-login
test 1/5: cat
test 2/5: ping6-localhost
test 3/5: ping-localhost
test 4/5: simple-dns-lookup
test 5/5: ssh-login
liw@havelock$ 
The first run failed, because the VM I'm testing against has some extra ports open. Some of the tests will require logging into the machine, via ssh, and for that one needs to specify the user to use. systool may overlap heavily on system monitoring tools, and possibly the production implementation should be based on those. I think it's best to design such a tool for the more general purpose of testing whether a system currently works rather than as an integrated part of a more specific larger tool. This lets the tool be useful for other things than just testing specific things about Debian. (The production implementation would then need to not have all the tests hardcoded, of course. SMOP.) The third tool I have not spent a lot of time on yet. One thing at a time. Given these tools, one would then need to decide how to use them. The easiest way would be to use them like lintian and piuparts: run them frequently on whatever packages happen to be in testing or unstable or experimental, and put links to the test reports to the PTS, and hope that people fix things. That is the easiest way to start things. Once there's a nice set of test cases and scenarios, it may be interesting to think about more aggressive ways: for example, preventing packages from migrating to testing unless the test suite passes with them. If the tests do not pass, one of four things is broken: If things are set up properly, the last one should be rare. The other three always require manual inspection: it is not possible to automatically know whether the test itself, or the code it tests, is at fault. it is, however, enough to know that something is wrong. If the tests are written well, they should be robust enough to not be the culprits very often. (Someone wanting to make a rolling distrbution, or even better, making a monthly mini-release, might benefit from this sort of automated testing.)

DebConf team: Small update about Travel Sponsorship for DebConf11 (Posted by Travel Sponsorship team)

As usual many people requested travel sponsorship to attend DebConf 11 and are now waiting eagerly on the outcome of their request. We are sorry, that we can t give you much details yet. To give you an idea of the time line, here is a small explanation of the process. About 100 people requested travel sponsorship. In total that would about 50 000 Euro. Problem is currently we don t exactly know, how much money we may spend on travel sponsorship ourselves (By the way: If a potential sponsor is reading this, please contact us; DebConf also accepts donations). So, we formed a team of 13 persons (the mail only lists twelve, one was added later). The team members will start to rate each and every single request in two criteria: Contributor Level and sponsorship amount. With limited amounts we obviously want to sponsor those first, who contribute a lot to Debian, preferably if they don t travel first class. After that s done, we ll have a long, but ordered list, but the team still needs to meet and discuss different cases and similar. Our team isn t really small and consists of people living all over the world, it s quite hard to actually schedule a meeting everyone can attend. (Which is not a bug, but a feature: More people from more countries should in the end result in a fair rating of the sponsorship requests.) So, in the end, we ll meet on 19th of June. Sounds late, please keep in mind, that every team member has to rate all sponsorship requests (including some research) and all of them are volunteers with actual daytime jobs. But even if you don t hear anything from us short after that date, don t panic! As said we really don t know, yet, how much money we can spend on travel sponsorship. Our sponsor-hunting-team is constantly working on the acquisition of more money, so every now and then the budget for travel sponsorship gets increased, meaning that we can mail a couple of more people on our ordered list. So it might take a while, till we get to all, but we try very hard to do so.

6 June 2011

Sam Dunne: Declarative Diversions - Design Spec 2

DECLARATIVE DIVERSIONS ====================== Introduction A diversion is when it is possible to have dpkg not overwrite a file when it reinstalls the package it belongs to, and to have it put the file from the package somewhere else instead. Declarative diversions involves a new control file with a declarative syntax which dpkg will parse and process directly as part of the package unpack and removal phases, eliminating the problems resulting from non-atomic handling of diversions. Topics There are a number of topics involved in implementing this kind of project * What syntax do we use for the new control file? * What dpkg does with the control file * How do we handle diversions to a non-existant directory? * How do we order unpacking a new package which adds a diversion? * How do we order unpacking a new package that removes a diversion? * How do we order removing a package which had a diversion? * How do we handle errors? * What happens to dpkg-divert? Details - Control File Syntax - It will conform to RFC2822 style with the following format: * Divert-From: * Divert-To: * Blank lines and lines beginning with # will be comments Divert-To will be optional and if it is ommitted then files being diverted will have their filename changed to file.distrib The above style has it s advantages, one of the main ones being that there is no need to escape whitespace within filenames. It also allows for more commands to be added at a later date. Details - Control File Handling - Within control.tar.gz the file should be named diversions This file is then copied to /var/lib/info/$package.diversions We need to store the control file for the currently-installed package so that we can detect declarative diversions that have been removed from the control file between versions (On-disk version and the to-be-unpacked version). If there is no previous version of a control file then all current diversions within the package are left in place and any new ones declared within the control file are added. If there is a previous version of the control file then the diversions listed within the file are compared with the diversions currently in place. If a diversion exists that is no longer listed in the control file then it is removed. If there is a diversion listed in the control file that doesn t currently exist then it is added. Details - Handling Diversions to non-existant directories - Diverting files to directories that don t exist can cause a number of problems. If the package does not own the directory it may be left orphaned on removal of the package The package is responsible for ensuring the availability of the target directory in the unpack phase. If a declarative diversion attempts to divert a file to a folder that neither exists already on the filesystem at unpack time, nor is shipped within the package, dpkg will return an error at unpack time. Details - Ordering Requirements - =>Unpacking a new package that adds a diversion *1. Check if package has a previous diversion control file *2. Copy control file to tmp directory *3. Calculate diversions to be added *4. Unpack folders *5. Add diversions *6. Unpack files *7. Copy control file to /var/lib/dpkg/info/$package.diversion *Performing diversions this way avoids trouble with creating folders and accidentally overwriting the wrong version of files. *Another major problem would be extracting a new version of the file and overwriting the old one. *This might cause the wrong version of the file to be diverted and the package to break. =>Unpacking a new package that removes a diversion *1. Check if package has a previous diversion control file *2. Copy control file to tmp directory *3. Calculate diversions to be removed *4. Check if dropped diversion results in a file conflict and abort the unpack if it does. *5. Unpack folders *6. Unpack files *7. Remove old files no longer needed in the package *8. Remove diversions *9. Copy control file to /var/lib/dpkg/info/$package.diversion *If a package has no previous diversion control file then no diversions will be removed and existing diversions can only be removed using dpkg-divert. *If removing a diversion results in a file conflict then dpkg will return an error at unpack time before anything is unpacked and the unpack is aborted. =>Removing a package which had a diversion *1. Remove files *2. Remove diversions *3. Remove folders *4. Remove control file at /var/lib/dpkg/info/$package.diversion *This ensures that all files, diversions and folders are removed correctly Details - Error handling Errors in diversions will have to handled with a great deal of care due to the fact that if they are not the package could be broken. This means that a great deal of checks must be done to ensure that all the files can be diverted properly before any actual diverting takes place. If they can t the package installation/update must be stopped and rolled back to avoid the package being installed incorrectly or broken. Details - dpkg-divert - When we impliment the new diversion method we should keep the current dpkg-divert. This allows maintainers to catch up with the new method without breaking their packages. It also allows maintainers to perform some operations that aren t support by the new method. All current functionality of dpkg-divert will be left intact for use by sysadmins and for maintainers to catch up with the new system. The two systems will work together without directly conflicting with each other. Example Usage #1 The file to be diverted is /usr/share/foo It needs to be moved to /usr/share/bar The syntax of the control file would be: #Start File Divert-From: /usr/share/foo Divert-To: /usr/share/bar #End File Example Usage #2 In this example the maintainer doesn t want to move the file to any specific folder The syntax of the control file would be: #Start File Divert-From: /usr/share/foo #End File This would divert the file to /usr/share/foo.distrib Footnotes - RFC2822 Guide: * http://www.faqs.org/rfcs/rfc2822.html First Email Thread on Declarative Diversions (First Message in Thread): * http://lists.debian.org/debian-dpkg/2011/05/msg00102.html Email thread on first design draft (First message in Thread): * http://lists.debian.org/debian-dpkg/2011/06/msg00019.html Declarative Diversions Wiki: * http://wiki.debian.org/SummerOfCode2011/DeclarativeDiversions My Blog for this Project: * http://blog.sam-dunne.com

Thorsten Glaser: synchronise environment between two shells

(First posting to Pl net Commandline! Tag: pcli) Vutral asked in IRC how to synchronise two shells environment while they re running. As you may know, POSIX systems cannot change a process environment vector after it has been started, only the process itself can. Well, the shell can, and we ll use a variety of things for this. This trick assumes you have $HISTFILE set to the same pathname in both shells (obviously, they run under the same user). It uses export -p to render the current list of exported variables, then transforms the list from newline-separated to a single big one-line export statement.
Then it transforms all remaining newlines (which will be part of a single-quoted string, since that s mksh(1) s export format) into the sequence '$'\n'' which means: terminate current single-quoted string, append $'\n' and open up a new single-quoted string immediately; concatenate these three.
Now, $'\n' is just a fancy way of saying newline, and part of mksh because David Korn (yes, the Korn in Korn Shell) strongly suggested to me that this functionality be included but, as we can see here, it pays off.
Finally, the so transformed string is prepended by unset \$(export); which, when executed, will cause the shell to unset (and unexport) all currently exported variables. The shell parameters that are not exported, i.e. not in the environment, are not affected by this code (except for $x and $nl, but whatever).
This string is then passed to read -s (plus -r and clearing IFS to enable raw mode), which means, read into the parameter $REPLY (which we conveniently don t use but it s trashed too, thus) but store into history at the same time. Ah hah! Now, the persistent history feature comes into effect! After running the below statement in the source shell, switch into the terminal running the destination shell, press Enter once on the empty line (Ctrl-U to empty it if it wasn t), then Cursor-Up ( ) to recall voil , an insanely large line with the previously created string sorta expanded and press Enter again to run it. Now your set of exported parameters is the exact same (minus if you exported IFS, nl, x or REPLY) as in the source shell.

I ve added extra spaces and a linewrap below, this is really just one big line:

nl=$'\n'; x=$(export -p); x=$ x//$ nl export/ ; IFS= read -rs <<<"unset \\\$(export);$ x//$nl/\'\$\'\\\\n\'\' "

Of course, this makes a nice function, for your ~/.mkshrc or somesuch.

Steve Kemp: So I chose fabric and reported a bug..

When soliciting for opinions, recently, I discovered that the python-based fabric tool was not dead, and was in fact perfect for my needs. During the process of getting acquainted with it I looked over the source code, it was mostly neat but there was a trivial (low-risk) symlink attack present. I reported that as #629003 & it is now identified more globally as CVE-2011-2185. I guess this goes to show that getting into the habit of looking over source code when you install a new package is a worthwhile thing to do; and probably easier than organising a distribution-wide security audit </irony>. In other news I'm struggling to diagnose a perl segfault, when running a search using the swish-a perl modules. Could it be security worthy? Possibly. Right now I just don't want my scripts to die when I attempt to search 20Gb of syslog data. Meh. ObQuote: "You're scared of mice and spiders, but oh-so-much greater is your fear that one day the two species will cross-breed to form an all-powerful race of mice-spiders who will immobilize human beings in giant webs in order to steal cheese. " - Spaced.

Next.

Previous.