How can you help free software?
Aside from donating to the excellent free software nonprofits out there, and contributing to a project by building software or other resources, there are things you can do to help the free software cause. The two biggest things I think are providing mentorship and gently normalizing free software.
Providing Mentorship
Allison Randal introduced me to the idea that mentorship doesn t have to be an ongoing process. This is to say, you don t have to sign up to be someone s best friend and advisor for life (though you certainly can). Providing short term, project or skill based, or one off mentorship is useful for building community because it makes people feel welcome and cared for and helps build skills that benefit free software.
I do a lot of proofreading and editing of people s writing especially people with minimal writing experience and/or non-native English speakers who are writing important documents in English. If the person is interested, I try to talk to them about their writing and why I m making these particular suggestions. I hope this helps them with their writing in the future.
Other examples are working on a particular project or skill, this can be helping them develop a particular skill (e.g. git outside of the command line), or giving advice on a project with a level of specificity and detail you re both comfortable with. These can, again, be one off things or things that require minimal effort/occasional conversation. I have some friends who I consider my Debian mentors who just answer functional questions whenever I have trouble doing something.
I also love love love talking with people about their free software trajectories, their goals and desires and dreams for their involvement in free software, whether that s finding a place in a community, developing a skill set, or other things about their future (like job hopes, schooling, etc). These conversations have been so helpful for me personally, and I like to think they help others.
Gently Normalizing Free Software
I think normalizing free software is very important to its success and adoption. It s not helpful to insist someone who has never done so before to create a Debian boot disk and install it. It is helpful to suggest using Big Blue Button or jitsi. If a friend wants help finding audio editing software, suggest they try audacity. I d go as far as to suggest doing this without explaining that it s free software, and instead focus on why it ll work and that it s available at no cost. If they like it, then it s a great time to talk about rights and freedoms. Of course if they already care about these sorts of things, if you re discussing privacy software, if anti-surveillance is an issue, or any number of other things, software freedom is a great thing to bring up!
Above all, just be nice.
Be nice. It s basically the best thing you can do for free software.
As some of the world knows full well by now, I've been noodling with Go
for a few years, working through its pros, its cons, and thinking a lot
about how humans use code to express thoughts and ideas. Go's got a lot of
neat use cases, suited to particular problems, and used in the right place,
you can see some clear massive wins.
I've started writing Debian tooling in Go, because it's a pretty natural fit.
Go's fairly tight, and overhead shouldn't be taken up by your operating system.
After a while, I wound up hitting the usual blockers, and started to build up
abstractions. They became pretty darn useful, so, this blog post is announcing
(a still incomplete, year old and perhaps API changing) Debian package for Go.
The Go importable name is pault.ag/go/debian. This contains a lot of utilities
for dealing with Debian packages, and will become an edited down "toolbelt"
for working with or on Debian packages.
Module Overview
Currently, the package contains 4 major sub packages. They're a changelog
parser, a control file parser, deb file format parser, dependency parser
and a version parser. Together, these are a set of powerful building blocks
which can be used together to create higher order systems with reliable
understandings of the world.
changelog
The first (and perhaps most incomplete and least tested) is a changelog file
parser.. This provides the
programmer with the ability to pull out the suite being targeted in the
changelog, when each upload was, and the version for each. For example, let's
look at how we can pull when all the uploads of Docker to sid took place:
funcmain()resp,err:=http.Get("http://metadata.ftp-master.debian.org/changelogs/main/d/docker.io/unstable_changelog")iferr!=nilpanic(err)allEntries,err:=changelog.Parse(resp.Body)iferr!=nilpanic(err)for_,entry:=rangeallEntriesfmt.Printf("Version %s was uploaded on %s\n",entry.Version,entry.When)
The output of which looks like:
Version 1.8.3~ds1-2 was uploaded on 2015-11-04 00:09:02 -0800 -0800
Version 1.8.3~ds1-1 was uploaded on 2015-10-29 19:40:51 -0700 -0700
Version 1.8.2~ds1-2 was uploaded on 2015-10-29 07:23:10 -0700 -0700
Version 1.8.2~ds1-1 was uploaded on 2015-10-28 14:21:00 -0700 -0700
Version 1.7.1~dfsg1-1 was uploaded on 2015-08-26 10:13:48 -0700 -0700
Version 1.6.2~dfsg1-2 was uploaded on 2015-07-01 07:45:19 -0600 -0600
Version 1.6.2~dfsg1-1 was uploaded on 2015-05-21 00:47:43 -0600 -0600
Version 1.6.1+dfsg1-2 was uploaded on 2015-05-10 13:02:54 -0400 EDT
Version 1.6.1+dfsg1-1 was uploaded on 2015-05-08 17:57:10 -0600 -0600
Version 1.6.0+dfsg1-1 was uploaded on 2015-05-05 15:10:49 -0600 -0600
Version 1.6.0+dfsg1-1~exp1 was uploaded on 2015-04-16 18:00:21 -0600 -0600
Version 1.6.0~rc7~dfsg1-1~exp1 was uploaded on 2015-04-15 19:35:46 -0600 -0600
Version 1.6.0~rc4~dfsg1-1 was uploaded on 2015-04-06 17:11:33 -0600 -0600
Version 1.5.0~dfsg1-1 was uploaded on 2015-03-10 22:58:49 -0600 -0600
Version 1.3.3~dfsg1-2 was uploaded on 2015-01-03 00:11:47 -0700 -0700
Version 1.3.3~dfsg1-1 was uploaded on 2014-12-18 21:54:12 -0700 -0700
Version 1.3.2~dfsg1-1 was uploaded on 2014-11-24 19:14:28 -0500 EST
Version 1.3.1~dfsg1-2 was uploaded on 2014-11-07 13:11:34 -0700 -0700
Version 1.3.1~dfsg1-1 was uploaded on 2014-11-03 08:26:29 -0700 -0700
Version 1.3.0~dfsg1-1 was uploaded on 2014-10-17 00:56:07 -0600 -0600
Version 1.2.0~dfsg1-2 was uploaded on 2014-10-09 00:08:11 +0000 +0000
Version 1.2.0~dfsg1-1 was uploaded on 2014-09-13 11:43:17 -0600 -0600
Version 1.0.0~dfsg1-1 was uploaded on 2014-06-13 21:04:53 -0400 EDT
Version 0.11.1~dfsg1-1 was uploaded on 2014-05-09 17:30:45 -0400 EDT
Version 0.9.1~dfsg1-2 was uploaded on 2014-04-08 23:19:08 -0400 EDT
Version 0.9.1~dfsg1-1 was uploaded on 2014-04-03 21:38:30 -0400 EDT
Version 0.9.0+dfsg1-1 was uploaded on 2014-03-11 22:24:31 -0400 EDT
Version 0.8.1+dfsg1-1 was uploaded on 2014-02-25 20:56:31 -0500 EST
Version 0.8.0+dfsg1-2 was uploaded on 2014-02-15 17:51:58 -0500 EST
Version 0.8.0+dfsg1-1 was uploaded on 2014-02-10 20:41:10 -0500 EST
Version 0.7.6+dfsg1-1 was uploaded on 2014-01-22 22:50:47 -0500 EST
Version 0.7.1+dfsg1-1 was uploaded on 2014-01-15 20:22:34 -0500 EST
Version 0.6.7+dfsg1-3 was uploaded on 2014-01-09 20:10:20 -0500 EST
Version 0.6.7+dfsg1-2 was uploaded on 2014-01-08 19:14:02 -0500 EST
Version 0.6.7+dfsg1-1 was uploaded on 2014-01-07 21:06:10 -0500 EST
control
Next is one of the most complex, and one of the oldest parts of go-debian,
which is the control file parser
(otherwise sometimes known as deb822). This module was inspired by the way
that the json module works in Go, allowing for files to be defined in code
with a struct. This tends to be a bit more declarative, but also winds up
putting logic into struct tags, which can be a nasty anti-pattern if used too
much.
The first primitive in this module is the concept of a Paragraph, a struct
containing two values, the order of keys seen, and a map of string to string.
All higher order functions dealing with control files will go through this
type, which is a helpful interchange format to be aware of. All parsing of
meaning from the Control file happens when the Paragraph is unpacked into
a struct using reflection.
The idea behind this strategy that you define your struct, and let the Control
parser handle unpacking the data from the IO into your container, letting you
maintain type safety, since you never have to read and cast, the conversion
will handle this, and return an Unmarshaling error in the event of failure.
Additionally, Structs that define an anonymous member of control.Paragraph
will have the raw Paragraph struct of the underlying file, allowing the
programmer to handle dynamic tags (such as X-Foo), or at least, letting
them survive the round-trip through go.
The default decoder
contains an argument, the ability to verify the input control file using an
OpenPGP keyring, which is exposed to the programmer through the
(*Decoder).Signer() function. If the passed argument is nil, it will not
check the input file signature (at all!), and if it has been passed, any
signed data must be found or an error will fall out of the NewDecoder call.
On the way out, the opposite happens, where the struct is introspected,
turned into a control.Paragraph, and then written out to the io.Writer.
Here's a quick (and VERY dirty) example showing the basics of reading and
writing Debian Control files with go-debian.
packagemainimport("fmt""io""net/http""strings""pault.ag/go/debian/control")typeAllowedPackagestructPackagestringFingerprintstringfunc(a*AllowedPackage)UnmarshalControl(instring)errorin=strings.TrimSpace(in)chunks:=strings.SplitN(in," ",2)iflen(chunks)!=2returnfmt.Errorf("Syntax sucks: '%s'",in)a.Package=chunks[0]a.Fingerprint=chunks[1][1:len(chunks[1])-1]returnniltypeDMUAstructFingerprintstringUidstringAllowedPackages[]AllowedPackage control:"Allow" delim:"," funcmain()resp,err:=http.Get("http://metadata.ftp-master.debian.org/dm.txt")iferr!=nilpanic(err)decoder,err:=control.NewDecoder(resp.Body,nil)iferr!=nilpanic(err)fordmua:=DMUAiferr:=decoder.Decode(&dmua);err!=niliferr==io.EOFbreakpanic(err)fmt.Printf("The DM %s is allowed to upload:\n",dmua.Uid)for_,allowedPackage:=rangedmua.AllowedPackagesfmt.Printf(" %s [granted by %s]\n",allowedPackage.Package,allowedPackage.Fingerprint)
Output (truncated!) looks a bit like:
...
The DM Allison Randal <allison@lohutok.net> is allowed to upload:
parrot [granted by A4F455C3414B10563FCC9244AFA51BD6CDE573CB]
...
The DM Benjamin Barenblat <bbaren@mit.edu> is allowed to upload:
boogie [granted by 3224C4469D7DF8F3D6F41A02BBC756DDBE595F6B]
dafny [granted by 3224C4469D7DF8F3D6F41A02BBC756DDBE595F6B]
transmission-remote-gtk [granted by 3224C4469D7DF8F3D6F41A02BBC756DDBE595F6B]
urweb [granted by 3224C4469D7DF8F3D6F41A02BBC756DDBE595F6B]
...
The DM <aelmahmoudy@sabily.org> is allowed to upload:
covered [granted by 41352A3B4726ACC590940097F0A98A4C4CD6E3D2]
dico [granted by 6ADD5093AC6D1072C9129000B1CCD97290267086]
drawtiming [granted by 41352A3B4726ACC590940097F0A98A4C4CD6E3D2]
fonts-hosny-amiri [granted by BD838A2BAAF9E3408BD9646833BE1A0A8C2ED8FF]
...
...
deb
Next up, we've got the deb module. This contains code to handle reading
Debian 2.0 .deb files. It contains a wrapper that will parse the control
member, and provide the data member through the
archive/tar interface.
Here's an example of how to read a .deb file, access some metadata, and
iterate over the tar archive, and print the filenames of each of the
entries.
dependency
The dependency package provides an interface to parse and compute
dependencies. This package is a bit odd in that, well, there's no other
library that does this. The issue is that there are actually two different
parsers that compute our Dependency lines, one in Perl (as part of dpkg-dev)
and another in C (in dpkg).
To date, this has resulted in me filing
threedifferentbugs.
I also found a broken package in the
archive,
which actually resulted in another bug being (totally accidentally)
already fixed.
I hope to continue to run the archive through my parser in hopes of finding
more bugs! This package is a bit complex, but it basically just returns what
amounts to be an AST
for our Dependency lines. I'm positive there are bugs, so file them!
funcmain()dep,err:=dependency.Parse("foo bar, baz, foobar [amd64] bazfoo [!sparc], fnord:armhf [gnu-linux-sparc]")iferr!=nilpanic(err)anySparc,err:=dependency.ParseArch("sparc")iferr!=nilpanic(err)for_,possi:=rangedep.GetPossibilities(*anySparc)fmt.Printf("%s (%s)\n",possi.Name,possi.Arch)
Gives the output:
foo (<nil>)
baz (<nil>)
fnord (armhf)
version
Right off the bat, I'd like to thank
Michael Stapelberg for letting me graft this
out of dcs and into the go-debian package.
This was nearly entirely his work (with a one or two line function I added
later), and was amazingly helpful to have. Thank you!
This module implements Debian version comparisons and parsing, allowing for
sorting in lists, checking to see if it's native or not, and letting the
programmer to implement smart(er!) logic based on upstream (or Debian)
version numbers.
This module is extremely easy to use and very straightforward, and not worth
writing an example for.
Final thoughts
This is more of a "Yeah, OK, this has been useful enough to me at this point
that I'm going to support this" rather than a "It's stable!" or even
"It's alive!" post. Hopefully folks can report bugs and help iterate on
this module until we have some really clean building blocks to build
solid higher level systems on top of. Being able to have multiple libraries
interoperate by relying on go-debian will be a massive ease.
I'm in need of more documentation, and to finalize some parts of the older
sub package APIs, but I'm hoping to be at a "1.0" real soon now.
DebConf15 Schedule
The DebConf content team is pleased to announce the schedule of
DebConf15, the forthcoming Debian Developers Conference. From a total of
nearly 100 talk submissions, the team selected 75 talks. Due to the high
number of submissions, several talks had to be shortened to 20 minute
slots, of which a total of 30 talks have made it to the schedule.
In addition, around 50 meetings and discussions (BoFs) have been
submitted so far, as well as several other events like lightning talk
sessions, live demos, a movie screening, a poetry night or stand-up
comedy.
The Schedule is available
online at the DebConf15 conference site.
Further changes to the schedule can and will be made, but today s
announcement represents the first stable version.
Featured Speakers
In addition to the previously
announced invited speakers, the content team also announces the following
list of additional featured speakers:
Long story short, we put in a bid to host Debconf 16 in Cape Town, and we got it!
Back at Debconf 12 (Nicaragua), many people asked me when we re hosting a Debconf in South Africa. I just laughed and said Who knows, maybe some day . During the conference I talked to Stefano Rivera (tumbleweed) who said that many people asked him too. We came to the conclusion that we d both really really want to do it but just didn t have enough time at that stage. I wanted to get to a point where I could take 6 months off for it and suggested that we prepare a bid for 2019. Stefano thought that this was quite funny, I think at some point we managed to get that estimate down to 2017-2018.
That date crept back even more with great people like Allison Randal and Bernelle Verster joining our team, along with other locals Graham Inggs, Raoul Snyman, Adrianna Pi ska, Nigel Kukard, Simon Cross, Marc Welz, Neill Muller, Jan Groenewald, and our international mentors such as Nattie Mayer-Hutchings, Martin Krafft and Hannes von Haugwitz. Now, we re having that Debconf next year. It s almost hard to believe, not sure how I ll sleep tonight, we ve waited so long for this and we ve got a mountain of work ahead of us, but we ve got a strong team and I think Debconf 2016 attendees are in for a treat!
Since I happened to live close to Montr al back in 2012, I supported the idea of a Debconf bid for Montr al first, and then for Cape Town afterwards. Little did I know then that the two cities would be the only two cities bidding against each other 3 years later. I think both cities are superb locations to host a Debconf, and I m supporting Montr al s bid for 2017.
Want to get involved? We have a mailing list and IRC channel: #debconf16-capetown on oftc. Thanks again for all the great support from everyone involved so far!
Welcome to Portland, the City of Roses! You may find it helpful to grab a copy
of the Campus Map. A few key
locations:
Main conference venue is (for registration, sessions, and daytime hacklabs)
on the 3rd floor of the Smith Memorial Student Union building at SW Broadway &
SW Harrison.
Dorms are The Broadway (SW 6th and SW Jackson) and Ondine (SW 6th and SW
College). Check in at The Broadway on the 2nd floor (Suite 210).
Night hacklabs are on the second floors of both Broadway and Ondine
residence halls.
Sponsored meals are served in the Ondine cafe at SW 6th and SW
College.
Show your badge at Ol Latte Coffee (food cart) on SW 5th & SW Harrison, to
get free drop coffee, espresso, and loose leaf tea, 10% off other drinks
and food, and an additional $1 off coffee drinks for the first 200 people every
day. Open Weekdays 9-12 and 3-4, and Sat-Sun 9-1:30.
There will be yoga at on the 2nd floor of The Broadway residence at 7am on
the 25th & 25th, and 28th & 29th. Sign up in advance on
the wiki.
When I last lived in Seattle, nearly a decade ago, I hosted the Debian Seattle Social email list. When I left the city, the mailing list eventually fell victim to bitrot.
When Allison Randall asked me about the list a couple months ago, I decided that moving back to Seattle was a good excuse to work with Allison and some others to revive the community. Toward that end, I ve put up a little website and created a new mailing list. It s hosted on Alioth this time which will be reliable than me. Since it has been years, we have not moved over the old subscriber list so you ll have to sign up again if you were on it before.
If you re a Debian developer or user and you d like to hear about infrequent Debian social gatherings in the Seattle area, you should sign up on the list!
The Ubuntu Technical Board conducts a regular review of the most popular Ubuntu Brainstorm ideas (previous reviews conducted by Matt Zimmerman and Martin Pitt). This time it was my turn. Apologies for the late arrival of this review.
Contact lens in the Unity Dash (#27584)
Unity supports Lenses, which provide a consistent way for users to quickly search for information via the Dash. Current lenses include Applications, Files, and Music, but a number of people have asked for contacts to be accessible using the same interface.
While Canonical's DX team isn't currently working on this for Ubuntu 11.10 or 12.04, we'd love somebody who's interested in this to get involved. Allison Randal explains how to get started, including some skeleton example code and several useful links.
Displaying Ubuntu version information (#27460)
Several people have asked for it to be more obvious what Ubuntu version they're running, as well as other general information about their system.
John Lea, user experience architect on the Unity team, responds that in Ubuntu 11.10 the new LightDM greeter shows the Ubuntu version number, making that basic information very easily visible. For more detail, System Settings -> System Info provides a simple summary.
Volume adjustments for headphone use (#27275)
People often find that they need to adjust their sound volume when plugging in or removing headphones. It seems as though the computer ought to be able to remember this kind of thing and do it automatically; after all, a major goal of Ubuntu is to make the desktop Just Work.
David Henningson, a member of Canonical's OEM Services group and an Ubuntu audio developer, responds on his blog with a summary of how PulseAudio jack detection has improved matters in Ubuntu 11.10, and what's left to do:
The good news: in the upcoming Ubuntu Oneiric (11.10), this is actually working. The bad news: it isn't working for everyone.
Making it easier to find software to handle a file (#28148)
Ubuntu is not always as helpful as it could be when you don't have the right software installed to handle a particular file.
Michael Vogt, one of the developers of the Ubuntu Software Center, responded to this. It seems that most of the pieces to make this work nicely are in place, but there are a few more bits of glue required:
Thanks a lot for this suggestion. I like the idea and it's something that software-center itself supports now. In the coming version 5.0 we will offer to "sort by top-rated" (based on the ratings&reviews data). It's also possible to search for an application based on its mime data. To search for a mime-type, you can enter "mime:text/html" or "mime:audio/ogg" into the search field. What is needed however is better integration into the file manager nautilus. I will make sure this gets attention at the next developer meeting and filed bug #860536 about it.
In nautilus, there is now a button called "Find applications online" available as an option when opening an unknown file or when the user selects "open with...other application" in the context menu. But that will not use the data from software-center.
Show pop-up alert on low battery (#28037)
Some users have reported on Brainstorm that they are not alerted frequently enough when their laptop's battery is low, as they clearly ought to be.
This is an odd one, because there are already several power alert levels and this has been working well for us for some time. Nevertheless, enough people have voted for this idea that there must be something behind it, perhaps a bug that only affects certain systems. Martin Pitt, technical lead of the Ubuntu desktop team, has responded directly to the Brainstorm idea with a description of the current system and how to file a bug when it does not work as intended.
This is my fifth and final report about dextools for the 2011 Google Summer of Code program. During the summer, I worked with Matt Zimmerman and Stefano Zacchiroli to create a replacement web dashboard for the Debian Derivatives Exchange (DEX). The dashboard displays a list of projects and their respective tasks and then allows users to easily update the status of these tasks. The dashboard also contains two graphs for tracking the progress of projects and providing instant recognition of all contributions to a project.
At the start of the summer, I knew that I wanted to to work on improving the DEX infrastructure. I had worked with the team on the ancient-merges project, and while a lot of good work was accomplished, the process for tracking our progress was a bit clunky and difficult to interact with. Anyone who read my initial proposal for the Summer of Code would probably agree that it was quite vague. I really did not have a solid plan for what I wanted to accomplish.
This began to change after my initial phone call with my mentor, Matt Zimmerman. We decided that the first thing I would work on would be a basic dashboard. Our plan was to have all of the data stored on the Debian BTS. This would allow Debian Maintainers to benefit from DEX without needing to learn about DEX s tools and workflows. The dashboard would support multiple derivatives and multiple projects for each derivative. Each project would be made up of a list of tasks, where each task is linked to a BTS bug.
I prepared a mockup and some initial code based on that plan. The dashboard was able to successfully display a list of BTS bugs and their information. It generated the lists by assuming that bugs would be usertagged with a user of debian-derivatives@lists.debian.org and a tag in the format of dex
. At this point, we started trying to make sure that the dashboard would work for things like the ancient-patches, python2.7, and the upcoming large-merges projects. It did not take long to determine that it would not work for all of these things. The ancient-patches project started with a simple list of patch names. While most of these patches eventually ended up as bugs on the BTS, they did not start this way. The dashboard would need to support specifying a list of task names. This meant that it would also need to store data itself and that not all data could be located on the BTS.
This is when we first started to define what a task is. The dashboard tracks tasks. It does not directly track bugs or patches. A task is nothing more than a title and status with an optional assignee, note, and bug. This made it simple to convert a list of patch names or a list of bug numbers to a list of tasks.
During the early early stages of the project, the dashboard moved around a lot. We hoped to have it end up on dex.alioth.debian.org, but we were not sure whether Alioth would meet all of our needs. This was right around the time of the Alioth migration. Thanks to some tips, I figured out that I could use a simple cronjob on wagner to pull the git repository for vasks. This would allow the running instance of the dashboard to always be up-to-date. The Alioth admins have also been quite supportive. They have installed several additional packages for me that were necessary to keep the dashboard running.
From the start, Matt felt it was important to have some public documentation about the project. This would allow us to point people to something other than my blog posts. I decided to put this information on the wiki. At one point, I also had a copy of the wiki page stored in the git repository. This file was updated via a cronjob, and I then committed and pushed it when I had code changes. I ultimately decided that the best approach would be to maintain the page on the wiki. A basic README file is included in the git repository that simply links to the wiki page. On wagner, I have a cronjob running to pull a copy of the wiki page so that it can be accessed via the web.
Since Matt was traveling/moving this Summer, he arranged for Stefano Zacchiroli to fill in for him as my mentor temporarily. It was at this point in the summer that I began working on the first graph script. The goal was to have a script to parse the list of tasks and generate a graph showing the number of open tasks versus time. This would allow us to easily track our progress, estimate when a project will be complete, and detect periods of slowing down. We decided that while we liked the Ubuntu burndown charts, they were a bit more complex than we needed, so we did not reuse their code.
A dashboard that can t be modified is not that useful. An early goal of this project was to allow users to update the dashboard via the web. This proved to be a bit more difficult than I initially thought. First, all of the html files that make up the dashboard are generated by a script. This means that if you modify one of the html files directly, all changes will be lost the next time the script runs. When you make a change on a web form and hit submit, you usually expect to see the changes the next time you visit the page. In order to accomplish this, I made the form processing script directly modify the html files. However, at this point, we did not have a way to locally store extended information about tasks, causing the python script to delete all changes made via the web form. This issue was rectified fairly quickly. There is now a changes file that stores all information submitted via the web form. The python script reads this file and applies the changes when it is generating the html files. A second problem concerned the text inputs that were being used in most of the cells on the table. By default, some fields would have text cut-off if the string was too long. Text inputs have a size attribute that is meant to specify the number of visible characters. I attempted to set this attribute in a script to the length of the input s value. Strangely, the inputs ended up with a lot of extra whitespace following the text. This resulted in the table being stretched horizontally. After many hours of research and testing, I was unable to find a solution to this problem. We ultimately decided to simply remove the size attribute and accept that text will be cut-off.
My code got some early testing from Allison Randal and other people working on the dex-ubuntu-python2.7 project. They were a huge help in providing feedback and finding bugs in the code. While the dashboard was unable to meet all of their needs (due to being in the early stages of development), I hope that it at least helped to make it easier to track the project s status.
Whenever a script is accepting input from a user, it is important to validate it. Although the dashboard uses a select box to limit the choices for the status , it is still trivial to submit an arbitrary status for a task. That is why I added some validation to the form processing script. It will ignore any unknown values, ensuring that all tasks have a valid status. The other fields are a bit more tricky. They were designed to be arbitrary text fields. As a result, almost anything can be entered. There is no way to tell if something is a title or a person. We thought of a few ways we could change this, but most of them involved locking down the dashboard and restricting changes. The old dashboard treated the fields as arbitrary text, so we decided to not include any validation in the new one.
One popular feature request was the ability to link directly to a project. Early versions of the dashboard had one main dex.html page that used javascript to pull in the various projects. To get around this, I first trying using the query string to allow users to specify a distribution and project. While this worked, it resulted in long and ugly URLs. It took a while for me to be able to implement a cleaner solution. However, I eventually split each project into its own static HTML file. This meant that people could simply copy the URL from the address bar to link to a particular project. The main reason for doing this was that we felt the typical person would be interested in working on a specific project; most people won t be interested in navigating between the different projects. This change also allowed links that utilized the #graph anchor to function properly. Before, due to the way the page went about loading the project, trying to specify a specific project and #graph in the URL did not work properly.
There is a plugin for jquery that allows tables to be sorted by particular columns. For tables that just contain text, this works fine without any issues. However, once I started using select boxes and text inputs, things got a bit messed up. After some research, I finally figured out how to use addParser to instruct the tablesorter plugin in how to sort each column in the table.
In some of the earlier versions, I used some zebra stripes to make the table easy to read. Thanks to a suggestion from Paul Wise (pabs), I got rid of this striping and modified the code to color the rows based on the task s status. Complete tasks are green, in progress tasks are yellow, and incomplete tasks are red. This makes it very easy to find tasks that still need work as well as measure the overall status of a project.
Sometimes, a large task list can feel very intimidating. That is why I added the ability to hide completed tasks with the check of a box. Matt wanted to take this one step farther; he wanted the ability to also hide tasks with a closed bug. This is why you will find 2 checkboxes at the top of the dashboard that allow you to custimize which tasks are visible. Eventually, I might add support for specifying this in the URL to allow groups to link to a particular view of the dashboard.
In order to make it as easy as possible for new contributors to get involved with DEX projects, I wanted to have a way to document what a project is about and how to handle tasks. I decided to use the wiki for this. All of the per-project documentation lives at http://wiki.debian.org/DEX//
. You can even take advantage of wiki markup to format the documentation. The script will parse the HTML files generated by the wiki, do some minor cleanup, and then display the documentation at the top of the page. My original plan was to use the ?action=raw output, but the lack of formatting proved too restrictive. My second plan was to try and use something like BeautifulSoup or a regex to filter non-whitelisted tags out of the wiki-generated html. This failed and proved to be unnecessary. The wiki does a pretty decent job of preventing malicious markup from ending up on the dashboard.
There is a second graph that is displayed on the dashboard. This graph shows the number of tasks each person has completed. The graph is sorted so that the tallest bars on the left, and the goal is to provide some immediate recognition of work done by contributors.
Since we essentially allow anyone to go ahead and modify the dashboard, we knew it was important to have a method in place for dealing with abuse. Any malicious edits made on the wiki can easily be reverted. This idea of reverting to an earlier revision is what inspired me to use a second local git repository to store the projects/ directory. This directory stores all of the changes made via the web interface. Whenever the form is submitted or the script runs to update the list of tasks, any changes made to the projects/ directory are committed. This means that if a malicious user decides to delete everything on the dashboard, we can quickly and easily revert the changes. The revision history might also be interesting to analyze at a future date.
When multiple people are collaborating on a project, an issue tracker can be quite useful. For some reason, we did not set one up until quite late into the project. Despite that, we are still using it to track known bugs and feature requests. The issue tracker will become even more useful once the dashboard starts to be utilized by more people.
I spent a fair bit of time working on having the dashboard immediately respond to changes. For example, if you change the status of a task, the row color will instantly update. The form will also be submitted in the background, eliminating the need to hit submit to save the change. I also use ajax in some new dialogs that allow the user to create new projects and/or tasks via the web. While these forms are submitted in the background via ajax, I am unable to have the changes show up on the dashboard immediately. The generation of the task list is a bit slow, so having the script run more often is not really a good idea.
The next step for the dashboard is to really open it up for testing. We hope to start up a new DEX project shortly. This will allow us to see which features work and which need fixing. It will also help us find some of the bugs that are probably hiding in the code.
Finally, if you are interested in helping out with DEX or dextools, my code is available in a git repository (http://anonscm.debian.org/gitweb/?p=dex/gsoc2011.git) on alioth. There is also the issue tracker (https://alioth.debian.org/tracker/index.php?group_id=100600&atid=413120) where you can report bugs and request new features. Finally, you can join the debian-derivatives mailing list, join #debian-derivatives on OFTC, or contact me directly via email or on IRC (nhandler).
I have really enjoyed working on dextools this summer. I would like to thank Matt Zimmerman, Stefano Zacchiroli, and everyone else who helped with the project. I look forward to working with all of you more in the future.
Since my last report, a lot of visible changes have been made. First, if you edit the value of any of the text inputs on the dashboard table, your changes will be saved when you hit the submit button. This feature still needs a bit of work though. For example, I noticed during testing that if you submit the form multiple times, you will sometimes end up with approximately duplicate entries in the changes file (the file that stores user-submitted changes to the table). I also need to implement some checks to ensure that the values submitted are actually valid (i.e. we probably only want to allow a few possible values for the status column).
Another big improvement is the addition of a graph at the bottom of each project s page. This graph tracks the number of open tasks versus time. The graph is re-generated once each day based on a progress file. This file simply lists a date and the number of open tasks on each line. While the graph does correctly show a line tracking the progress, it still has some issues to sort out. For example, the x-axis does not properly display a time increment. Work on the graph was blocked for a while due to not having a fully-functioning dashboard. Now that the dashboard is functioning, I should be able to start working on fixing these graph issues.
Several people approached me to request the ability to link directly to a particular project. That way, users do not need to mess around with the select boxes. Above each table is a title. If you click on the title, the URL should update to link directly to the project. This is done with the distro and project query parameters. For example, a link to dex.html?distro=ubuntu&project=ancient-patches will take you directly to Ubuntu s ancient-patches project. Attempting to specify an invalid distro or project will cause the query parameters to be ignored.
While looking at the table title, you will also notice a new (Graph) link. This link will take you directly to the bottom of the page so you can view the graph. Currently, it is not possible to link directly to the graph anchor tag of a specific project. Attempting to do so will cause the #graph portion of the URL to be ignored. However, if you click on the actual graph, it will load a full-size graph.png that you can link to.
One rather significant issue that has shown up is that the table is very wide. On most monitors, you will have to scroll horizontally to view all of the columns. This is due to the fact that most of the cells are made up of text inputs (they don t have a background or border, but if you click on some text in the table, you should see the box) rather than plain text. The size parameter for a text input was meant to specify how many characters should be visible. However, while I have the size parameter set correctly for each input, it is showing up much wider than the text. I will continue to play around with this, since excluding the size parameter results in some values getting cut-off.
As mentioned in my last report, I have continued to work with Allison Randal to make the dashboard more useful. I have received feedback directly from her and indirectly from other people working on the python2.7 project. During my last phone call with my mentor, Stefano Zacchiroli, we agreed that it would be very helpful to spend some time documenting how to go about adding a new project to the dashboard and how to utilize all of the dashboard s features. The goal of this is to not require people to go through me or any other individual to be able to use the dashboard. So while the Summer of Code is meant to focus on programming, I will probably spend some time adding some documentation, as a program is rather useless if nobody knows how to use it.
Another bug that I will be working on has to do with sorting the table. I had this feature working early on in the project. However, the sorting broke when I starting using text inputs instead of plain text. I have done some research, and I hope to get the sorting re-implemented in the near future.
One other feature that has been requested by several people is the ability to hide completed tasks. This has been on my todo list for a while. Matt Zimmerman wanted to take this a step farther. He felt that the status of the task should be separate from the bug. This means that you can have an open task with a closed bug or a closed task with an open bug. The user could then have some more control over what data they are seeing. I will probably attempt to add some check boxes to control whether certain information is displayed. The user will also be able to specify this directly in the URL. This means that a team could link directly to all open tasks in a project.
It is also worth noting that thanks to some help from the fantastic Alioth admins, I now have a live instance of the dashboard running on Alioth. While the URL might change in the future, I hope to keep it running on Alioth.
If you are interested in helping out with DEX or dextools, my code is available in a git repository on alioth. You can also join the debian-derivatives mailing list, join #debian-derivatives on OFTC, or contact me directly via email or on IRC (nhandler).
A lot of work has taken place since my last report. First, since my original mentor, Matt Zimmerman, is going to be moving, he helped arrange for Stefano Zacchiroli to fill in for him temporarily. I have had a call with Stefano, and I am looking forward to getting to work with him more over the next couple of weeks.
Shortly after my first report, we realized that having the dashboard only display bugs severely limited its usefulness. In our original design, we had it pulling a list of bugs with specific usertags from the BTS. This forced all information to be added to bugs, which we viewed as a good thing since it made the information available to the Debian Maintainers without requiring them to familiarize themselves with DEX. The issue was that when looking back at the ancient-patches project or forward to the big merges project, a large portion of the work will be triaging and determining whether or not the changes are even applicable in Debian. If they aren t, there is no reason for a bug to be filed on the BTS. As a result, I spent a fair amount of time re-writing most of the dashboard. It still supports finding bugs on the BTS that have been tagged with a specific DEX usertag, but it is now task-based rather than bug-based.
A task can be a bug number, a package name, a patch name, or anything. It is merely something that needs to be done for a project. Each task will have a name, an assignee, a status, a note, and a bug associated with it. There are two ways to define a list of tasks for a project. The first is to tag bugs on the BTS with a user of debian-derivatives@lists.debian.org and a tag with a format of dex-<distribution>-
<project> (i.e. dex-ubuntu-ancient-patches). The script will convert the bugs to tasks and automatically set all of the fields. The second method is to specify the list of tasks in a plain text file on the server the scripts are being run on. Eventually, this list will be able to be specified via a web interface. Currently, you are only able to specify the task name in the file. This means that all of the other fields for a task created in this manner are initially blank. I am still deciding on the best way to specify additional information about each task in the file. Once I decide on a method, it should not be too difficult to modify the script to parse and store that additional information.
For the ancient patches DEX project, everyone involved had to update a status file stored in a VCS on alioth. This was rather annoying, so we decided early on that we want to be prevent this from being necessary for future projects. This means that the dashboard needs a way for users to modify the information on it. For tasks that are based on bugs, this is rather easy; you simply use the BTS email interface to adjust the bug. However, for other tasks, we needed the ability to modify them on the dashboard. I have managed to get this partially implemented. Currently, all cells on the table are text input fields (with css being used to hide the boxes). This means that you can click on any cell and edit its contents. Hitting the submit button will execute a script to save these changes. This is the tricky part. The dashboard is a collection of html files generated by a python script. The script is meant to be run by cron every n minutes. When you update something via a web interface, you usually expect to see the changes instantly. This means that we can t wait for the python script to run again and re-generate the html files. As a result, I currently have it setup so that when you submit the form, it will go and modify the html files to reflect the changes. The next time cron runs the python script, the html files will be re-created, and the changes made by the user will be lost. Once I come up with a method for storing additional information about a task, it shouldn t be too hard to make changes made by a user via the web interface permanent.
I also did some work on a script to generate graphs. There are two types of graphs I hope to be able to display on the dashboard. One graph will help show which individuals have done the most work on a project. I have not started working on this graph yet. The second type of graph is meant to show the number of open tasks versus time. This will allow people to quickly see the progress of a project, estimate how long the project will take to complete, and notice if things start to slow down. I have some initial code for this graph using matplotlib. However, due to the many changes that were made to the dashboard, I had to hold off a bit on this graph, as it relies on certain information provided by the dashboard. I hope to sort out some of the last few issues with this graph, such as getting the axes to display properly, and then merge it with the dashboard sometime in the next few weeks.
Allison Randal approached me a while ago to see if my Summer of Code project might be in a state where it could be used to assist with a Python 2.7 DEX project that she was working on. For a while, I did not think that it would be ready. However, last week, I helped her get the related bugs usertagged appropriately, and they now show up in the DEX dashboard. While the dashboard is still not complete, it does allow her and everyone else working on the project to clearly see the tasks (bugs) that they need to focus on and the current status of each of them. I plan to try and incorporate any feedback I receive from people working on this project into my designs.
All of my code is available in a git repository on alioth. Keep in mind that it should not be considered stable at this point. Further status updates will be posted on my blog, and I will try to keep the wiki page updated. I am still trying work out where we will host the actual dashboard. In the meantime, you can try out my testing version of it.
During the next few weeks, I will be focusing my efforts on coming up with a method for storing additional task information. That will allow me to support permanently storing changes made by users via the web interface. I also hope to finish the first graph script and merge it with the dashboard. Finally, I hope to find a permanent home for the dashboard, as having it on my personal server is not a viable long-term solution.
Finally, if anyone is interested in getting involved with DEX, feel free to join the debian-derivatives mailing list, join #debian-derivatives on OFTC, or contact me via email or on IRC (nhandler).</project></distribution>
It s been a few months since Zack and I announced the DEX project, which aims to improve the Debian ecosystem by working jointly with derivative distributions.
Our first milestone
The goal of our first project, nicknamed ancient-patches, was to clear out an old batch of a few hundred Ubuntu patches whose status was unclear. We couldn t tell which ones had been merged into Debian, which were waiting in the BTS, and which had yet to be submitted to Debian. All of them were several years old.
I m pleased to announce that this project is now complete. Thanks to help from David Paleino, Colin Watson, Nathan Handler and Steve Langasek, we were able to clear over 95% of the patches in a matter of days. These were the easy ones: patches which were obsolete, or had already been applied. We discussed the remainder, and resolved all of the patches whose status was still unclear. This left the harder ones: patches stalled in the BTS, and patches where there was no consensus about what to do with them.
One of the stalled patches was merged into Debian via an NMU, eliminating the delta between Debian and Ubuntu. Another had been submitted to Debian by a third party, but was no longer shipping in Ubuntu, so we considered it obsolete for purposes of this project.
This has left only two patches out of the original list of 277. Both of them are filed in the BTS and have been discussed with the relevant maintainer team. One of them is expected to be obsoleted when a new upstream version is packaged, which implements similar functionality. The other is being discussed with the upstream developers, but there is no conclusion yet about whether it can be merged upstream or in Debian.
Conclusions
Although we weren t quite able to clear the whole list, we still consider the project to be a success because:
We ensured that all of the patches received due consideration for inclusion in Debian
We proved the concept of DEX, with developers from Debian and derivatives cooperating on a common goal and sharing tools
Most importantly, we learned from the experience
What s next
In the most recent DEX update on debian-derivatives, I highlighted a few important events for DEX:
Our second major project, nicknamed big-merges , will begin soon. Our goal is to identify the few packages which are most diverged between Debian and Ubuntu, and work to get them as close to identical as possible. If you have suggestions for packages to focus on, let us know!
Allison Randal is beginning a DEX project to implement the Python 2.7 transition across Debian and Ubuntu
Zack is organizing a derivatives BoF at DebConf 11
We re looking forward to seeing DEX develop further. If you d like to get involved, come and join us on the debian-derivativesmailing list or IRC (#debian-derivatives on freenodeOFTC).
Matt Zimmerman and Stefano Zacchiroli
As I mentioned recently, the Ubuntu Technical Board is reviewing the most popular topics in Ubuntu Brainstorm and coordinating official responses on behalf of the project. This means that the most popular topics on Ubuntu Brainstorm receive expert answers from the people working in these areas.
This is the first batch, and we plan to repeat this process each quarter. We ll use feedback and experiences from this run to improve it for next time, so let us know what you think.
Power management (idea #24782)
Laptops are now outselling desktops globally, and laptop owners want to get the most out of their expensive and heavy batteries. So it s no surprise that people are wondering about improved power management in Ubuntu. This is a complex topic which spans the Linux software stack, and certainly isn t an issue which will be solved in the foreseeable future, but we see a lot of good work being done in this area.
To tell us about it, Amit Kucheria, Ubuntu kernel developer and leader of the Linaro working group on Power Management, contributed a great writeup on this topic, with technical analysis, tips and recommendations, and a look at what s coming next.
I am going to attempt to summarize the various use profiles and what Ubuntu does (or can do) to prolong battery life in those profiles. Power management, when done right, should not require the user to make several (difficult) choices. It should just work providing a good balance of performance and battery life.
IP address conflicts (idea #25648)
IP addressing is a subject that most people should never have to think about. When something isn t working, and two computers end up with the same IP address, it can be hard to tell what s wrong. I was personally surprised to find this one near the top of the list on Ubuntu Brainstorm, since it seems unlikely to be a very common problem. Nonetheless, it was voted up, and we re listening.
There is a tool called ipwatchd which is already available in the package repository, and was created specifically to address this problem. This seems like a further indication that this problem may be more widespread than I might assume.
The idea has already been marked as implemented in Brainstorm based on the existence of this package, but that doesn t help people who have never heard of ipwatchd, much less found and installed it.
What do you think? Have you ever run into this problem? Would it have helped you if your computer had told you what was wrong, or would it have only confused you further? Is it worth considering this for inclusion in the default install? Post your comments in Brainstorm.
Selecting the only available username to login (idea #6974)
Although Linux is designed as a multi-user operating system, most Ubuntu systems are only used by one person. In that light, it seems a bit redundant to ask the user to identify themselves every time they login, by clicking on their username. Why not just preselect it? Indeed, this would be relatively simple to implement, but the real question is whether it is the right choice for users.
Martin Pitt of the Ubuntu Desktop Team notes that consistency is an important factor in ease of use, and asks for further feedback.
So in summary, we favored consistency and predictablility over the extra effort to press Enter once. This hasn t been a very strong opinion or decision, though, and the desktop team would be happy to revise it.
Icon for .deb packages (idea #25197)
Building on the invaluable efforts of Debian developers, we work hard to make sure that people can get all of the software they need from Ubuntu repositories through Software Center and APT, where they are authenticated and secure. However, in practice, it is occasionally necessary for users to work with .deb files directly.
Brainstorm idea 25197 suggests that the icon used to represent .deb packages in the file manager is not ideal, and can be confusing.
Matthew Paul Thomas of the Canonical Design Team responds with encouragement for deb-thumbnailer, which makes the icon both more distinctive and more informative. He has opened bug 685851 to track progress on getting it packaged and into the main repository.
I have reviewed the proposed solutions with Michael Vogt, our packaging expert. Solution #1 is straightforward, but we particularly like solutions #5 and #10, using a thumbnailer to show the application icon from inside each package.
Keeping the time accurate over the Internet by default (idea #25301)
It s important for an Internet connected computer to know the correct time of day, which is why Ubuntu has included automatic Internet time synchronization with NTP since the very first release (4.10 warty ). So some of us were a little surprised to see this as one of the most popular ideas on Ubuntu Brainstorm.
Colin Watson of the Ubuntu Technical Board investigated and discovered a case where this wasn t working correctly. It s now fixed for Ubuntu 11.04, and Colin has sent the patches upstream to Debian and GNOME.
My first reaction was hey, that s odd I thought we already did that? . We install the ntpdate package by default (although it s deprecated upstream in favour of other tools, but that shouldn t be important here). ntpdate is run from /etc/network/if-up.d/ntpdate, in other words every time you connect to a network, which should be acceptably frequent for most people, so it really ought to Just Work by default. But this is one of the top ten problems where users have gone to the trouble of proposing solutions on Brainstorm, so it couldn t be that simple. What was going on?
More detail in GNOME system monitor (idea #25887)
Under System, Preferences, System Monitor, you can find a tool to peek under the hood at the Linux processes which power every Ubuntu system. Power users, hungry for more detail on their systems inner workings, voted to suggest that more detail be made available through this interface.
Robert Ancell of the Ubuntu Desktop Team answered their call by offering to mentor a volunteer to develop a patch, and someone has already stepped up with a first draft.
Help the user understand when closing a window does not close the app (idea #25801)
When the user clicks the close button, most applications obediently exit. A few, though, will just hide, and continue running, because they assume that s what the user actually wants, and it can be hard to tell which has happened.
Ivanka Majic, Creative Strategy Lead at Canonical, shares her perspective on this issue, with a pointer to work in progress to resolve it.
This is more than a good idea, it s an important gap in the usability of most of the desktop operating systems in widespread use today.
Ubuntu Software Centre Removal of Configuration Files (idea #24963)
One feature of the Debian packaging system used in Ubuntu is that it draws a distinction between removing a package and purging it. Purging should remove all traces of the package, such that installing and then immediately purging a package should return the system to the same state. Removing will leave certain files behind, including system configuration files and sometimes runtime data.
This subtle distinction is useful to system administrators, but only serves to confuse most end users, so it s not exposed by Software Center: it just defaults to removing packages. This proposal in Ubuntu Brainstorm suggests that Software Center should purge packages by default instead.
Michael Vogt of the Ubuntu Foundations Team explains the reasoning behind this default, and offers an alternative suggestion based on his experience with the package management system.
This is not a easy problem and we need to carefully balance the needs to keep the UI simple with the needs to keep the system from accumulating cruft.
Ubuntu One file sync progress (idea #25417)
Ubuntu One file synchronization works behind the scenes, uploading and downloading as needed to replicate your data to multiple computers. It does most of its work silently, and it can be hard to tell what it is doing or when it will be finished.
John Lenton, engineering manager for the Ubuntu One Desktop+ team, posts on the AskUbuntu Q&A site with tools and tips which work today, and their plans to address this issue comprehensively in the future.
Multimedia performance (idea #24878)
With a cornucopia of multimedia content available online today, it s important that users be able to access it quickly and easily. Poor performance in the audio, video and graphics subsystems can spoil the experience, if resource-hungry multimedia applications can t keep up with the flow of data.
Allison Randal, Ubuntu Technical Architect, answers with an analysis of the problem and the proposed solutions, an overview of current activity in this area, and pointers for getting involved.
The fundamental concern is a classic one for large systems: changes in one part of the system affect the performance of another part of the system. It s modestly difficult to measure the performance effects of local changes, but exponentially more difficult to measure the network effects of changes across the system.
When I gave a Revealing Errors talk at Lug Radio Live USA, I had the
misfortune of being up against Robert Love's talk on Android which many
people at the conference wanted to see -- myself included! One person
who showed up to my talk anyway was Allison Randall. She was
apparently entertained enough to invite me to give a short version of
the talk as one of the keynote presentations at OSCON 2008!
In the talk, I covered the ideas behind my Revealing Errors project
and quickly walked through a few examples that showcase what I'm trying
to do. I'm happy with the result: a couple thousand people showed up for
the talk despite the fact that it was at 8:45 AM after the biggest
"party night" of the conference!
For those that missed it for whatever reason, you can watch a video
recording that O'Reilly made and that I've embedded below.
A larger version of the Flash video as well as a QuickTime version is
over on blip.tv and I've created an OGG Theora version for all
my freedom loving readers.
In the lands where the camel roams, the white (albino) camel is a rare and revered individual. The White Camel Awards recognize the many significant contributions made by the unsung heros of the Perl community. The efforts of these volunteers collectively make the Perl language and the Perl community better for all of us.– The Perl Foundation.
2007 Recipients
Allison Randal
Allison is at the center of the Perl community. She’s been president of The Perl Foundation, a leader and manager of various parts of the Perl 6 and Parrot efforts, as well a Perl author and editor. Her latest contribution to Perl is version 2 of the Artistic License, under which most open source Perl code, and Perl itself, is licensesd “under the terms of Perl itself”.
Tim O’Reilly
You may think of Tim as the guy who published Programming perl and Learning Perl, but he also kick-started the current form of the Perl community by giving it a place to come to together once a year. O’Reilly & Associates started The Perl Conference in 1997. Perl Mongers, the organization that helped start Perl users groups all over the world, started at that first Perl Conference. O’Reilly Media has been incredibly gracious and helpful to the Perl community.
Norbert E. Gr ner
Norbert help start the German Perl Workshop in 1999 and now is involved in several of the Perl conferences and workshops that take place in Europe. He’s the chair of the YAPC::Europe committee.