oh, we work too

My current mailfilter has two features which increase my day-to-day productivity:

1. a "tickler," which is a reminder system inspired by the tickler file component of David Allen's Getting Things Done action management method: I can submit emails and notes to the tickler along with a timestamp in the future, and the tickler delivers the mail (or note) to my inbox when the timestamp has passed.
2. the ability to delay certain types of mail (e.g. Debian mail is held until the weekend, while news items and the like are only delivered at night).

My current implementations work alright, but they're brittle and crap. What's worse is that the two features could be combined and handled by one and the same tool, but I implemented them differently for now: The tickler consists of a Maildir, to which I can submit mails, either by mailing a note to a specific email address (currently broken, thus not linked from here), or by adding a tickle stamp (X-Tickle header) to a message with this script, saving it to the local tickler mailbox and asking offlineimap to shove it to the server. On this server, the tickler queue is regularly scanned by a script, which resubmits mail whose timestamp has expired to my mailfilter, where it is treated somewhat specially as a resubmitted mail. After almost three months with this setup, I can identify the following shortcomings:

• Since the queue is only processed on the server, I cannot register a mail with the tickler without going online and synchronising my mail before the timestamp expires. So if I am on my way to a conference on Monday and tell the tickler to resubmit a given message on the next day, but I don't connect before Wednesday, I won't get the mail on Tuesday (because the mail does not actually reach the tickler until Wednesday, when it's immediately resubmitted). I have not found an algorithm that would let me run a queue processor on the server and my laptop without the two causing mail duplication and potentially interfering with each other.

• I keep all my mail in a single Maildir to make searches easy. Since mail registered with the tickler is stored in a separate Maildir, I often need to search two Maildirs if I am looking for a specific message. This is awkward.

• The queue processor has to iterate the entire queue, which is fine for a hundred messages, but it does not scale.

• I have not found a way to ask mutt to tickle me about a message I send without going to the sent messages folder and scheduling the message for tickling manually. One approach would be record=no and a custom sendmail script, which would store mail to the store or tickler Maildirs, depending on whether the X-Tickle header is present, but I want to avoid a custom sendmail script for reasons unknown. Another approach might be to set bcc to the appropriate tickle address, but this would result in duplicate mails, unless I set record=no individually.

When I implemented the delay queue, I knew of these problems and went a different way: delayed mail is stored in a Maildir and a (msgid, timestamp, filename) tuple is inserted into an sqlite3 database on the mail server. This script regularly processes the queue. If I just sent you the chills, at least we have the same taste. There are numerous problems with this approach, the foremost being that Maildir filenames are not guaranteed to be constant: mails jump between the new and cur directories, and tags, such as seen are encoded in the filename (thus, symlinks also cannot be used). My script now uses an ugly heuristic, which at least makes it work. I should investigate whether inodes could be used instead as I think those wouldn't change throughout the lifetime of a mail, at least while it's not moved between folders. I initially considered just dumping messages to files and encoding the timestamp in the files' mtime, but then I would not be able to access the queue with mutt in case I needed to fetch a delayed mail prematurely, or if I wanted to synchronise the queue with offlineimap as well. The past few days, I've been condensing experiences from both approaches and am working out a new technique to combine both features. In essence, I think the database/index approach is the best, if I can figure out a way to uniquely identify mail message files, ideally across folders. Assuming I can use inodes for that, delayed mail would then be stored into the delayed Maildir and an (inode, timestamp) tuple saved into the database. Tickler mail would be stored along with all other mail in my store Maildir and would get a similar entry in the database. This approach solves some problems and leaves others. Assuming I synchronise the store Maildir remotely (which I do), then I can easily fathom making modifications via IMAP which causes orphan records in the database (if only IMAP would allow me to store key=value pairs for mails ). Furthermore, I'd have to submit mails to the tickler by bouncing them to an email address, and deleting the local copy, unless I want duplicates. If now the mail is somehow dropped, I've lost mail. Still unhappy about all of this, still searching for a better implementation, I'd appreciate any feedback! NP: Luminous Flesh Giants: Duma I Upadek

* In my constant carvings about sane email client, I found that claws in lightweight and very well email client. Icedove to claws migration was not so easy, but script did it! And, it was fun to read Planet Claws-mail too! I hope that someday, enigmail will work (againg) with icedove!

Graphs wanted for Homepage and Vcs-XXX field stats I've recently set up a couple of stats page for some fields usage in Debian packages: one for the Homepage field, one for the Vcs-XXX fields. As Raphael observes, those stats need percentage and graphs, and even Lucas asked me about that back in Extremadura, also providing a sample gnuplot script. These days my time is quite lacking, and I'm also missing the time to put its sample script to work So I've made available the source code which is generating the two stats above (see the "source code ... is available" notice at the bottom of the pages). Patches are welcome for generating percentages and graphs, thanks in advance

So I am done teaching my first two classes of 2007 and now I am nearly done with the syllabus for The One (yes one, they give new faculty a course release, thank the powers that be!!) class I am teaching next semester: Topics in Digital Media. Unlike this semester where the name of the syllabus game was a motley collection of chapters/articles, next semester I am concentrating on whole books although we won’t be reading all of them in their entirety so over the next few weeks I have to perform some whittling magic. It is just nice to have it done before I head away South to visit the family.

Since most of the readers of this blog come from Debian Planet where you can’t really see if there are comments, I thought I would link to the comments from my recebt blog post on blogging as many make some nice points. Additionally, here is a link to Joe Reagle’s post aptly entitled blog anxieties where he places a tripartite religious framework to understand the life cycle of a blog. I also agree with Alisa who notes that blogging is as much about the writer as is the reader (and I tend to post stuff for the sake of remembering, archiving etc) but I still think that the issue of overabundance is huge for I am as much a a consumer as I am a producer of blogs. I love reading people’s blogs but I just can’t keep up if they do it too often. Tenured Radical also paid a visit and well, if you have not checked out her blog, she is a fantastic, fantastic writer/thinker and what I like about her blog most is that she weaves together just the right ratio of personal and public (for lack of better words). What I mean is that she tells her stories from the vantage point of her experiences, of her position etc. I can imagine a real person behind the posts, which is always part of the fun of blog reading, a form of imagination that frankly is not there with mainstream news and journalism.

The Open Web Awards are accepting votes and well, my vote is going to Kaltura and if you don’t know what the do, or who they are, check em out. You might be surprised to find some really cool video editing software that is pushing collaboration to new heights.

(continued from "This is Delicious Crack") Essentially, a .msi file is a relational database, and everything that should happen during an installation is stored as an entity. The benefit is obvious: there can not be any bugs in the packaging that cannot be rolled back easily, as the Installer always knows what is happening. Well, in theory. This approach has a massive drawback: you cannot really do anything that was not anticipated. To solve this, the Installer allows you to provide a DLL that takes over a bit of the installation procedure - which still ought to be described in the database. When properly written, these DLLs can be reused and become a kind of standard, such as DIFxApp, which is the handler for device driver installation. The last remaining difficulty here is that Visual Studio does not know hardware drivers at all (there is a separate SDK), and so it supports neither building nor installing them, which means that I have to edit the .msi file generated here by hand to add the information for the driver installer.

For those who want to see NYC streets turned into more humane, hospitable and especially bike-friendly places, this blog is for you. I love NYC because of its active street life but there is certainly almost endless room for massive improvement and these folks are pushing just for that. And for those involved in disability rights activism, Stop Eugenics looks like a good (and important) new blog.

It is nearly the end of the semester and in order to do all the test-making magic and grading I had to do before I am re-swamped with a stack of new papers (on the 17th of December to be exact), I spent the weekend put, grading every last response papers that I had my students write throughout the semester. When I finished those, I proceeded to get sick with the cold that seems to keep recirculating among faculty and students. Today I had to shift attention to my apartment, which, out of neglect had been turned upside down and inside out, to tidy up some before a slew of guests arrive tomorrow night. To get through the weekend and the cleaning-spree, I had a lot of help from my friends, mostly in the form of Daft Punk (in specific, their Alive album), and then when I wanted something more mellow, Damian Jurado took over (Medication and Ohio are two of my favorites), but then the song that really did the most to get me into that special zone was TV on the Radio’s aaaaaaaaamazing Wolf Like Me. Generally speaking, you can’t go wrong with wolf songs, can you? but this one takes the cake. Now to sleep off this cold…

So, the CoRoT team apparently had a meeting in Paris on Monday. I've been waiting with baited breath (and intermittent internet access, for assorted reasons) for the results of a "Press Conference", that according to the blogosphere rumour mill, and a (started by a breathless interim report) was going to rewrite the sciences of exoplanets and stellar seismology. There were rumours of special editions of Nature (or Science, or A & A), Earth-mass planets (possible according to the apparent spec. of CoRoT). This mattered to me as I'm finishing up a literature review for a PhD and the work is on exoplanets, and depends a lot on stellar variation. CoRoT could rewrite large sections of my conclusions. So, it looks like I finish my review and just have to be patient while they release their results in the proper peer-reviewed manner. But it would probably be a good idea for the CoRoT team to set up a blog, as Rumour Central, to respond to mentions and rumours about their project that appear on the 'net. Kudos, for example, to Wesley Traub for taking the time to answer my questions aboout Darwin/TPF in this manner. Tags astronomy, exoplanets, thesis, corot

I use Claws-Mail (née Sylpheed-Claws) for my email. One nice feature I found a couple of months ago was subject-line rewriting: A number of mailing lists I subscribe to add a totally obnoxious prefix on each outgoing message. I’ve yet to think of a good reason to do this - I can only assume it’s for far less capable mail clients. Or something. This results in much cleaner message list. You can probably do this in procmail, but having it in the presentation layer has the advantage that the rewrite is non-destructive - pressing Reply constructs a subject line from the original.

An essay I wrote for the Wikimedia Foundation fundraising drive was just published on the the foundation's Why Give blog. The essay, titled Wikimedia and the Free Culture Movement, discusses the movement for free culture, Wikimedia's central role in it, and the importance of supporting the foundation because, I argue, the immediate success of the free culture movement is intimately tied up in Wikimedia's efforts. It is very exciting to see an essay I wrote linked prominently from the top of every page in Wikipedia! It is also exciting to imagine that I might help the Wikimedia foundation at this important time in that organization's life. I am giving to two organizations in support of two causes this year: the Wikimedia Foundation in support of free culture and the Free Software Foundation in support of free software. No other two groups are as committed or are doing as much to build a world where knowledge, and the tools we use to use, produce, and communicate it, are and remain free.

This was informally confirmed by someone at Google: Given this layout code:

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
  android:orientation="vertical"
  android:layout_width="fill_parent"
  android:layout_height="fill_parent"
  id="@+id/ll">
</LinearLayout>

And this Java code:

public void onCreate(Bundle icicle)
 
    super.onCreate(icicle);
    setContentView(R.layout.main);
    LinearLayout ll = (LinearLayout)findViewById(R.id.ll);
    RadioGroup rg = new RadioGroup(this);
    rg.setLayoutParams(new LinearLayout.LayoutParams(
            LinearLayout.LayoutParams.WRAP_CONTENT,
            LinearLayout.LayoutParams.WRAP_CONTENT));
    ll.addView(rg);
    RadioGroup.LayoutParams rlp = new RadioGroup.LayoutParams(
        RadioGroup.LayoutParams.WRAP_CONTENT,
        RadioGroup.LayoutParams.WRAP_CONTENT);
    RadioButton foo = new RadioButton(this);
    foo.setText("foo");
    foo.setLayoutParams(rlp);
    rg.addView(foo);
    RadioButton bar = new RadioButton(this);
    bar.setText("bar");
    bar.setLayoutParams(rlp);
    rg.addView(bar);
    RadioButton baz = new RadioButton(this);
    baz.setText("baz");
    baz.setLayoutParams(rlp);
    rg.addView(baz);
 

The resulting radio buttons fail to be mutually exclusive. This is a bit frustrating, as Hecl doesn't utilize the XML layout system. Or rather, you can use that if you want, or you can create everything in Hecl. The individual at Google says the problem will be fixed in a future SDK release, so hopefully things will work in the near future. I also hope it's open sourced soonish, so that problems like this can be fixed without requiring the intervention of "the hand of Google". Of course, in this market, there's also the issue of things being shipped on hardware, where open source or not doesn't matter much if the bug is out there, because you simply can't fix it on millions of devices. Hopefully the product that ships will be very solid.

It seems that -- again -- I've managed to catch something nasty on the way back (probably I should only check in my coat if I know that it won't take long until boarding). So, two days later than anticipated, I am officially back and working. I have yet to clear the autobuilder backlog of the last three days (these ARM boxes are blazing fast these days; and a lot of packages failed due to GNOME being uninstallable) and also catch up with the real world (I turn off my phone when I have a migraine attack, keeping a log of calls but this time there's just a batch of unknown numbers with some calls having dropped off the list already -- please send emails instead of calling if it is important, there is a reason why I don't use my phone for business). In other news, this posting needs paragraphs.

* Flickr pics says whole story. Foss.in == Fun! I learned a lot while managing Debian/Ubuntu stall. Next year we will have more better stall and representation from community. Debian Tshirts were given to 3 nice people (Thanks - Prasad, Amod and Anivar) who maintain stall during my mysterious gaps. Sorry to Cydork - yeah, size does matter! * Thanks to team at foss.in, You people rocks (always)! * Finally, back /home (as usual, flight was delayed). Kavin is crawling much intelligent way and demands things (he want to chew everything..). So, time to alert!

I’m very surprised about the popularity of an old post of mine, regarding my experiences with CompUSA. It continues to collect horror story comments, the last one coming less than three weeks ago. While any company has its detractors (especially any company dealing directly with the public), it seems odd to me that people continue to be motivated enough to post to my blog, of all places, their tales of woe. For me, life has been very CompUSA-less of late. Indianapolis now has a Fry’s, one of only two east of the Mississippi as of this writing, and for someone in the relatively tech-starved Midwest, it is a godsend. (People from the west coast: please stifle your laughter as best you can.) And evidently enough of these horror stories have been passed around that they felt the need to close over half their stores in February. The Indy store was spared that time, but not for long.

The electronics retailer decided to finish what it had started earlier this year, announcing that it would sell or close the remainder of its stores in the US after the holiday season. The company, controlled by Mexican retail management company Grupo Sanborns since 1999, has been sold to Gordon Brothers Group, a restructuring firm that will be responsible for selling off the remainder of its assets.

In an abstract sense, less competition in the electronic retail business isn’t ever good. But it’s arguable that we’ve never had so much competition in the electronic retail business if you count the Internet stores that have sprung up all over. And I’m certainly happy to see an outfit that will slander people for profit go belly-up.

Seems my work in Extremadury during the QA meeting was worth it. Just added some new output to the removals page generation, which now also shows the date the last upload was done, together with the content of the Maintainer-field from that upload. And it was (very unexpected) an easy thing to add this, once I had the right SQL query. I like ruby.

Ok, so two people replied to yesterday's post about triple negations - Dato (by email) and MadCoder. Both, said basically the same thing: false and && true are silly noops. And yes, knowing this, I added them. Why? Clarity... At least having them at the end of a test shows the statement is of conditional nature (and not just another obscure attempt to do $ DEITY -knows-what). They at least look cleaner than a one-line-squashed if block in a makefile. To me, at least ;-)But... If you noticed this post's title, it goes beyond this comment - One of the most benefical effects I noticed when I installed Jaws 0.7 (over 0.6, of course) is that I no longer had the swarms of spambots flooding me - I often had hundreds of comments a day, and nowadays I hardly get any spam. Now, I fail to see what is so strange in my blog's comment forms (it does not even have any obvious Javascript, although it does obfuscate a bit the source of the captcha image). And you are not the first Debian people to complain you cannot post comments to my site. Strangely, few non-Debian people have ever complained.And yes, the spam has stopped, almost completely.So, Debian guys: Are you human?

My domains DNS is still hosted with the company I registered it at. I'm planning to move it to a different company early next year. So when a friend asked me for secondary nameserver exchange, I already set up the new DNS.So my current setup is like this:Primary and secondary nameserver are at my old provider, serving their copy of the DNS zone (which obviously lists their nameservers as NS)The A and MX records point to my server, which will sometime also be the new master NS. This host has an own copy of the zone file, which agrees on the 'regular' entries, but lists my server as well as the friends two servers as NS. This is the only link from the domain to the friends servers. Let me emphasize this: neither my server nor my friends servers are currently listed in the .tld database. Their NS entries still point to the old providers server. I'm planning to change that in January.Now my friend told me, that he had about 10 email delivery attempts to my domain in his logs, obviously coming from some spammers.WTF? In order to link my domain to his server, you'd need to

• find the official NS for my domain
• lookup my A or MX record (not the NS record!)
• use this record as new NS
• lookup an NS record, using my new NS
• use this NS record as MX

What is the reason to jump through all those hoops? Do many admins configure a secondary NS to be an unlisted, unprotected relay for incoming email?Is it common for secondary NS to receive random emails from spammers?