Search Results: "ana"

21 December 2007

Ross Burton: Pimlico on Maemo Chinook

Every since the Maemo Chinook beta was relased, people have been asking when we're going to make Pimlico packages available. I'll skip over the fact that Pimlico is open source, so building a package yourself is trivial (I suppose its a good thing that there are users who can't do that!). Now, Pimlico consists of Contacts, Dates and Tasks. Contacts doesn't have a Maemo port as such and the device already has an addressbook of sorts, so I tend to leave that until last. That leaves Dates and Tasks, both of which require an all-new Evolution Data Server to be built, because Nokia strip down EDS for Maemo and don't install the calendar component to save disk space. This generally shouldn't be a problem as we've been building replacement EDS packages for some time now which adds the calendar component and restores some functionality to the common libraries. So, Rob set about re-syncing our packaging with the Chinook EDS (there are several Maemo-specfic patches we obviously need to include) and started a build. Then our plan started to fall apart. To cut a long story short, the Chinook's Application Manager has an additional sanity check which wasn't in previous releases. The gist of it appears to be that a package foo from repository A cannot upgrade package foo from repository B. Or, libebook from maemo.org cannot be upgraded to libebook from o-hand.com. I can see how this can stop people accidently breaking their device by installing broken core packages, but it's also stopping us provide core packages with enhanced functionality. Never fear, we have a plan. It's not incredibly pretty and will take a day or so of tiresome recompiles to get working, but we'll get there. That said, we've all been busy and now it is the Christmas holiday... so I wouldn't bet on being able to run Dates and Tasks on your N800/N810 before the new year. As a reward for patience, however, there are Contacts 0.8 packages for Chinook in our repository. NP: Last.fm Recommendation Radio

Andrew Pollock: [tech] Waiting for a software RAID-1 to resync is like watching paint dry

I'm so annoyed with myself. So today was the day I was going to upgrade the disks in daedalus. I'd finally managed to convince Dell to sell me two new 140Gb disks to replace to two 70Gb disks that were fast approaching capacity. I'd organised for access to the datacentre from 10am. I arrived early. I didn't manage to get into the datacentre until about 11am, because they were having problems remotely unlocking the door, and I had to wait until an air conditioning repair guy turned up, who was fixing some air conditioning problem. I hadn't had a lot of time to plan how I was going to do the upgrade, but I figured it was going to be pretty straightforward. Each disk is partitioned into three partitions, all of which are mirrored using software RAID. The first partition is the root filesystem, which is just an ext3 filesystem, mirrored. The second partition is swap, also mirrored, and the third partition is an LVM physical volume. The first two partitions are only a couple of gigabytes. So the basic plan was to remove the second disk (disk 1, aka /dev/sdb), put in the new bigger disk, load the partition table from the original disk 1, resync the three mirrors, and then boot off the new disk 1, and rinse and repeat with disk 0. This was all going swimmingly, when right at the end (in fact after the third partition finished syncing according to the kernel), /dev/sda decided it had a few unreadable sectors. This made the software mirroring have a dummy-spit, and decide it needed to restart the syncing from scratch. I'd seen this before with a bad disk. So I wishfully hoped it was something to do with the new disk being bigger than the old disk, so tried to roll back to the original disk 1. As was pretty much expected, disk 0 exhibited the same problems at the same point. So the problem was disk 0, not the new disk 1. So it seemed like I just managed to pick the wrong half of the mirror to work from. I thought I'd try manually dd'ing the third partition from /dev/sda to /dev/sdb, ignoring the errors, but that didn't result in a usable half of a mirror either. Interestingly, these bad sectors hadn't made themselves known until this point. The RAID array had been otherwise healthy, and SMART hadn't uttered a peep. I think at this point, after spending about 3 hours fighting with it, I decided to move up a layer. I could successfully sync /dev/md0 (the root filesystem) and /dev/md1 (the swap device), and the only reason I could think of these bad sectors not giving me grief already, was because they were right at the end of the disk, and I hadn't quite used all the free physical extents in LVM yet. Lucky I hadn't decided to grow things any further. So I decided that rather than trying to convince this existing /dev/md2 to sync with the new /dev/sdb3, I'd just build a new degraded RAID-1 on /dev/sdb3 and move all the physical extents from the degraded RAID-1 on /dev/sda3 to the new degraded RAID-1 on /dev/sdb3. I tried the logical volume with /tmp on it first, since it was fairly sacrificial. It worked fine. So I tried the largest logical volume, /srv, which was 30Gb. That went fine as well, so I moved the rest. All of them went without complaint. So then I removed the now unused /dev/md2 from the volume group, and finally managed to boot with just the new disk 1. Then I was able to put in the new disk 0, and just do a standard RAID-1 rebuild onto it. Hooray for LVM saving the day yet again. And this is where I should have stopped. At this point it was about 3pm. I'd been aiming to leave by 4pm to pick up a rental car that I had to get by 5pm. I'd wanted to also upgrade to Etch while I was physically in front of the box, in case there were any nasty kernel/udev issues that made the box unbootable. The catch was, I needed to grow /var, which was quite full. The intention had been that once I'd got the two new disks in and all synced up, I'd create a fourth mirrored partition, and add this to LVM as a new physical volume, then I could just keep merrily growing things. It felt a little bit dirty having to make a new partition to get at the extra space, and I'd reading about mdadm's "grow" mode, whilst I was sitting around waiting for things to sync. So I thought to myself, I'd just delete the third partition, and recreate it using the remaining additional cylinders I now had at my disposal, then use this grow feature of mdadm to tell Linux the array was now bigger, then resize the physical volume, and go from there. This is where things went a bit pear-shaped. I deleted and recreated (larger) the third partition on each disk, and rebooted. I was then messing around with mdadm trying to update /etc/mdadm/mdadm.conf, since the UUID of /dev/md2 had changed, when I discovered that /dev/md2 had completely disappeared. It seems that recreating that partition made Linux completely fail to see /dev/sda3 and /dev/sdb3 as RAID-1 members, and so because of the fairly transparent nature of Linux's RAID-1, LVM had seen the underlying physical volumes, and just decided to run with /dev/sda3 as the physical volume, and everything had continued to work. I just had no redundancy any more. So it was about 3:20pm at this stage, and I was starting to panic, thinking I'd never get this sorted out before I had to leave, and I'd end up returning to the US with a non-redundant mess on my hands. Then I remembered that I had the backups of the partition tables of the old disks, from before I removed them. So I thought I'd go back to that state, and just have the additional cylinders unallocated. I hadn't yet attempted to grow the RAID-1 (partly because it had vanished), so in my haste, I didn't see this being a problem. The problem was, I'd booted into multi-user mode, and things had merrily been operating on just half the mirror. I restored the partitions, and rebooted, and lo and behold, /dev/md2 magically reappeared, already 100% consistent according to /proc/mdstat. I was somewhat surprised by that, but I booted back into multi-user mode again, only to discover that the /var filesystem was panicking and remounting itself into read-only mode. This was pretty weird, given that during boot, it was considered clean by e2fsck. So I was really getting bothered by this stage, so I rebooted into emergency mode (single-user mode in Debian starts way too much), and manually started up LVM, and forcibly fscked everything. There was a bit of filesystem damage to /var, /home and /srv. Thinking I'd now fixed the problem, I rebooted into multi-user mode again, but again /var, which was considered clean during the fscking/mounting phase of booting, panicked shortly after mounting, and remounted itself read-only. So I rebooted again, broke the mirror, and forcibly fscked the filesystems again (only operating on what was on /dev/sda3. Then I rebooted without incident into multi-user mode and resynced the mirror. At this point I ran away. So I'm really annoyed with myself. This should have been a blog post about how crap Linux's software RAID is at error recovery, and how LVM saved the day yet again, but it was overshadowed by me overreaching in my haste, and managing to cause silent data corruption. The morals of this story: I think that is all. I'm just so pissed off that what could have been a perfect (enough) upgrade with zero data-loss, even considering I had a bum disk, wasn't.

Clint Adams: Y la copla y la luz

That was the best cover of the Super Marios Bros. theme song ever.

Ian Wienand: Playing with the x86 PMU

A discussion about commas lead to an excuse to have a play with the IA-32 processor performance managment unit (PMU). To start, take two versions of a program to count the number of commas in a text file — one in C and one in Python. The C one runs faster on the input data set of ~60MiB of random data, but why? The CPU performance monitors give are the key to getting some idea of where the programs spend their time. I like to use perfmon2 because it's what I know, but Oprofile can do it too. All the available events for IA-32 are described in the manual; I currently of no better way of finding out about them than just reading it. On Itanium I reccommend Caliper which, for the most common situations, does most of the work for you and presents it in a nice report. Intel's Vtune also does a similar thing. The first thing to investigate is if the CPU is getting enough instructions to keep busy. The IFU_MEM_STALL metric is a good place to start as it is triggered when the instruction fetch pipeline is stalled, presumably waiting on either the ITLB or the trace buffer (Icache). 
$ pfmon -e CPU_CLK_UNHALTED,IFU_MEM_STALL ./comma < ./randomcommas
340559375 CPU_CLK_UNHALTED
   192115 IFU_MEM_STALL
$ pfmon -e CPU_CLK_UNHALTED,IFU_MEM_STALL python -Sc "import sys; print sum(l.count(',') for l in sys.stdin)" < ./randomcommas
1287100
4571257047 CPU_CLK_UNHALTED
 71981750 IFU_MEM_STALL
That works out to 0.05% of total cycles for the C version and 1.5% for the Python version, neither of which sets off immediate warning bells. If it did, we could start drilling down to things like the L2_IFETCH and ITLB_MISS events, or the BR_* branch events to try and see why the CPU is having to wait to get its next instruction. Next it is useful to find the CPI (cycles per instruction). This is calculated by the ratio of retired instructions against the number of CPU cycles; since a superscalar machine can issue more than one instruction per cycle this should ideally be much greater than 1 (for example, an Itanium can execute up to 6 instructions each cycle). 
$ pfmon -e INST_RETIRED,CPU_CLK_UNHALTED ./comma < ./randomcommas
542953593 INST_RETIRED
340612036 CPU_CLK_UNHALTED
$ pfmon -e INST_RETIRED,CPU_CLK_UNHALTED python -Sc "import sys; print sum(l.count(',') for l in sys.stdin)" < ./randomcommas
1194455205 INST_RETIRED
4569931735 CPU_CLK_UNHALTED
This works out at a CPI of 1.59 for the C version and 0.26 for the Python version. The Python version is clearly spending a lot of time waiting, because it isn't even able to issue one instruction every cycle. At this point it seems the CPU has enough instructions to do, but it is sitting around waiting to get through those instructions. This suggests the waiting is related to getting data from the cache. The load and store requests from the L2 cache are accounted to the L2_LD and L2_ST events respectively. These have the ability to mask out cache lines in different states of the MESI protocol, but for this we don't care so just ask pfmon to show us everything. 
$ pfmon -e L2_LD:M:E:S:I,L2_ST:M:E:S:I ./comma < randomcommas 
102505 L2_LD:M:E:S:I
   167 L2_ST:M:E:S:I
$ pfmon -e L2_LD:M:E:S:I,L2_ST:M:E:S:I python -Sc "import sys; print sum(l.count(',') for l in sys.stdin)" < ./randomcommas
3278774 L2_LD:M:E:S:I
  10457 L2_ST:M:E:S:I
This shows us that the Python version does quite a few more stores than the C counterpart. Considering this program should simply be reading the input stream and counting the number of commas, we do not expect much store traffic at all. This suggests the Python version is doing some extra copying, for whatever reason (maybe some Python expert can pinpoint it?). We can drill down a bit more into the memory related latencies. The DCU_LINES_IN event gives the total number of lines allocated in the cache. Another event, DCU_MISS_OUTSTANDING, gives a weighted measure of the cycles spent waiting for a cache line to be brought in. Each cycle spent waiting is weighted by the number of outstanding cache misses (I think the Pentium M I'm using can have up to 4 cache miss requests outstanding at once) and has some caveats, but can be considered a rough estimate of the time spent waiting for a cache line to be brought in. Therefore divding DCU_MISS_OUTSTANDING by DCU_LINES_IN gives us an approximate metric of how long a cache miss takes. 
$ pfmon -e DCU_MISS_OUTSTANDING,DCU_LINES_IN ./comma < randomcommas 
769736 DCU_MISS_OUTSTANDING
102387 DCU_LINES_IN
$ pfmon -e DCU_MISS_OUTSTANDING,DCU_LINES_IN python -Sc "import sys; print sum(l.count(',') for l in sys.stdin)" < ./randomcommas
99810150 DCU_MISS_OUTSTANDING
 4240179 DCU_LINES_IN
So that works out to 7.5 cycles for the C version and 23 cycles for the Python version. This seems to strongly suggest that it is memory traffic that is weighing the Python version down. That is only the initial phase; the results give a high level idea of why one program is running slower than the other. The initial analysis phase generally consists of taking the ratios of certain events to try and get some idea in your head of what the program is doing. Then comes the really hard work; drilling down to figure out how to fix it! Some useful references:

20 December 2007

MJ Ray: Current Issues / Severn Barrage Opposition Gathers

The Wildfowl and Wetlands Trust is hosting a meeting about wildlife concerns over the Severn barrage tonight at their place further up the river.  I've heard these concerns mentioned vaguely whenever the dam is discussed, but I don't really know what they are - does anyone?

The press release is a bit short on details and a quick look around the websites of the invited organisations didn't show anything obvious, except for Severn Barrage is not the answer, News: Tidal energy projects must not override existing environmental protection and Engineering our future nature: the dilemma of the Severn barrage on the WWF Cymru site.  Does that cover all the concerns, or are there more?

I'm still sort of surprised that the nation is seriously considering trying to put a barrier against the second-largest tidal flow in the world (Canada's Bay of Fundy seems to be the largest) and Britain's longest river, but I guess that's how our energy use is these days.

Lucas Nussbaum: Re: Is it hard for new contributors to help Debian? Can we improve things a bit?

Andreas Schuldei writes about my blog post about making it easier to contribute to Debian:
It is much more gratifying for a contributor if his effords have
immediate effects.

[…]

So what can people who want to help Debian do to achive more imminent gratification? Pick projects that let you help directly with direct svn/git/whatnot access (Security team, Debian Edu, Debian-Installer…)
While I agree that Instant Gratification is very important, I don’t think that the proposed solutions solve anything. It’s easy to have things sleep in an SVN repository instead of the BTS (lots of teams do that, sometimes for good reasons, like the lack of sponsors - hint: Games team). It doesn’t make things any better. What we need is tasks whose results will be available without too much wait in Debian unstable, for everybody. Also, I must admit that I know very little about the inner workings of the Security Team. But I have the idea that it involves quite a lot of procedures (helping with bugs tagged security probably doesn’t, but going further than that probably does). Contributing to Debian already involves A LOT of procedures. A new contributor, even very good technically, can easily get lost in all the different ways to package stuff and solve common problems. So we should identify tasks that don’t have huge requirements. Ubuntu has the concept of bite-size bugs. I started working on a page explaining the different ways someone can contribute to Debian. The goal is to provide a good entry point (not something that replaces existing documentation) and keep it at a manageable size. Its focus is mainly to replace the emails we write to people asking “I’d like to contribute, but I don’t know what I can do or where I should start!”. The page is available on wiki.d.o/HelpDebian/Start. Don’t hesitate to improve it (see also wiki.d.o/HelpDebian for a TODO list). When it will be good enough, I plan to move it out of the wiki (the content shouldn’t change much anyway) to eg http://help.debian.net/, so we can improve the design (I really like the idea of adding pictures of past Debconfs like Christian Perrier did in his FOSS.in talk).

Riccardo Setti: new metacity package

hey guys,  here in italy are 3.02 a.m and i'm still drunk (as my friend XX said:"i have a problem with alcol.. i like it") , but i made a good thing for all the crazy guys who run experimental, i've build latest metacity (the GNOME window manager). The cool thing of version 2.21.5 (experimetal) is that it is build with the composition manager enabled by default. all my changes are in the debian pkg-gnome svn, but i'm not sure if i should upload it right now, so feel free to take it on my home on people . 

19 December 2007

David Welton: Android Task List / Selector

Credit for finding out how to do this goes to dims, who has been doing lots of cool Android hacking lately: http://davanum.wordpress.com/2007/12/18/android-task-manager-primitive-prototype/ If you're interested in Android and what it can do, you should be following his online journal! As a way to keep pushing and testing Hecl, I decided to see if I could copy what he'd done in Hecl. It has indeed produced a few improvements to the Java code. That done, making it work was short and sweet: 
set layoutparams [linearlayoutparams -new  FILL_PARENT WRAP_CONTENT 
set layout [linearlayout -new $context]
$layout setorientation VERTICAL
# Create ourselves some commands to access Android internals.
java android.app.ActivityManagerNative activitymanagernative
java android.app.IActivityManager iactivitymanager
java  android.app.IActivityManager$TaskInfo  taskinfo
java android.content.ComponentName componentname
java java.util.List javalist
set am [activitymanagernative getdefault]
set tasks [$am gettasks 10 0 [null]]
set len [$tasks size]
set tasklist [list]
for  set i 0   < $i $len   incr $i   
    set baseactivity [[$tasks get $i] -field baseactivity]
    lappend $tasklist "Task: [$baseactivity getpackagename]"
 
$layout addview [textview -new $context -layoutparams $layoutparams \
         -text "Currently running tasks"]
set lview [basiclist $context $tasklist -layoutparams $layoutparams]
$layout addview $lview
$lview requestfocus
set callback [callback -new [list [list SelectTask]]]
$lview setonitemclicklistener $callback
heclcmd setcontentview $layout
proc SelectTask  parent view position id   
    [activitymanagernative getdefault] movetasktofront $position
It could have been squeezed further by avoiding some of the temporary variables, and perhaps also by implementing a mapping between java.util.List and Hecl lists. I think one of the next things I want to do is work on some infrastructure so that people will be able to actually work on scripts on their phone. That's sort of a departure from the past (most phone keyboards would be sheer torture for inputting long scripts), but it's a cool demo in any case.

Russell Coker: Bad Project Management

I have just read a rant by Sean Middleditch about bad project management [1]. He describes his post as “personal, rather angsty, and especially whiny” but I think it’s useful and informative. He makes some interesting technical points about PHP programming (I wasn’t aware that there were so many ways of easily getting things wrong and having difficulty to get them right). But of course this isn’t all limited to PHP, the web site WorseThanFailure.com has anecdotes about mistakes of similar calibre being implemented in every language imaginable. Sean is apparently considering leaving the computer industry after having numerous bad experiences of having highly paid people mess up projects while he gets paid a lot less to try and fix the worst of the bugs and get the systems working in production. I understand what it’s like, I have occasionally idly contemplated leaving the industry after bad projects. However the fun of working on free software combined with the amounts of money that I can earn in the computer industry made me quickly abandon such ideas. His stories in some ways resemble my experiences in working as a contractor, most of my contracts have been profoundly weird for various reasons (I’ll use the WTF [2] category of this blog to document some of them). I had two theories as to why I ended up in so many strange contracts, one was that I was in some sort of Twilight Zone and the other was that taking contracts based on the amount of money offered puts you at high risk of being employed by people who have no financial pressure to do things in a sensible manner. My advice to anyone in such a situation is to try and find a contract position paying an unreasonable amount of money. Getting more than $80 an hour (the rate Sean cites as being paid to the idiots who cause problems) is going to be difficult, but getting $50 or $60 an hour is much easier to achieve and should be enough to alleviate the pain of working on doomed projects.

Share This

Michal &#268;iha&#345;: Looking for Windows testers of Wammu

After half day of playing with Wine, MinGW and friends, I managed to be able to automatically build Windows versions of both python-gammu and Wammu. However I do not have environment to actually test these versions (except for Wine), so I would really welcome if some Windows users could do some testing. I expect some strange crashes at beginning, but hopefully we will be able fix it to produce working Wammu version for Windows in future. You can get the installer here: http://tmp.cihar.com/wammu-0.26-setup.exe It's NOT final or stable version, but as the only changes which happened since last release were either minor or related to cross compilation, it should be equal to combination of Gammu 1.16.0, python-gammu 0.24 and Wammu 0.25, so don't be confused by versions. PS: Okay, when application works in Wine, it does not mean it will work in Windows, see bug report for details.

18 December 2007

Martin F. Krafft: Taxes and wars

I was considering importing my car to Switzerland. When I moved in 2002, I asked the customs office about the process and was told that I'd need to pay taxes and fees, so I postponed the import until Day Unknown. Today, a very friendly customs official told me that this is not true and that I could have imported my car duty-free if I had done so as part of my move. Slightly annoyed, I went on to discover what those duties would be today. I found out that there are seven different types of fees I'll have to pay, and tried to understand the reason for these fees:

  1. To import a car, you need a form 11.010, downloadable from the Swiss Customs Office. Because each download causes extra work for the administrator, they charge a nominal fee of CHF 0.70 to get the PDF file.

  2. I take that form to the customs office. There, the officials will assemble around the car and attempt to lift it. I don't know why they do that, but they charge you between CHF 12 and CHF 15 per 100kg of weight. I think it must be recompensation for the hard physical labour. Also, each customs official can lift 100kg, which explains those figures they seemed a bit arbitrary at first.

  3. Next, they'll ask 4% of the estimated value of the car as a car tax. They do this because the value of a car is directly related to the area it covers, and since Switzerland is so small, you effectively have to buy a couple of square metres for the car. It's not quite as easy as that though: once I bought the area under my car, I have to move the car away, because I cannot park my car on the area I just bought. I have to buy parking space for that. If I don't want to buy a parking space, I have to keep the car moving. In order to keep the car moving I need to fork over CHF 40 per year for a permit to drive the car on Swiss highways.

  4. Now, I pay 7.6% value-added tax. It makes no difference that the car isn't new, that I am not selling or buying it, nor that I have already paid 16% VAT in Germany when I bought it. Since I am about to "consume" the car in Switzerland, I have to pay consumer tax. Since I also consume the physical labour the officials put into the car lifting exercise, as well as the area under the car I bought, I pay VAT for (car value + lifting tax + area under car). Unfortunately, the European Union will not pay back the consumer tax I previously paid as part of the export, since I already consumed the car there.

  5. I then get a report about the import. This costs CHF 20. I don't need this report, but I have to buy it. The reason is that the computer admins are so overworked serving PDF files to future importers that they have not had time to stop the computers from printing this report, and the special paper and toner used by all administrative offices in Switzerland costs about CHF 20 per page. An alternative explanation is that CHF 20 is the smallest bit of "money" in Switzerland, everything else is "change".

  6. I will also have to pay annual taxes for possession of a car, the value to be determined by the custom officials. Since I already paid for the area under the car while moving and while stationary, I think the annual car tax is levied for environmental reasons. A lot of people drive through Switzerland to get from Germany to Italy or from Austria to France, pumping a lot of exhaust gas into the Swiss air. This makes them guilty, just like inhabitants of Switzerland who own a car are guilty. Due to legal reasons, the Swiss government can only punish Swiss people for those sins, so the annual car tax is a little higher to cover for all environmental damage. It is doubtful where the money goes, though. I also have to continue paying taxes in Germany, because I still have a residence with my parents.

  7. Finally, I have to purchase a car insurance. That insurance does not insure the car, it insures the others on the road. Actually, it is a driver-car-dependent insurance, meaning that the others are insured only if I am the driver of my car. If I drive another car, or someone else drives my car, the others will not be insured. unless I pay higher insurance fees. Because cars with a larger area under them and those that require more people to lift them are more dangerous, the insurance fee is proportional to the car density. At least I can give up my German insurance and be covered by the Swiss insurance. However, I cannot keep my German insurance and be covered in Switzerland. I think this is because the head of customs department and the CEO of the insurance company have sex with each other.

It's been a great day, given how much I learnt. I finally understand why people have fought wars over taxes. I think I may leave the car in Germany. PS: Yes, this post may be considered hyperbole. NP: The Pineapple Thief: 8 Days Later

17 December 2007

Thijs Kinkhorst: Greetings from Delft

The two principal thigs I despise:
  1. People with folding bikes on the train. (They're very common here; I'm not sure how well known this concept is internationally, but it seems quite a self-explanatory term to me.)
  2. People that complain that up-market restaurants "always serve such small portions".
On a more positive note, I'm here in Delft for Rudy's promotion tomorrow. The hotel is set in a pervasive seventies style (deliberately), but still does offer wireless internet.

16 December 2007

Joey Hess: The Man from Earth

I turned this movie off because it became clear in the first few minutes that it was a very low budget movie shot in a hurry[1] with actors who reminded me of ones from not particularly awesome plays[2]. Then for some reason I went back to it. And soon it got interesting. And then I got thuroughly sucked in. And then I was really sad that the storytelling was over. I think it's been a very long time since I was sad that a science fiction movie was over[3]. Maybe 1980? This movie is set all in one room in a cabin, and has no higher technology in it than a cell phone, and yet it's amoung the top two purest peices of science fiction I've seen performed. And the lack of special effects, especially CGI[4], was a very nice change. There's a good chance you'll not enjoy this movie, but I hope you get a chance to see it and find out. You won't find it in theaters, there's a DVD and it's on Netflix, or you can download it and paypal the director and cast a few bucks.
[1] In about a week as it turns out.
[2] Which is probably true of some of them. I only recognised one.
[3] Snip: silly-yet-true-analogy-between-typical-SF-movies-and-bags-of-potato-chips
[4] Snip: hating on most CGI

15 December 2007

Zak B. Elep: To Australia and Back: OSDC Brisbane 2007

Its been a couple of weeks since the Open Source Developers’ Conference in Brisbane, but it was only now that I’ve got the time to blog about it. :) This is due to some work that got piled in my backlog (I haven’t been lazy enough to have this work all automated yet :(). First off, my trip to Brisbane was a long one, due to the UNDP travel agency giving me a route taking me to Bangkok first, because of my exchange grant’s papers made in short notice. No matter: this being my first ever international trip, this meant that I would be visiting two different countries in just one shot. Bangkok was just an overnight stay though, so I didn’t see much, but their new airport was nevertheless impressive. Arriving there at almost midnight, the only bright places I saw were the big Suvarnabhumi airport and the Novotel where I had my layover. After some sleep, I had a pleasant morning, but only a short one at Bangkok, as my flight to Brisbane followed after: DSC06879.JPG The Brisbane flight was long, and I arrived just an hour before midnight. Now, my next problem was getting a place to stay. I took an airport shuttle to the Royal on the Park, where the conference will be held, first to see if there are any free rooms, or barring that, some recommendation on nearby places to stay, even if only for the night. I was quite anxious because I wasn’t able to arrange for accommodation prior to this trip (short notice, remember?), so I decided to go directly to the venue to be the best way of trying my luck. And try my luck it was! First, I was greeted by a front desk manager who turned out to be a fellow Filipino, and he helped me get a room in the hotel. At the time though, he could not guarantee that I can stay there for the duration of the trip (only for the night) but he would help me to find alternate accommodation instead on the next day. I thought that was good enough, since it was already midnight by the time I reached the hotel, and just a bed to sleep on for the night would do great for me to worry about moving elsewhere later. And the next day: Day 1 of OSDC: DSC06880.JPG I was up rather too early that day; the worrying about moving elsewhere settled in too early as well, and had made me check out very early so I can hopefully look for that alternate accommodation by lunchtime. One by one, the attendees came, and soon enough more than a couple of hundred people were filling the main conference room, waiting for OSDC to begin. And we began with a keynote from Rusty Russell on C, the humbling language and Ian Clatworthy’s talk on distributed version control systems. After the keynote, I was notified by the hotel management that instead of moving elsewhere, I can just remain there for the duration of the conference. I was quite relieved! For the remainder of the day, I listened in on the Perl stream of talks from Kirrily Robert, Josh Heumann, and Jacinta Richardson, on packaging Perl modules, Perl 6, and Perl code optimizations, respectively. Kirrily’s talk was timely since I am writing a couple of Perl modules to be released on the CPAN (one, Tie::Amazon::S3, is now up ;) On the morning Day 2, I listened in on Mark Rees’ talk on testing web applications using twill and a scripting language, like Python. I then followed Adam Kennedy’s talk on the CPAN 1.5, learning about CPAN::Mini, the CPANPLUS (which IIRC will be standard in Perl 5.10) and the Tiny modules in the process. More on the Perl stream continued with another talk from Josh Heumann on intermediate Perl testing, providing a humorous yet informative strategies not really just for testing Perl scripts or modules but for software testing in general: DSC06888.JPG Later in the afternoon was Paul King’s talk on agile developer practices for dynamic languages, covering Groovy and Ruby. That was later followed by Leslie Hawthorn’s talk about the Google Summer of Code and the announcement of the Highly Open Participation contest for the young geeks who are not yet in college or university but would like to work on open source projects just like the SoC-ers. After the day’s schedule of talks, there was a break before the conference dinner, allowing also for a short keysigning session and CACert identity check. Then, we had the conference dinner, sponsored by Google. There was also a game held: every table makes up a team who, given some Play-dohs and imagination, are to make up a bug’; the best bug’ wins a free book. Unfortunately, my table/team didn’t win any books, but that was beside the point anyway, as the bugs that were made up were quite something: DSC06898.JPG DSC06895.JPG On the last day, the morning talks I attended were all about Ruby and Rails. Nic Williams gave an introductory talk on Rails, then Paul King gave another talk, this time on Grails (the Java version of Rails,) and closing the morning session was Keith Pitty’s talk on Ruby for Java shops. For the afternoon, I followed Adam Kennedy’s talk on optimizing projects for wetware. DSC06936.JPG For the closing keynote, Nat Torkington talked about the future of software: DSC06945.JPG And with a day left before returning home, I went to sightseeing mode, bought a couple of books, and the mandatory pasalubong for the relatives. At midnight, my flight took me back to Bangkok, another long trip that touched down at the crack of dawn: DSC07034.JPG After a few hours’ wait, I took the plane back to Manila, returning home just after lunchtime. This being my first international trip, making it alone seemed to be quite an adventure. But I wasn’t really alone in this endeavor: kudos goes to the UNDP International Open Source Network for their exchange grant program, to my relatives for their support, to Free and Open Source Software for really making things rock (and free ;), and to $DEITY for being there. Until the next trip!

Lars Wirzenius: EoC wiki and domain

Enemies of Carlotta, my little mailing list manager, now has a wiki, courtesy of Michael Durgener. It also has its own domain, courtesy of Andreas Barth. See: http://www.e-o-c.org/. There is also now a mailing list for commits to the EoC 2 development branch. To subscribe, e-mail eoc-commits-subscribe@liw.iki.fi .

Russell Coker: AISA

When I worked for Red Hat I joined AISA [1] (the Australian Information Security Association - formerly known as ISIG). Red Hat marketting paid for my membership so it was a good deal, I went to meetings (which often had free drinks), said good things about Red Hat security, and it cost me nothing. I was recently asked why I chose not to renew my membership, I didn’t have time to give a full answer so I’ll blog it now. AISA offers discounts on some conferences, books, and training related to computer security, if you plan to purchase such things then they do offer good deals. However I have little time to attend conferences at the moment, not enough time to read all the free Internet resources related to computer security, and feel no need to pay for such training. If at any time I plan to attend a conference where the discount for AISA members is equal or greater than the AISA membership fee then I can easily re-join. AISA membership seems largely to consist of managers and consultants not technical people or people doing R&D type work. This isn’t a bad thing if you are a manager or consultant, but when attending AISA meetings I don’t meet the type of people I meet at events such as SecureCon [2], Linux Conf Au [3], RuxCon [4], and the SE Linux Symposium [5] (which I think is not going to be held again for a while). Meetings of my local LUG [6] typically have more people doing serious technical work related to computer security than the AISA meetings I’ve attended. The AISA code of Ethics has as it’s second criteria “I will comply with all relevant laws“. Some laws can not be obeyed by decent people (study some German or Russian history or what is happening in China right now for examples). Many other laws should not be obeyed. Many countries (including Australia) have enacted many laws which should not be obeyed in the name of the “war on terror“. A final thing that irked me about AISA is their professional membership system (click on this link and download the AISA_Professional_Membership_Requirements_Nov_2006 document for details). It seems that I don’t qualify because I don’t have one of the listed certifications, and a public credit on the NSA web site [7] doesn’t count (yes, I asked about this). I’m not overly worried about this, I figure that any clique that won’t accept me also won’t accept a significant portion of the people that I want to associate with - so we can hang out elsewhere. I don’t recall there being any great benefit to professional membership apart from the possibility of adding it to your business card if you are so inclined (I don’t recall ever putting B.Sc [8] on a business card and don’t plan on adding anything less). There are some real benefits to AISA membership, but not for me.

Share This

14 December 2007

Martin F. Krafft: Finally on the right track

It's now three weeks since I am officially a Ph.D. student at the University of Limerick, but I've been too busy to do anything about it. Today, I finally updated my research webpage and sat down to tell you the tale of how I successfully defended my research, but failed to leave the country the next day. As is customary in the British system, I initially enrolled at UL as a Master student. In October, I then applied for a transfer to the Ph.D. track, submitted a transfer report and prepared for a live defence, according to the UL code of practice. On 23 November, I presented my research to my examiners, Dr. P r gerfalk and Patrick Healy, who found my work to be at Ph.D. level and approved of my transfer. I was moderately nervous going into the presentation, not really knowing what was expected of me. In the end, however, the presentation went well and the ensuing discussion provided valuable feedback. Thank you, P r and Paddy, as well as Norah (who chaired), and Brian (my supervisor) for all your time, help, and support. After the defence, Mel waited outside with a beer in hand, and it being Friday and all, I happily popped the cap and drank. Plans to head to The Stables gave way to chilling on the couch in 110 (my home away from home), before we headed to town for the Kila gig. We arrived early, had some beers, enjoyed the show, had some more beers, went home, and had some more beers to Guitar Hero, and when I crawled into bed, I was shocked to find out that I had lost all track of time and it was in fact already 5:30 on Saturday morning, instead of the perceived 2 o'clock. See, the source of this shock was a plane that left Dublin the next day around noon, and a rental car that was to get me there, leaving the house at 8. But when my alarm shattered my drunk dreams at 7:30, it quickly became clear (or well, everything was blurred, actually), that I would have a hard time getting into the car, and I would certainly never manage to navigate it to Dublin, a three hour drive. So I called up the airline and the rental company and rescheduled for the next day (only one flight from Dublin to Zurich a day), and went back to dreamland. I am entirely unsure whether my Ph.D. transfer defence was enough of a reason to get smashed. I still had a good time, though as always in Ireland, especially with this crowd NP: The Flower Kings: The Rainmaker

13 December 2007

Raphael Geissert: HOWTO create and maintain a repository using reprepro and DeBaBaReTools

DeBaBaReTools seems to work good enough that writing an early-stage HOW TO seems to worth the time :)
The first thing to do is of course fetch the sources:
$ svn export https://neo.yahostme.com/svn/DeBaBaReTools/trunk/ DeBaBaReTools


Build and install the package:
$ cd DeBaBaReTools; dpkg-buildpackage -b; dpkg -i ../debabaretools_0.2_all.deb


$ dpkg -L debabaretools
/.
/etc
/etc/DeBaBaReTools
/etc/DeBaBaReTools/common.conf
/usr
/usr/bin
/usr/bin/wannaBuild
/usr/bin/needsBuild
/usr/bin/needsReBuild
/usr/bin/buildQueue
/usr/bin/checkSums
/usr/bin/wannaUpload
/usr/bin/processIncoming
/usr/share
/usr/share/doc
/usr/share/doc/debabaretools
/usr/share/doc/debabaretools/copyright
/usr/share/doc/debabaretools/ENV
/usr/share/doc/debabaretools/changelog.gz
/usr/share/doc/debabaretools/TODO
/usr/share/DeBaBaReTools
/usr/share/DeBaBaReTools/backends
/usr/share/DeBaBaReTools/backends/config
/usr/share/DeBaBaReTools/backends/config/reprepro.sh
/usr/share/DeBaBaReTools/backends/probe.sh
/usr/share/DeBaBaReTools/backends/fetch
/usr/share/DeBaBaReTools/backends/fetch/dget.sh
/usr/share/DeBaBaReTools/backends/verbosity.sh
/usr/share/DeBaBaReTools/backends/lock
/usr/share/DeBaBaReTools/backends/lock/default.sh
/usr/share/DeBaBaReTools/backends/setDefault.sh
/usr/share/DeBaBaReTools/backends/buildQueue
/usr/share/DeBaBaReTools/backends/buildQueue/plaintext.sh
/usr/share/DeBaBaReTools/backends/signature
/usr/share/DeBaBaReTools/backends/signature/gpgv.sh
/usr/share/DeBaBaReTools/backends/check
/usr/share/DeBaBaReTools/backends/check/default.sh
/usr/share/DeBaBaReTools/backends/build
/usr/share/DeBaBaReTools/backends/build/pbuilder.sh
/usr/share/DeBaBaReTools/backends/upload
/usr/share/DeBaBaReTools/backends/upload/dput.sh
/usr/share/DeBaBaReTools/backends/parsers
/usr/share/DeBaBaReTools/backends/parsers/default.sh
/usr/share/DeBaBaReTools/backends/repository
/usr/share/DeBaBaReTools/backends/repository/shared.sh
/usr/share/DeBaBaReTools/backends/repository/reprepro.sh
/usr/share/DeBaBaReTools/backends/misc.sh
/usr/share/DeBaBaReTools/backends/codenamesTable
/usr/share/DeBaBaReTools/backends/codenamesTable/release.sh
/usr/share/DeBaBaReTools/backends/codenamesTable/reprepro.sh
/usr/share/DeBaBaReTools/backends/config/default.sh
/usr/share/DeBaBaReTools/backends/fetch/default.sh
/usr/share/DeBaBaReTools/backends/buildQueue/default.sh
/usr/share/DeBaBaReTools/backends/signature/default.sh
/usr/share/DeBaBaReTools/backends/build/default.sh
/usr/share/DeBaBaReTools/backends/upload/default.sh
/usr/share/DeBaBaReTools/backends/repository/default.sh
/usr/share/DeBaBaReTools/backends/codenamesTable/default.sh


Now you have to create a base reprepro repository. I strongly recommend setting the REPREPRO_BASE_DIR ENV variable if possible.

A recommended repository structure is (in repoitory/):
.:
./build:
./build/cache:
./build/data:
./build/data/logs:
./build/tmp:
./conf:
distributions
incoming
options
tiffany.py
uploaders
./db:
./debian:
./debian/dists:
etch
oldstable
sarge
sid
stable
testing
unstable
./debian/dists/...
./debian/pool/...
./incoming:
./incoming/accepted:
./incoming/new:
./incoming/rejected:
./pub:
./pub/UploadQueue:

Note that the build/, incoming/ and pub/ directory structures are DeBaBaReTools' defaults, if you want to change them you will have to set some variables at the configuration file ($HOME/.DeBaBaReToolsrc).

The distributions file must at least contain the Codename, Suite and Architectures entries for DeBaBaReTools to work correctly. Note that the current code used to correlate the suites with the codenames greps for the given codename/suite including two lines before and two after the match, so the entry must be on that scope.
E.g.
$ cat conf/distributions
Codename: sid
Suite: unstable
Architectures: i386 amd64 source
Components: main contrib non-free
Uploaders: uploaders
DebIndices: Packages Release .bz2 .gz .
DscIndices: Sources Release . .gz .bz2
Contents: .bz2 .gz .
Tracking: minimal includechanges keepsources


conf/incoming must name at least one rule. This is because when installing, DeBaBaReTools will grep for all the rule names and execute them one by one.
The rest of reprepro's setup is trivial.

When building packages with pbuilder, DeBaBaReTools expects the basetgz's to be named like $PBUILDER_CACHE/$CODENAME.tgz where $PBUILDER_CACHE is an environment variable defaulting to /var/cache/pbuilder.

Tweaking DeBaBaReTools:
Because we are using reprepro and pbuilder we'll give some hints to DeBaBaReTools so it uses a reprepro-specific and/or pbuilder-specific backend where possible.

$ echo 'HINT_ALL="reprepro pbuilder"' > $HOME/.DeBaBaReToolsrc


The .DeBaBaReToolsrc file is sourced by all scripts, so you might even set variables like $REPREPRO_BASE_DIR there.

/usr/share/doc/debabaretools/ENV documents only some of the multiple environment variables that can be specified to customise DeBaBaReTools' behaviour. It is important to say that variables which are empty will be filled with a default value.

You may now want to modify .dput.cf by adding a 'local' target for your repository:
[local]
fqdn = localhost
method = local
incoming = /home/raphael/Deb/repository/pub/UploadQueue
allow_unsigned_uploads = 0
run_dinstall = 0


So let the fun begin! :)

To make sure everything is ok let's run processIncoming in verbose mode:
VERBOSE=1 processIncoming
processIncoming v0.1 started.
processIncoming v0.1 stopped.

Not getting that output means something is not ok.

All the scripts will try to source a file called $HOME/.$ APP_NAME rc, where $APP_NAME is the name of the script, e.g. processIncoming. This can be used to customise them even more.

Tweaking the scripts behaviour:

processIncoming offer some options such as:


wannaBuild:



When you should call which DeBaBaReTools script:
processIncoming takes care of accepting package uploads and installing them when told to do so.
needsBuild processes the .changes files accepted by processIncoming and tells the build queue manager that a given package needs to be built, or when the binary package is also uploaded, or is already built.
wannaBuild only takes care of building source packages (note that arch: all-onnly packages WON'T be built, they must be uploaded together with the source package).

So the workflow is:
processIncoming -> needsBuild -> INSTALL=1 processIncoming

A package won't be installed in the repository if it isn't handled before by needsBuild.

Here's a demonstration (note the usage of VERBOSE=1):
$ dput local kcometen3_1.1-1_source.changes
Checking Signature on .changes
...
Uploading to local (via local to localhost):
Successfully uploaded packages.
Not running dinstall.
sh: /usr/bin/mini-dinstall: No such file or directory
Error while executing command.
Continuing...

The package is now at pub/UploadQueue

$ VERBOSE=1 processIncoming
processIncoming v0.1 started.
Processing /home/raphael/Deb/repository/pub/UploadQueue/kcometen3_1.1-1_source.changes...
mv'ing "/home/raphael/Deb/repository/pub/UploadQueue/kcometen3_1.1-1.diff.gz" to "/home/raphael/Deb/repository/incoming/accepted/"
mv'ing "/home/raphael/Deb/repository/pub/UploadQueue/kcometen3_1.1-1.dsc" to "/home/raphael/Deb/repository/incoming/accepted/"
mv'ing "/home/raphael/Deb/repository/pub/UploadQueue/kcometen3_1.1-1_source.changes" to "/home/raphael/Deb/repository/incoming/accepted/"
mv'ing "/home/raphael/Deb/repository/pub/UploadQueue/kcometen3_1.1.orig.tar.gz" to "/home/raphael/Deb/repository/incoming/accepted/"
processIncoming v0.1 stopped.

After checking sums and signature, the upload is accepted.

$ VERBOSE=1 needsBuild
needsBuild v0.2 started.
Architectures: i386 amd64 all
Now processing /home/raphael/Deb/repository/incoming/accepted/kcometen3_1.1-1_source.changes
Adding kcometen3 to be built in i386 for the sid distribution
Adding kcometen3 to be built in amd64 for the sid distribution
mv'ing "/home/raphael/Deb/repository/incoming/accepted/kcometen3_1.1-1.dsc" to "/home/raphael/Deb/repository/incoming"
mv'ing "/home/raphael/Deb/repository/incoming/accepted/kcometen3_1.1.orig.tar.gz" to "/home/raphael/Deb/repository/incoming"
mv'ing "/home/raphael/Deb/repository/incoming/accepted/kcometen3_1.1-1.diff.gz" to "/home/raphael/Deb/repository/incoming"
mv'ing "/home/raphael/Deb/repository/incoming/accepted/kcometen3_1.1-1_source.changes" to "/home/raphael/Deb/repository/incoming"
needsBuild v0.2 stopped.

The package is now marked to be built on i386 and amd64 for sid (note that distro to codename conversion is performed) and moved to incoming/ so processIncoming is able to install it.

$ VERBOSE=1 processIncoming
processIncoming v0.1 started.
processIncoming v0.1 stopped.

There's nothing under pub/UploadQueue and nothing is installed (do you remember the INSTALL=0 option? :))

$ INSTALL=1 VERBOSE=1 processIncoming
processIncoming v0.1 started.
No distribution found for 'kcometen3_1.1-1_source.changes'!
There have been errors!
No distribution found for 'kcometen3_1.1-1_source.changes'!
There have been errors!
No distribution found for 'kcometen3_1.1-1_source.changes'!
There have been errors!
making diffs between /home/raphael/Deb/repository/debian/dists/sid/main/source/Sources and /home/raphael/Deb/repository/debian/dists/sid/main/source/Sources.new:
generating diff
Exporting indices...
processIncoming v0.1 stopped.

The errors can be safely ignored, it is just that reprepro couldn't install kcometen3_1.1-1_source.changes for the specified install rule (I've one for each distro, that's why).

$ VERBOSE=1 wannaBuild
wannaBuild v0.1 started.
Fetching list of packages to be built for unstable
Fetching source package kcometen3 v1.1-1
Use of uninitialized value in string eq at /usr/bin/dget line 377.
cat: /home/raphael/Deb/repository/build/data/needsBuild.unstable: No such file or directory
Starting build backend...
Executing sudo pbuilder --update --basetgz /var/cache/pbuilder/unstable.tgz
Starting build process
Verifying signature of /home/raphael/Deb/repository/build/kcometen3jb3362/kcometen3_1.1-1_i386.changes...
gpgv: no valid OpenPGP data found.
gpgv: the signature could not be verified.
Please remember that the signature file (.sig or .asc)
should be the first file given on the command line.
Fetching list of packages to be built for testing
Fetching list of packages to be built for stable
Fetching list of packages to be built for oldstable
wannaBuild v0.1 stopped.

There are still some things that need to be fixed (e.g. the no such file or directory message).
Other than that, the output is just fine: the package wasn't automatically signed so when trying to check the signature it failed to find it thereby not uploading the package automatically.

Now we cd to build/kcometen3jb3362/, review the build log, the .changes file, and we finally sign it.
Afterwards, we upload the package and install it:

$ VERBOSE=1 processIncoming
processIncoming v0.1 started.
Processing /home/raphael/Deb/repository/pub/UploadQueue/kcometen3_1.1-1_i386.changes...
mv'ing "/home/raphael/Deb/repository/pub/UploadQueue/kcometen3_1.1-1_i386.changes" to "/home/raphael/Deb/repository/incoming/accepted/"
mv'ing "/home/raphael/Deb/repository/pub/UploadQueue/kcometen3_1.1-1_i386.deb" to "/home/raphael/Deb/repository/incoming/accepted/"
processIncoming v0.1 stopped.

$ VERBOSE=1 needsBuild
needsBuild v0.2 started.
Architectures: i386 amd64 all
Now processing /home/raphael/Deb/repository/incoming/accepted/kcometen3_1.1-1_i386.changes
mv'ing "/home/raphael/Deb/repository/incoming/accepted/kcometen3_1.1-1_i386.deb" to "/home/raphael/Deb/repository/incoming"
mv'ing "/home/raphael/Deb/repository/incoming/accepted/kcometen3_1.1-1_i386.changes" to "/home/raphael/Deb/repository/incoming"
needsBuild v0.2 stopped.


$ INSTALL=1 VERBOSE=1 processIncoming
processIncoming v0.1 started.
No distribution found for 'kcometen3_1.1-1_i386.changes'!
There have been errors!
No distribution found for 'kcometen3_1.1-1_i386.changes'!
There have been errors!
No distribution found for 'kcometen3_1.1-1_i386.changes'!
There have been errors!
making diffs between /home/raphael/Deb/repository/debian/dists/sid/main/binary-i386/Packages and /home/raphael/Deb/repository/debian/dists/sid/main/binary-i386/Packages.new:
generating diff
Exporting indices...
processIncoming v0.1 stopped.


done :)

We can now setup some cronjobs:
*/10 * * * * processIncoming
*/5 * * * * needsBuild
@daily INSTALL=1 processIncoming

needsReBuild and uploadQueue still need to be written, so don't expect them to do anything right now.

I hope people start using it so I receive more feedback :D

Martin F. Krafft: Delaying mail delivery

My current mailfilter has two features which increase my day-to-day productivity:
  1. a "tickler," which is a reminder system inspired by the tickler file component of David Allen's Getting Things Done action management method: I can submit emails and notes to the tickler along with a timestamp in the future, and the tickler delivers the mail (or note) to my inbox when the timestamp has passed.
  2. the ability to delay certain types of mail (e.g. Debian mail is held until the weekend, while news items and the like are only delivered at night).
My current implementations work alright, but they're brittle and crap. What's worse is that the two features could be combined and handled by one and the same tool, but I implemented them differently for now: The tickler consists of a Maildir, to which I can submit mails, either by mailing a note to a specific email address (currently broken, thus not linked from here), or by adding a tickle stamp (X-Tickle header) to a message with this script, saving it to the local tickler mailbox and asking offlineimap to shove it to the server. On this server, the tickler queue is regularly scanned by a script, which resubmits mail whose timestamp has expired to my mailfilter, where it is treated somewhat specially as a resubmitted mail. After almost three months with this setup, I can identify the following shortcomings: When I implemented the delay queue, I knew of these problems and went a different way: delayed mail is stored in a Maildir and a (msgid, timestamp, filename) tuple is inserted into an sqlite3 database on the mail server. This script regularly processes the queue. If I just sent you the chills, at least we have the same taste. There are numerous problems with this approach, the foremost being that Maildir filenames are not guaranteed to be constant: mails jump between the new and cur directories, and tags, such as seen are encoded in the filename (thus, symlinks also cannot be used). My script now uses an ugly heuristic, which at least makes it work. I should investigate whether inodes could be used instead as I think those wouldn't change throughout the lifetime of a mail, at least while it's not moved between folders. I initially considered just dumping messages to files and encoding the timestamp in the files' mtime, but then I would not be able to access the queue with mutt in case I needed to fetch a delayed mail prematurely, or if I wanted to synchronise the queue with offlineimap as well. The past few days, I've been condensing experiences from both approaches and am working out a new technique to combine both features. In essence, I think the database/index approach is the best, if I can figure out a way to uniquely identify mail message files, ideally across folders. Assuming I can use inodes for that, delayed mail would then be stored into the delayed Maildir and an (inode, timestamp) tuple saved into the database. Tickler mail would be stored along with all other mail in my store Maildir and would get a similar entry in the database. This approach solves some problems and leaves others. Assuming I synchronise the store Maildir remotely (which I do), then I can easily fathom making modifications via IMAP which causes orphan records in the database (if only IMAP would allow me to store key=value pairs for mails ). Furthermore, I'd have to submit mails to the tickler by bouncing them to an email address, and deleting the local copy, unless I want duplicates. If now the mail is somehow dropped, I've lost mail. Still unhappy about all of this, still searching for a better implementation, I'd appreciate any feedback! NP: Luminous Flesh Giants: Duma I Upadek

Philipp Kern: Up-to-date Debian Keyring

First of all cheers to James for a new keyring upload. If I knew that I would have updated my subkey expiry once again before, heh. And the changes mail is not yet available anyway, maybe it was too large? Neither the PTS, nor d-d-changes lists it (yet?). It's shown in "Latest version" in the PTS, though. Michal, I've got a simple script in /usr/local/sbin/update-debian-keyring which I run once in a while, as the keyring in the archive is outdated more often than not: 
#!/bin/sh
rsync -az --progress keyring.debian.org::keyrings/keyrings/debian-keyring.gpg /usr/share/keyrings/debian-keyring.gpg
Now this won't solve the who-uploads "problem" completely as the Debian Maintainer keyring is still missing. As far as I know it is not rsync-able (as it's not managed by the keyring powers but on ftp-master) and probably not even directly wget-able. At least the imported GPG blob is not put into the pool, just the source package is. It's available as a file for DDs in merkel.debian.org:/srv/ftp.debian.org/keyrings/debian-maintainers.gpg, though. Update: buxy told me that the keyring upload was done by noodles, not troup. Sorry about that, but I was unable to check that.

Next.

Previous.