Search Results: "ema"

6 January 2009

Julien Valroff: DSPAM-Community - first RC: call for testing

I was happy to receive Dov Zamir s email announcing the creation of a fork of the DSPAM anti-spam filter, called dspam-community. I have been using DSPAM for a while now, and am very happy with it, but was quite anxious that the project is left out by Sensory Networks (the company took over DSPAM in May 2007). I give all my confidence in the fork as the original author, Jonathan Zdziarski, is part of the new development team! I have prepared EXPERIMENTAL packages of the first RC of DSPAM-Community.

These are only meant for testing and should not be used in a production environment! I haven t tested these packages in depth (only tested installation or upgrade from previous official/unofficial packages).
There might still be problems due to the new names. I have built packages for i386 and amd64 architectures. They are built against current unstable distribution. They might be used on Ubuntu, but I haven t tested this yet. I have setup a quick page for these packages. All the applied patches are available for review and possible merge into git: http://packages.kirya.net/dspam-community/debian-patches/ To use these packages, add these lines to your /etc/apt/sources.list:
deb http://packages.kirya.net/dspam-community/ experimental main
deb-src http://packages.kirya.net/dspam-community/ experimental main
The repository uses secure-apt, you can hence check packages integrity thanks to my dedicated GPG key:
gpg --keyserver wwwkeys.eu.pgp.net --recv-keys FBABB737
gpg --export -a FBABB737   apt-key add -
or
wget -O - http://packages.kirya.net/Kirya.netDebianpackagesVerificationKey.asc   apt-key add -
Fingerprint: 97A0 69D9 A150 687E 599F F168 EEFB 43B2 FBAB B737 Feel free to test the packages and the first DSPAM-Community RC.
Any comment is welcome! To the official DSPAM packaging team: feel free to take these packages as a basis for your work in case you are interested in packaging DSPAM-Community!

Uwe Hermann: Running homebrew and open-source software on your Wii, using the Twilight Hack and the Homebrew Channel

Wii I've been owning a Nintendo Wii for quite a while now, but so far only played an occasional game of tennis or the like. Over the holidays I got The Legend of Zelda: Twilight Princess, which is a really nice game in itself. What is even more interesting though, is that this game allows you to run homebrew software on your Wii. What you usually want to do, is to use the so-called Twilight Hack to install the Homebrew Channel on your Wii (you'll need an SD card and the Twilight Princess game for this). Once the process has finished you won't need the game anymore though. So, here's a quick tutorial on how to install the Homebrew Channel. I basically followed the Setting up your Wii for Homebrew HOWTO from wiibrew.org. Requirements An SD card max. 2GB in size (not an SDHC card!), an SD card reader in your PC or laptop, and the Twilight Princess game (which you must play at least once). Backup First, you'll want to backup your Twilight Princess save games (if any). For that, we format the SD card (it needs a FAT16/FAT32) filesystem.
 $ mkfs.vfat /dev/mmcblk0p1 (you may need to change "mmcblk0p1" to whatever fits your setup)
Now insert the SD card into the Wii, start the Wii, go to Wii Options --> Data Management --> Save Data --> Wii and copy your Twilight Princess save games onto the SD card. Then, poweroff, remove the SD card, insert it into your laptop/PC and copy the save games to a safe place.
 $ mount -t vfat /dev/mmcblk0p1 /mnt
 $ mv /mnt/private ~/zelda_savegames
Twilight Hack and Homebrew Channel The Legend of Zelda: Twilight Princess Next up: download the Twilight Hack and Homebrew Channel, and put the files on the SD card:
 $ wget http://hbc.hackmii.com/dist/twilight-hack-v0.1-beta1.zip
 $ unzip twilight-hack-v0.1-beta1.zip
 $ cp -r private /mnt
 $ wget http://hbc.hackmii.com/dist/the_homebrew_channel-1.0.1.tar.gz
 $ tar xfvz the_homebrew_channel-1.0.1.tar.gz
 $ cp the_homebrew_channel-1.0.1/boot.dol /mnt
 $ umount /mnt
Now remove the SD card, insert it into the Wii and power-up the Wii. Go to Wii Options --> Data Management --> Save Data --> Wii, delete the Wii Twilight Princess save games, then copy the "Twilight Hack" save game from the SD card onto the Wii (you need to select the correct one for your region). Quit the menu, start Twilight Princess, load the "Twilight Hack" savegame and finally talk to the person in front of you (do not perform any other actions, or the Wii may crash!). Follow the remaining instructions on the screen and you're done. You now have an additional channel named Homebrew Channel where all your homebrew programs/games (and your own Free Software programs you may write/port) will appear. Installing homebrew software So far, there are no real applications in the Homebrew Channel, you need to put all the homebrew software you want on the SD card. There's a huuuge list of applications and games to choose from, many of them Free Software with source code, some however are binary-only. Basically, you put each application in a sub-directory of apps/ on the SD card, the most important file in every sub-directory is always boot.dol. If you have a boot.elf file instead, you'll probably need to convert it using the ELF to DOL converter. Restoring Twilight Princess save games You may now want to restore your Twilight Princess save games onto the Wii, you no longer need the "Twilight Hack" save game. Put your ~/zelda_savegames directory on the SD card and name it private/ again. Insert the SD card into the Wii and copy the save games on the Wii (similar procedure as above). Have fun with homebrew software on your Wii, or even better write your own software or port existing Linux applications/games!

Benjamin Mako Hill: Debian Bug Squashing at MIT

I was thrilled to be part of a successful Debian bug squashing party organized by MIT's Student Information Processing Board on December 13th. Greg Price, who helped organize the event, did a wonderful write up which he sent to the debian-devel email list. I though it was worth mentioning the BSP now because I think it's a wonderful model that I'd love to see replicated in Debian and beyond. The event was initiated, organized, run, and executed by people with little or no direct experience with the project. While the organizers went out of their way to recruit several Debian developers and other experts to the party, these experts' role was more in answering questions and helping others. The the majority of the participants -- around 30 of them in total -- had no previous experience doing Debian development. While the 11 bugs closed or dealt with are the most visible outcome, I'm not sure that it is the most important. The event acted as an important learning experience for everyone involved and, perhaps most centrally, an important first step for most participants from using free software to giving back and participating in the community. You don't need experience, connections, or a @project.org email address to organize a party like this for Debian or your own free software project. All you really need is a few people, some technical knowledge, an Internet connection, and the desire to make it happen. Other things can help, of course. In particular, the SIPB folks have packaged up some scripts they used to select bugs to work on and put them online.

4 January 2009

Jeff Bailey: ... 2008

I try to practice Inbox Zero, and this past year has been brutal far staying on top of my Inbox. When I started pruning the home email a week or so ago (having just gotten my work email box down to zero). It's giving me a pretty good review of 2008. And, umm.. wow:

Spinal tumor, surgery, hating Mountain View (the place, not the job there. My coworkers are pretty uniformly awesome), having a toddler, not being able to lift my toddler, crazy mood altering drugs, visa renewal, a funeral, unexpected pregnancy, weddings, buying a house, mid-term miscarriage, Angie's hospitalisation from that, pneumonia, a move home, having to withdraw from my Master's courses 3 times over the course of the year for medical reasons - eventually causing me to withdraw from the program. Extend it by another 3 months into 2007 (because while I'd love to believe that stress reset at the start of a calendar year...), and we have another wedding, a move to the US and a new job.

I've just gotten to an email where I took notes on what the risks were for the angiogram before I'd told many friends. There's a small pile of emails and such from people wishing me well that I never managed to reply to. I also find myself thinking back to the people I didn't hear from, and the reevaluation of my communities and friendships that came along with that.

The lyrics from Counting Crows come to mind: "It's been a long December and there's reason to believe that maybe this year will be better than the last."

Here's to 2009. =)

Andrew Pollock: [tech] Mixing electricity and water: monitoring the cat water bowl with Nagios

(this is something I've had "in production" for many months now, I just haven't had the time or energy to do a proper write up about what I did) We have a cat water bowl, it looks like this: The cat water bowl Under "normal" circumstances, it usually lasts about seven days. So when our weekly routine is happening, we'll refill it on a Saturday whilst doing house chores, and it'll last until the following Saturday when it gets refilled. Unfortunately, sometimes our routine gets disrupted, and we forget. Sometimes, we travel and have a house-sitter, who may not pay as close attention to such things as ourselves. Once, one of our cats was licking the condensation off a chilled bottle of soft drink that was on the kitchen counter one evening, before we realised the water bowl needed refilling. Naturally, we felt like terrible pet owners. So I think it was some time around the 2008 Maker Faire, that I hatched the idea of having some sort of water sensor on the cat bowl, which would communicate to one of the various computers in house. At the Maker Faire, I bought a copy of Making things talk, and an Arduino starter kit, which consisted of a Diecimila board and a make-it-yourself proto shield. I also bought a little electronics starter kit, which consisted of a breadboard and various components, and a USB-TTL cable. I decided to use Bluetooth to communicate with the board, as I already had my MythTV setup using a Bluetooth keyboard and mouse, and it was within range of the water bowl. I decided against using Zigbee, because I didn't know anything about it, and I didn't want to add (or learn about) yet another wireless infrastructure just for this project. I should point out that I know very little about electronics. I'd never owned (or really used) a soldering iron until I embarked on this project. I took a basic soldering class at the Tech Shop, but I'd already assembled the proto shield by the time I took the class, so I'd pretty much figured it out. I had a very naive vision that I could just basically shove two wires in some water and it'd close a circuit and that would be my water sensor. Of course this didn't work, so I started hunting around on the Internet for a circuit that would do this. I happened upon a circuit (I don't seem to have retained the URL, so I can't link to it), which just consisted of a couple of transistors and some resistors. So I headed off to Fry's to try and buy the transistors I needed. I quickly discovered that I didn't have sufficient information to identify the transistors that I wanted, but I did happen to stumble upon a cheap assemble-it-yourself water alarm. It consisted of a PCB, and a transistor and some resistors and a buzzer. I bought a couple of these instead. Between studying the PCB and the circuit diagram that came with the alarm, I was able to reproduce it on my breadboard instead of on the PCB. Sure enough, placing the two probes in water closed the circuit. I replaced the buzzer with an LED so I could see what was going on. I attached the circuit to the Arduino proto shield, and had it feed into one of the digital I/O ports. I wrote some quick and dirty Wiring code so that when water not present (i.e. the circuit was open and no current was detected on that I/O port) the LED was switched on. Really at this point, I didn't need a microcontroller, I could have presumably achieved the same thing with a NOT gate. At this point, I wanted to make the sensor remotely queryable. I bought a BlueSMiRF Silver Bluetooth modem, which I attached to the TX and RX lines of the board (I first configured it by attaching the USB-TTL cable to it and using Minicom on my laptop). I extended the Wiring code to provide a rudimentary prompt, and accept a command to check if water was present. I think around this time it also dawned on me that I could use the digital I/O pins as a switch. When they're "high" they provide power. So rather than constantly running a current through the water, I only needed to briefly power up the water detection circuit, see if the circuit closed or not, and then report if water was present if it did. I much preferred this, as at the time, I was endeavouring to power the whole sensor off a 9 volt battery. I figured I'd get much better battery life if I wasn't running a current through the water the entire time. I should point out that I did some "tongue tests" in a glass of water while the circuit was powered up, and couldn't detect a difference between when the circuit was on or off. The last thing I wanted to be doing was zapping the cats! At this point, the Wiring and Arduino work was pretty much complete. I setup ser2net on the MythTV server, so that I could just connect to port 4000, and be connected to the water sensor.
apollock@icarus:~$ telnet teevee 4000
Trying 172.16.0.9...
Connected to teevee.andrew.net.au.
Escape character is '^]'.
ser2net port 4000 device /dev/rfcomm0 [9600 N81] (Debian GNU/Linux)
waterbowl> s
Water is present
waterbowl>
telnet> q
Connection closed.
apollock@icarus:~$
The Wiring code running on the Arduino board is checked in here. Next, I wanted to monitor this with Nagios. One thing I found with the Bluetooth connection was that it wasn't all that reliable. Not every connection to port 4000 resulted in a connection with the water sensor. I elected to write some standalone code that submitted results to Nagios by way of a passive check, rather than having Nagios try to actively monitor it. Again, trying to conserve energy, I decided to only check the sensor once every 8 hours. So I wrote a Python daemon, which tried to be as robust as possible. If at first it doesn't make a connection on its 8 hourly schedule, it keeps trying until it does. Nagios itself has some freshness detection for this monitor, so if no passive result is submitted within 8 hours and one minute, Nagios alerts that something is possibly wrong with the sensor itself. (The original intent was to deal with the situation where the battery went flat) This is the Nagios service definition I've got. Some of it may be unnecessary or redundant:
define service  
        host_name                       teevee
        service_description             Cat water bowl
        check_command                   check_stale!2!"Check water bowl monitor is on and reachable"
        normal_check_interval           480
        notification_interval           240
        active_checks_enabled           0
        check_freshness                 1
        freshness_threshold             28860
        max_check_attempts              1
        check_period                    24x7
        use                             generic-service
        stalking_options                o,w,c
        contact_groups                  everyone
 
The code for the Python daemon is checked in here. So everything was going swimmingly, apart from I couldn't get even 24 hours of monitoring on my conservative 3 times a day schedule, and the Bluetooth modem configured for all of its power saving options, and the water sensor itself only being powered on when it was performing a check. There wasn't much more I could do to try and reduce power consumption, at least with my limited electronics knowledge. Perhaps using Zigbee instead of Bluetooth would have helped, but I still think I'd have been going through 9 volt batteries more quickly than I'd have liked. I also had a brief foray into solar powering the board. I thought maybe the lighting in the house would be sufficient to power the board. Of course this didn't work out. I'd also have needed to make the code running on the board more self-sufficient, and have it just provide a water status indication whenever it had enough power to do so, instead of being a fairly dumb device like it currently is. This all felt a whole level more complicated, and out of my league, and I wasn't interested in attempting this sort of remote-sensing exercise at this time. The Arduino board can also be powered by USB, so as I already had some long USB type A to type B cables (that had funky lights in them to boot), I decided to just get a wall wart that had a USB type A receptacle, and powered the Arduino board that way. So much for being completely "wireless". (I could have also gotten a general purpose DC power supply that was capable of spitting out 9 volts, but I doubted I'd get one with a sufficiently long cable, which is why I went for the USB-powered option, as I already had a cable long enough) Speaking of wires, the most challenging part was the probes for the water sensor. As they were going to be permanently in the cats' drinking water, I didn't want to contaminate the water with them. I figured plain untinned copper wire would be okay, since water pipes are copper. Finding untinned, unstranded copper wire was a real challenge. I started out using some FM antenna cable, but that was stranded, and the shielding was a nightmare to strip. It was also reasonably difficult to make go where I wanted it. What I really wanted was something more solid, that would flex and stay where I put it. I cannibalised a spare IEC power cable, but it was also stranded copper wire. I finally managed to obtain some solid-core CAT-5 cable from a hardware store, and this has worked exactly as I wanted. I haven't done any further work on the setup since getting the CAT-5 cable for the probe. Further improvements that I'd like to do at some point: The finished product I had a lot of fun with this project. I had a real sense of achievement, being able to go from concept to completion, and learn a few things about electronics along the way. I'm normally not a fan of messing around with hardware. Some photos of the project are here.

Neil Williams: Installing Emdebian Grip with D-I

A few experiments have led to a partial route to pre-seeding Emdebian Grip from a standard Debian Lenny ISO. A few issues arise:



Overall, it is possible to install Emdebian Grip - with a little help - using Debian Installer. A few more changes should make it possible to select a standard Emdebian Grip installation without X and an Emdebian Grip XFCE desktop.

Remaining issues include:


A bespoke config package can be arranged for this purpose - it won't be compliant with Debian Policy, at least as far as the status of Policy for Lenny.

Lots more to do but Lenny is still in-progress so there's time to get a usable installation before FOSDEM.

3 January 2009

John Goerzen: Garmin Nuvi 500

I ve owned two different GPSs in the past: a Garmin GPS III, and the Garmin eMap. Both are based on similar underlying firmware. This week, I did some research on more modern GPS units and decided to buy a Garmin nuvi 500. Here s my review. Overview The Garmin nuvi 500 is one of only two models in the nuvi line that are waterproof. The nuvi 500 and 550 are both designed as hybrid units: useful both in the car and outdoors. The 500 includes street-level maps (they appear to be the same quality as Google) for the entire United States, detailed topographical maps for the entire United States, and a global basemap. It also includes a microSD slot (or is it miniSD - I forget) for additional maps that you can buy anything from marine maps to other countries. It also includes a default POI (points of interest) database containing over 5 million points: restaurants, gas stations, parks, hospitals, you name it. Most contain an address and a phone number in additional to the coordinates. Unlike GPS units that you find built in to some cell phones, this is all stored on flash memory on the unit: no Internet connection required. The nuvi 500 is a portable yellow pages, topographical map, and incredibly detailed street atlas all in one. Car Use The nuvi 500 comes with a car charger and suction cup windshield mount in the box. (It doesn t come with an AC charger, but it can charge over USB.) It also comes with yay a user-replaceable battery. The windshield mount is very sturdy and I am happy to have one that isn t permanent. In the car, the device performs admirably. I have read some other reviewers that have compared its routing to other GPSs and found that the nuvi generally picks the same route as Google, and almost always a better route than other GPSs. If you deviate from the selected route, it automatically re-calculates a new route for you. It will show you the next turn coming up, and has either 3D or flat map displays. There s a one-touch where am I feature. It displays your current coordinates, some sort of street address suitable to read to someone ( 1234 S. Main or I-95 exit 43 type of thing), along with buttons to show you the nearest police stations and hospitals. The unit also features multi-stop routing. You can either tell it where all you re going in a defined order, or tell it all your stops and let it create an optimal route. This feature works, but it your stops are close by (and involve some of the same roads), it may wind up skipping some stops thinking you ve made your first one. The speaker announces upcoming maneuvers, though it doesn t have a synthesizer for street names. It s also supposed to work with Bluetooth headsets, though I haven t tested that feature. I found the map quality to be excellent. It seems to be on par with Google (in fact, I think both use data from Navteq). I was surprised with how many country dirt roads and side roads it knows about and of course, regular city streets are all on there, with one-way indications and the whole lot. In fact, the quality of coverage was so good that I was surprised when it missed roads or got things wrong. Out in rural areas, or small towns, this happens from time to time: it thought that an abandoned railbed was an unnamed road and wanted me to drive on it (it was clearly not drivable), and in some other cases also thought some abandoned roads were still drivable. I don t think this is a device issue though; it s an underlying data issue, and everyone else probably has the same problem. Its arrival time estimates are quite accurate, and its interface is smooth and easy. It has some optional accessories I haven t tried, such as real-time traffic reports from wireless sources, boating mode, etc. Outdoor and Geocaching Mode Our other main use for the device is outdoor hiking and geocaching. It really shines here. It has special support for the GPX files from geocaching.com. The device supports paperless caching . Not only can it put caches on the map, but if you download the GPX file for your caches, you ll also get the full description, hint (behind a separate button, of course), most recent 5 logs, and the like right there on your screen. You can also log your finds on the device, and upload a file from it to geocaching.com later to log them on the site. This is an incredible time saver over my old method: printing out a bunch of maps, downloading waypoints to the eMap, taking notes, then logging things later. I found outdoor mode not quite as refined as the auto mode, however. It kept forgetting that I wanted to use an arrow as my current position indicator (the default hiking boots for walking mode didn t provide an accurate enough direction indication for my tastes). Finally realized that saying don t ask me about the configuration was the key to getting it to remember the configuration. It sometimes took a surprisingly long time to realize we weren t standing still any longer. On the other hand, the quality of the GPS receiver was amazing. It even got a strong signal in my house. And I wasn t even sitting at a window. The topographical maps are a nice addition, and the breadcrumb mode is always helpful when geocaching and hiking, too. My natural way of holding it meant that I accidentally turned it off a few times, because I had a finger holding it right over the power button. But it powers back up and re-obtains the signal quite rapidly. The different modes (automobile, outdoor, bicycle, and scooter) are mainly different collections of settings: 3D map or flat, what indicator to use, to navigate off-road or along roadways, to ignore one-way indications or heed them, etc. PC link The nuvi 500 has a USB port. Plug it into the PC, and you see its internal flash drive, vfat formatted. You can upload and download GPX files there, store photos if you like. Be careful what you mess with though, because its built-in maps and system data are also on that drive. If you have the SD card inserted, that will also be presented to your computer when you plug the device in. Garmin has some Windows software to make it easier to upload/download stuff, but I haven t tried it. Annoyances As you can tell, I really like this GPS, but there are a few things about it that annoy me. The #1 thing that annoys me isn t actually the GPS itself, but Garmin s website. I went to register the device online, but it wouldn t let me because I don t have the Windows-only browser plugin on my Linux machine, and it wanted to talk to the GPS for some reason. I went to send them a support request about that, only to discover after I had typed in the request that their email us form is broken in Firefox on all platforms. Bad show, Garmin. The on-screen keyboard (it s a touch-screen device with only one hard button for power) isn t QWERTY layout; it s alphabetical layout, and it makes me inefficient when entering data. I found myself logging my finds on the unit, but taking notes about them on paper because that was faster. Garmin has a feature listed on their website for a toggle between QWERTY and alphabetical layout, which they apparently offer on only their more expensive GPSs. What? There is no reason to not but that in all your firmware. The device lacks a few features I was used to on my GPS III and eMap. It doesn t support any kind of real-time position indication to the PC; all communications is just accessing stored data on the internal flash drive. I used to think that was a nice feature, but in reality, I haven t used it in years. It also lacks the display of the exact location of each GPS satellite, though the incredible quality of its receiver means that I don t really care any more. (I used to use that information to help figure out which window to put it by if in a car/train or something.) It also lacks the level of configuration that was present in the settings screens on the older units. There s no battery saver mode (sample every 5 seconds when going at a constant velocity instead of every 1) like the older units had. The sun and moon screen likewise is gone, but added is awareness of timezones; the Nuvi 500 can show you the local time at your present position synced up with the GPS satellites. The compass is not the most helpful screen, though after some practice, it is functional. The documentation about it is confusing, but really the thing that was more confusing was this: in walking mode, the arrow that indicates what direction you re walking in updates faster than the arrow that indicates what direction you should be walking in. Once I realized what was going on there, it was easier to use. The compass does tell you what your bearing is, in degrees, but only when you are not seeking a destination. It will not tell you the bearing to the destination, though you can estimate it from the simulated compass face on the display. When seeking a specific point, especially through terrain with obstacles such as trees, it is useful to be able to use a compass for the final approach because a GPS unit can t tell you which direction it s pointed only which direction it s moving, so when you re not moving or moving slowly, it s not helpful. I did some experimentation today with the compass screen, as well as my real compass, and was able to navigate to my destination rather precisely using the combination of them. That said, a more functional compass screen would still be better. Conclusion Overall, I m very happy with the nuvi 500. It s not the same as a top-of-the-line device in either the outdoor or automotive category, but on the other hand, it s cheaper and more convenient than buying two devices with similar features. The geocaching features are excellent, the build quality is excellent as well. The system is stable and performs well. (Some other reviews worried about whether the case is solid enough; it seems quite solid to me.) I wish there were a faster way to toggle between 3D and flat map views, and forgetting about my walking mode icon is annoying, but other than that I have very little to complain about. Garmin s geocaching features (found on this unit and several other in their lineup) is great.

Andre Luis Lopes: Em poca de crise, mesmo blog, mas com motor recauchutado

Para n o fugir ao clich , aproveitei os feriados de final e in cio de ano para atualizar o Wordpress e fazer uma limpeza b sica no blog (troca de temas, desabilita o/habilita o de plugins, etc). Ainda estava utilizando a vers o 2.3.2 e atualizei direto para a vers o 2.7. Aparentemente, tudo foi bastante tranquilo. Demorei mais para fazer backups (como medida de precau o, n o que fosse algo estritamente necess rio) do que para concluir totalmente o processo de atualiza o. A equipe do Wordpress est de parab ns por tornar a experi ncia de atualiza o algo bastante suave, ainda mais em meu caso, com um pulo de uma vers o jur ssica diretamente para a ltima. Agora que o Wordpress est usando e abusando de conceitos Web 2.0/AJAX tudo ficou mais intutito. A interface de postagens, por exemplo, ficou bem mais amig vel em minha opini o. Outra novidade foi que o ScribeFire voltou a funcionar 100% agora, mas com as melhorias sens veis na interface de postagens provavelmente vou acabar abandonando-o. Quem sabe agora com as novas facilidades eu n o me anime a voltar a postar com maior frequ ncia ?

2 January 2009

Michael Casadevall: Notes from Underground, Part 1

For those following d-devel, you may notice that I've recently been working on improving one of the cornerstones of Debian infrastructure; the Debian Archive Kit, or dak for short. Most DDs and DMs don't notice dak exists expect when trying to determine why their latest upload was rejected, and then yelling at the powers that be. I'm here to shead some light on this mythicial beast.

First off, a quick history lesson:

dak (also known as projectb) is a replacement for Debian's original archive software, known simply as dinstall. dinstall itself was a fairly large perl script that does what dak process-unchecked/process-accepted does today. James Troup did a fairly decent summary of dinstall, and its issues

James Troup's README.new-incoming (from dak's git repo):

The old system:
---------------
o incoming was a world writable directory

o incoming was available to everyone through http://incoming.debian.org/

o incoming was processed once a day by dinstall

o uploads in incoming had to have been there > 24 hours before they
were REJECTed. If they were processed before that and had
problems they were SKIPped (with no notification to the maintainer
and/or uploader).

dak's first commits were in 2000, and rolled out onto ftp-master.d.o sometime in 2001 or 2002 (I can't find an exact date for this). Since then, dak is also used on security.d.o, and on backports.org (fun fact for bpo people; the dak installation there is now up to date, and tracking git's tip).

So now that you know the history lesson, what specificially does dak do is the next question. Simply put, dak is the glue that binds the rest of the Debian's backends together; both britney and wanna-build/buildd depend on it. It handles management of uploads to the archive, handles stable release updates, as so forth. It is also the only Debian archive software that uses an actual database backend, and scales fairly well handling over 10,000 packages, and 12 architectures. Unfortunately, there are also a lot of issues with dak as it stands.

Sections of the code base have bitrotted over the years; legacy and legacy-mixed support have died, the import-archive function is shot (more so now than ever, see below), the test suite is non-functional (never a good sign), the docs are out of date, and in many places non-existant, doing a release (both point and full) requires editing the database and so forth.

In addition, dak, while written in python, is written in a fairly procedural style, and and some very ugly code in some places. For instance, the original Debian Maintainer code was handled by having the uid's in the database prefixed by dm: vs having a flag somewhere, and had some hardcoded variables like checking for "unstable", as well as quite a few bugs which caused interesting behavior when uploading to a non-unstable suite such as experimental or one of the proposed queues. (for those of curious, I recommend checking the dak git tree to see what the old DM code looked like, and then aside from the design, find the two major bugs which caused a lot of the weirdness with DMs). It should be stated that the last merge from redid the DM code and design sanely using the new update framework.

These issues have lead to the genesis of the dak v2 project, which is an attempt to replace dak with a module, rewritten from the ground up to be more secure and modular, although its not gotten very far as of writing. I personally don't believe that the current iteration of dak is so bad as scrapping and rewritting is necessary. Instead, I've been working to implement v2 features in dak by aggressive refactoring and cleanup, with the hope of negating the need for a rewrite.

So now thats out of the way, I bet you probably are interested in my .plan for dak. Well, lets go over I've implemented so far.

* An update database framework for dak, which will allow for easy database upgrade and migration, vs the "does it work yet?" approach to applying schema updates. Simply type dak update-db, and your done!

* 822 formatted output for queues (http://ftp-master.debian.org/new.822); this information is now used on DDPO pages

* Rewriting DM management code to have more of a brain than the previous implementation.

What's next on the TODO list

* Content file generation from the database (part of removal of apt-ftparchive, but thats another blog post ;-)).

Oh, as a side note to my current readers, my blog has changed names to "Notes from Underground", after one of my favorite novels, and futher in reference to exploring the mysterious underground that is Debian's backend code. We're also now on Planet Debian :-).

Joey Hess: new year

I'm trying to work on having days that are somehow individually memorable this year. So far.. 0 (leap day) Finally tackled the chapter on monads. I'd read various explanations a year ago, but was swimming in syntax I didn't understand. After percolating for a year, and learning to read the syntax better, monads turned out to make very simple sense. (I can't say the same about Johnny Monad.) I had been meaning to write sometime about a method I used in ikiwiki to let expressions in a mini-language, that normally are evaluated to match a set of pages, instead be evaluated to explain why they succeed or fail. It's a cute technique, though hard to explain. Now I'm pretty sure it's just a monad. So I don't have to explain it! 1 Visiting Abram's falls this time of year, the canyon is in constant wintry shadow. The falls are not frozen, but have icicles twice my height, and there are rank upon rank of icicles all down the walls, an ice cathederal. It's a bright sunny day, but on the whole hike, I only get into the sunlight once, briefly, at the top of the giant steps. Then back into the shade. Back at my car, I'm suprised that it's only 3 pm, feels like it should be 5. Made a pecan pie with daddy's pecans and eggs. 2 A grey day with snow and worse. The paper's rss feed repeats "dozens^Whundreds of wrecks" over and over, as if to make up for there being no 60 point type. I'm reading Ted Nelson's book Geeks Bearing Gifts. The chapter summaries seem better than the actual book. And the on-demand printing makes me think I'm reading a poorly laid out web page, rather than something typeset. But I love that he goes all the way back to the invention of the alphabet and of hierarchical categorization and suggests all the basis for modern computers is arbitrary and/or wrong. Eating ginger duck downtown I look up and a pizza delivery guy has slid out of control right in front of me and crashed.

Wouter Verhelst: Sven Luther and Debian mailinglists

A few days ago, I forwarded a mail from Sven Luther to the debian-vote and debian-project mailinglists. The reason I did so was that, apart from the last two paragraphs, the mail's tone was reasonable, polite, and did not try to rehash the old arguments. Also, and more importantly, I thought it was a useful contribution to the discussion at hand. I then received a private email in response to that forward from someone who shall remain nameless that expressed disapproval of this forwarding. Not because of the content (which i could understand), but because of the person whose mail I forwarded. I am shocked beyond words that such a thing is possible. Mind you, I'm not supporting at this point that we just allow Sven back in the project; the decision to throw him out was based on solid arguments, and though I am sad that it was necessary, I do support the decision that has been made. However, I would think we would have expelled him because of the problems that resolved around his person, rather than because we didn't like his name, background, or, say, the color of his skin. Problems can be solved. Personalities can change. Throwing a person out of the project because of interpersonal problems is one thing. Forbidding him to ever join it again is quite another. I'm not saying we should just forget the whole thing, pretend it never happened, and move on. However, I do not think that just because the man has a history, we should ignore his useful contributions. Although I disagree with some of the suggestions he made, I do happen to think that this particular contribution was useful, and so I forwarded it. I do disapprove of some of the things he's done, but that doesn't mean I disapprove of the person; I do believe any person deserves a minimum of respect. Apparently some people in this project disagree with me on that, however. I'm not sure what I'll do about that, but suffice to say that I will not be part of a project that considers a person's history to be of more importance than a person's current behaviour. Let's hope this is just the opinion of one person...

MJ Ray: New Years Resolution: Avoid These Bad Ideas?

I m not a big new year resolution maker (new year is just a side-effect of a calendaring system), but if you are, I suggest the following resolutions to avoid three bad ideas:
  1. Change your passwords and do not set it to the ship number for the Starship Enterprise, or any other of The Top 500 Worst Passwords of All Time - if you re a sysadmin, make sure you are comparing user passwords against these sort of lists;
  2. Fight against cinema-style ratings for web-sites and the campaign against free speech [because of] a wider public interest which that implies;
  3. (Debian developers) Vote against the proposal for future GRs would need 30 other [DDs, not people - the proposal is wrong] to support because There seems little rationale to support it. The more I ve looked, the more places I ve not found evidence for such a large seconding requirement and I know a few anecdotes about raising numbers too high and accidentally killing an organisation
Happy New Year!

1 January 2009

Jeff Bailey: Launchpad email noise finally crushed!

Launchpad's email handling is awful. Truly awful. I left a pile of teams I was on to reduce the noise, and vaguely considered leaving the core-dev team so that I would never have to see another "merge request" come through that I couldn't possibly ever care about.

But I have solved the problem!

I remembered that my email can cope with jbailey+FOO@raspberryginger.com and still delivery it, giving me something to filter on. So effectively immediately, all mail from launchpad now skips my inbox is tagged in case I do actually need it. The only thing I need to improve this is a feature to auto-delete emails after 30 days labeled with a certain tag.

What a lovely start to 2009.

Sandro Tosi: Compress MySQL Tables

I got this long running task that (fsck!) died almost at the end of the job, and it logs its operation in a MySQL table.

Since I want to keep those "partial" results for further inspection (even after I'll fix and let the tool run again until the end) I don't want to waste disk space (it's ~650mb), so why don't compress that table?

With MySQL (and MyISAM tables like mine), it's pretty easy: go into the directory of the db schema:
# cd /var/lib/mysql/tabs_compare/
# ls -l
total 659500
-rw-rw---- 1 mysql mysql 65 2008-09-21 00:35 db.opt
-rw-rw---- 1 mysql mysql 12840 2008-12-25 00:44 log_1230162297.frm
-rw-rw---- 1 mysql mysql 675299244 2009-01-01 20:34 log_1230162297.MYD
-rw-rw---- 1 mysql mysql 1024 2009-01-01 20:37 log_1230162297.MYI
Pretty big :) Now, check the table:
# myisamchk log_1230162297.MYI
Checking MyISAM file: log_1230162297.MYI
Data records: 4293747 Deleted blocks: 0
- check file-size
- check record delete-chain
- check key delete-chain
- check index reference
- check record links
Hopefully there is no error, so we can compress that table:
# myisampack log_1230162297.MYI
Compressing log_1230162297.MYD: (4293747 records)
- Calculating statistics
- Compressing file
43.05%
And the result it's really nice:
# ls -l
total 375572
-rw-rw---- 1 mysql mysql 65 2008-09-21 00:35 db.opt
-rw-rw---- 1 mysql mysql 12840 2008-12-25 00:44 log_1230162297.frm
-rw-rw---- 1 mysql mysql 384558282 2009-01-01 20:34 log_1230162297.MYD
-rw-rw---- 1 mysql mysql 1024 2009-01-01 20:49 log_1230162297.MYI
The compress command didn't even take a full core on my machine and almost took 20secs to complete.

Now rebuild indexes (if any) and recheck the table to be error free:
# myisamchk -r log_1230162297.MYI
...
# myisamchk log_1230162297.MYI
...
now tell MySQL to reload those table with:
mysql> flush tables <schema>;
Keep always in mind that compressed tables are read-only, so no DML commands, while DDL are allowed (so you can DROP it, for example).

Ana Beatriz Guerrero Lopez: New Year s proposal: look at your reported bugs!

Happy New Year everybody! I have a little New Year s proposal for you all, users and developers. Take a look to the bugs you have reported in Debian, this can be very easily done by browsing:
http://bugs.debian.org/cgi-bin/pkgreport.cgi?submitter=USER@DOMAIN.NET
where USER@DOMAIN.NET is the email address you used for reporting. Then update the bug reports information in the following way: Alternatively, you can look at the bugs of your favorite package and help the maintainer. Go to http://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=PACKAGE replacing PACKAGE with the application name, look at the bugs and send a email to BUGNUMBER@bugs.debian.org with all the information you can gather about the bug: if you can reproduce it (and if yes, how), versions affected, also you can make a patch :) etc. Above, I only gave basic hints of how to interact with the Debian BTS, if you want to know more, read the documentation at http://www.debian.org/Bugs/.

Biella Coleman: 2008

When I return to PR, my blog tends to lie fallow. Between taking care of the house, outdoor distractions (like the very overgrown garden and the very enticing beach), and most especially visiting my mother, I turn away from the computer, from my analytical mind, from writing. But since we have edged into a new year, I thought I would spend a little bit of time ruminating on 2008, which was one of the most unforgettable for me. With each passing year, it seems that 12 months bleed right into the next 12 months. But within this swirl, there are a handful of years that retain their sharp edges (1992, 1996, 2005, 2005-2006 are the most memorable). 2008 now sits among those as it is the the year that a tube was placed into my mother s stomach and soon after she was taken from the hospital into a nursing home without even the chance to say goodbye to the dog who was her steadfast companion and the house, which was her home for over 20 years. Over the last 6 years while I have not written much about my mother and her Alzheimer s, I did so fairly consistently with at least 3 entries per year, that is until March 2008 when she was put in the nursing home. Soon after being admitted, I wrote one entry and since then have not had the heart to commit much to words, something of which I am ashamed of and have sworn I would write about at least once before another years passes us by. I don t have the desire to constantly write about my experiences with her but it seems to be that Alzheimer s is erased far too often from public view and I would rather not contribute to that type of very pervasive silencing either. The recipe for my silence has been made up of various ingredients. One was I was just really really busy. Last semester was frenetic so I often did not have an ounce of room/time to think, much less blog about such a difficult situation (and I spent a lot of time going back and forth to PR as well). Another is that I was trying to avoid the heartache that arises when you focus in, with some detail, care and attention, on something that is unavoidably painful. Each time I tried, I ultimately failed and told myself, I would do so later. Finally I was blogging less and less about these types of subjects so it just seemed a little out of place every time I tried to commit something to words. For me, one of the hardest things to accept and this probably fed into my silence is that I played a part in prolonging her life when I know she wanted out (with a capital O U and T). But at the time the feeding tube was placed in her, she was walking and talking and it did not seem right to refuse her food (and I hoped that with some food, she would then start to eat on her own as she can swallow just fine) not to mention that her caretaker (a devout Catholic) and my sister would have never at the time been ready to refuse the feeding tube and I was not exactly quite ready either. Well, what we did not foresee was that 5 days in the hospital did her in and when she left, she could no longer walk, she could barely talk, and yet due to the feeding tube, she gained back the dramatic weight that she had shed over a year so that her body at least looked far more alive than it had been for years thanks to the steady stream of nutrition being slowly pumped into her body. I just know she would not want to be alive under these conditions and yet at this point, there is not much I can do legally except refuse certain classes of treatments if she gets sick (such as respirator) or wait until she becomes completely vegetative and then go through the legal hoops, that ultimately may not be successful, to remove the feeding tube. It has been so hard for so many months that as I late I have been trying to approach the whole thing with at least one dash of Zen-like detachment (by which I mean an embrace/acceptance of suffering in this world) and just spend as much time as I can with my mother, which I did a lot over the summer and I am now doing this winter. I am most probably moving back over the summer as well as my sister is not faring all that well after being with her for so many years. Even though she no longer knows my name, or who I am, I can tell she notices my presence and there are days she even seems to be happy. In fact, over the summer, she had some pretty lively days when we could manage to joke about almost nothing. Those days were rare and precious and today they don t seem to happen at all (she did seem a bit excited that it was a New Year though I could not tell if she really knew what I was saying). Over the summer, generally I would climb into her bed and we would co-nap for a couple of hours. And then the worst were the days she was hysterical and crying, and the absolutely worst was when she was crying, pleading for her parents. Those were days that did not break my heart but my soul. Thankfully those days of hysterics seem to be happen less and less and my mother just seems to sleep a lot. I sometimes like to think she is like those really old cats or dogs who seem to spend interminable hours sleeping and resting and don t seem to be suffering all that much though I can still tell she is still in a bad mood from time to time. She also has a wonderful roommate, Ana, who also pretty advanced Alzheimer s though certainly not as bad as my mom. Unlike my mother, she is happy as a clam, so much so she is always saying things like Isn t this a great day? How wonderful the day is, etc. etc, which is a nice counterbalance to my mom! I sort of like that my mom s roommate is, personality wise, the polar opposite of my mother, who is not a fundamentally unhappy soul, but she was emotionally erratic one with quite a few peaks and valleys and this is still very much the case. Having Ana around (and her family) has transformed the way I have seen this illness and my mother s condition. I am certainly not thrilled that she is alive and in fact think how we end (or don t end) life is something we as a society should be ashamed of. I clearly want to see her released, freed like a bird from the cage of this world (my mother loved loved birds I think because they symbolized freedom), but spending time with Ana, who again seems quite content, made me realize how part of the Alzheimer s experience is rooted in one s personality. My mom has always been difficult and when she was sick but still quite functional, her personality made the illness difficult for her and those around her. She never really had much luck changing her karmic personality and it sure is not going to change now. But as I said she does seem to rest a lot. I just hope her last few months, possibly years, are if nothing else peaceful and tranquil.

Biella Coleman: 2008

When I return to PR, my blog tends to lie fallow. Between taking care of the house, outdoor distractions (like the very overgrown garden and the very enticing beach), and most especially visiting my mother, I turn away from the computer, from my analytical mind, from writing. But since we have edged into a new year, I thought I would spend a little bit of time ruminating on 2008, which was one of the most unforgettable for me. With each passing year, it seems that 12 months bleed right into the next 12 months. But within this swirl, there are a handful of years that retain their sharp edges (1992, 1996, 2005, 2005-2006 are the most memorable). 2008 now sits among those as it is the the year that a tube was placed into my mother s stomach and soon after she was taken from the hospital into a nursing home without even the chance to say goodbye to the dog who was her steadfast companion and the house, which was her home for over 20 years. Over the last 6 years while I have not written much about my mother and her Alzheimer s, I did so fairly consistently with at least 3 entries per year, that is until March 2008 when she was put in the nursing home. Soon after being admitted, I wrote one entry and since then have not had the heart to commit much to words, something of which I am ashamed of and have sworn I would write about at least once before another years passes us by. I don t have the desire to constantly write about my experiences with her but it seems to be that Alzheimer s is erased far too often from public view and I would rather not contribute to that type of very pervasive silencing either. The recipe for my silence has been made up of various ingredients. One was I was just really really busy. Last semester was frenetic so I often did not have an ounce of room/time to think, much less blog about such a difficult situation (and I spent a lot of time going back and forth to PR as well). Another is that I was trying to avoid the heartache that arises when you focus in, with some detail, care and attention, on something that is unavoidably painful. Each time I tried, I ultimately failed and told myself, I would do so later. Finally I was blogging less and less about these types of subjects so it just seemed a little out of place every time I tried to commit something to words. For me, one of the hardest things to accept???and this probably fed into my silence is that I played a part in prolonging her life when I know she wanted out (with a capital O U and T). But at the time the feeding tube was placed in her, she was walking and talking and it did not seem right to refuse her food (and I hoped that with some food, she would then start to eat on her own as she can swallow just fine) not to mention that her caretaker (a devout Catholic) and my sister would have never at the time been ready to refuse the feeding tube and I was not exactly quite ready either. Well, what we did not foresee was that 5 days in the hospital did her in and when she left, she could no longer walk, she could barely talk, and yet due to the feeding tube, she gained back the dramatic weight that she had shed over a year so that her body at least looked far more alive than it had been for years thanks to the steady stream of nutrition being slowly pumped into her body. I just know she would not want to be alive under these conditions and yet at this point, there is not much I can do legally except refuse certain classes of treatments if she gets sick (such as respirator) or wait until she becomes completely vegetative and then go through the legal hoops, that ultimately may not be successful, to remove the feeding tube. It has been so hard for so many months that as I late I have been trying to approach the whole thing with at least one dash of Zen-like detachment (by which I mean an embrace/acceptance of suffering in this world) and just spend as much time as I can with my mother, which I did a lot over the summer and I am now doing this winter. I am most probably moving back over the summer as well as my sister is not faring all that well after being with her for so many years. Even though she no longer knows my name, or who I am, I can tell she notices my presence and there are days she even seems to be happy. In fact, over the summer, she had some pretty lively days when we could manage to joke about almost nothing. Those days were rare and precious and today they don t seem to happen at all (she did seem a bit excited that it was a New Year though I could not tell if she really knew what I was saying). Over the summer, generally I would climb into her bed and we would co-nap for a couple of hours. And then the worst were the days she was hysterical and crying, and the absolutely worst was when she was crying, pleading for her parents. Those were days that did not break my heart but my soul. Thankfully those days of hysterics seem to be happen less and less and my mother just seems to sleep a lot. I sometimes like to think she is like those really old cats or dogs who seem to spend interminable hours sleeping and resting and don t seem to be suffering all that much though I can still tell she is still in a bad mood from time to time. She also has a wonderful roommate, Ana, who also pretty advanced Alzheimer s though certainly not as bad as my mom. Unlike my mother, she is happy as a clam, so much so she is always saying things like ???Isn t this a great day???? ???How wonderful the day is,??? etc. etc, which is a nice counterbalance to my mom! I sort of like that my mom s roommate is, personality wise, the polar opposite of my mother, who is not a fundamentally unhappy soul, but she was emotionally erratic one with quite a few peaks and valleys and this is still very much the case. Having Ana around (and her family) has transformed the way I have seen this illness and my mother s condition. I am certainly not thrilled that she is alive and in fact think how we end (or don t end) life is something we as a society should be ashamed of. I clearly want to see her released, freed like a bird from the cage of this world (my mother loved loved birds I think because they symbolized freedom), but spending time with Ana, who again seems quite content, made me realize how part of the Alzheimer s experience is rooted in one s personality. My mom has always been difficult and when she was sick but still quite functional, her personality made the illness difficult for her and those around her. She never really had much luck changing her karmic personality and it sure is not going to change now. But as I said she does seem to rest a lot. I just hope her last few months, possibly years, are if nothing else peaceful and tranquil.

Dirk Eddelbuettel: R/Finance conference in Chicago in April: Call for Papers

The following went out to the R-announce and R-SIG-Finance mailing lists a few days ago. The conference already has a very strong lineup of invited speakers, and we are now asking R / Finance users from both academia and industry to submit suitable one-page abstracts:
Call for Papers The Finance Department of the University of Illinois at Chicago (UIC),
the International Center for Futures and Derivatives at UIC, and
members of the R finance community are pleased to announce R/Finance 2009: Applied Finance with R on April 24 and 25, 2009, in Chicago, IL, USA Confirmed keynote speakers include:
Patrick Burns (Burns Statistics)
David Kane (Kane Capital)
Roger Koenker (U of Illinois at Urbana/Champaign)
David Ruppert (Cornell)
Diethelm Wuertz (ETH Zuerich)
Eric Zivot (U of Washington)
We invite all users of R in Finance to submit one-page abstracts or
complete papers (in txt/pdf/doc format). We encourage papers both on
academic research topics and related to use of R by Finance practitioners.
Presenters are strongly encouraged to provide working R code to accompany
the presentation/paper. Datasets need not be made public. Please send submissions to committee@RinFinance.com.
The submission deadline is January 31st, 2009.
Submissions will be evaluated and submitters notified via email on a rolling basis. Additional details about the conference will be announced as available. For the program committee:
Gib Bassett, Peter Carl, Dirk Eddelbuettel, John Miller,
Brian Peterson, Dale Rosenthal, Jeffrey Ryan
See you in Chicago in April!

31 December 2008

Andrew Pollock: [tech] On unlocking a Vonage-provided Linksys RTP300

Well that was a bit of process. The best set of instructions I found were here. I think a lot of the other instructions I found assumed you were at firmware version 1.00.62. The CYT unlocker program seems to only work with this firmware version. The CYT unlocker program is a Windows console-mode program. It seems to run under Wine, but I switched to Windows before I realised that possibly my problems running it under Wine were related to the firmware version on the RTP300 and not a problem talking to the network. All of the remaining steps on the above page could have been performed on Linux. In fact, the CYT unlocker program merely makes a HTTP POST to a specific URL (and I think it could just as easily be an HTTP GET) and then starts up a webserver on port 2400 to hand an XML blob of provisioning information when the RTP300 requests it as a result of the HTTP POST. A Linux version would be easy to implement. In all of the futzing around, I noticed this gem in the HTML source of the web interface:
*********************************************************
*   Copyright 2005, CyberTAN  Inc.  All Rights Reserved *
*********************************************************
This is UNPUBLISHED PROPRIETARY SOURCE CODE of CyberTAN Inc.
the contents of this file may not be disclosed to third parties,
copied or duplicated in any form without the prior written
permission of CyberTAN Inc.
This software should be used as a reference only, and it not
intended for production use!
Now that I've unlocked it, I guess I should try and make it talk to my Asterisk box...

Biella Coleman: odd-old-san-juan-dog



odd-old-san-juan-dog, originally uploaded by the biella.
We may not have did up my house like this OSJ one but you did a fine job > thanks (and thanks for laughing with me over this odd dog and his portrait!)

Next.

Previous.