Search Results: "ed"

11 July 2010

Thibaut Girka: Fifth GSoC report

Ok, here is the (rather short) fifth report! First, and most important (but not so interesting, as I haven't had any review yet): I've submitted the glamo drivers usptream! There is still a long way to go before having a working Debian kernel on the FR, but I think it's still an important step (well, even more important would be its acceptance in mainline). Another thing is the uboot-envtools-udeb/uboot-installer thing. I haven't started it yet, but I have a rough idea of how to do that. Last but not least, the installation "script". It is meant to run on the host, and takes care of pushing necessary files to the FR and boot it, working around any issue ("slow typing" u-boot for instance). I've designed it in an extensible way, but now, I wonder if I've overdone it. However, it's here, and it's easy to strip down if we want something simpler.

Thorsten Glaser: Back home

Bordeaux was very nice (and towards the end much cooler it s actually hotter here at more than 50 north too warm to think, or do anything) but the LSM/RMLL was very french. They ll be in Stra burg and L ttich the next two years so we can probably be expected to attend. I don t think I can eat duck (which, in south-west france, is a vegetable) or like all that classic french multi-course food so much, but I had enough Couscous Merguez and Th la menthe fra che and similar good stuff. Many people spoke English and actually asked me whether I do (probably they couldn t bear me trying to spea^W^W^Wbutchering the language of the Grande Nation) and in general were a friendly bunch. I did see some people with machine guns in the city on the last day, though. No idea what/why didn t dare asking Just another reason to boycott flying: Mario Lang (one of the speakers) was apparently held on the airport and treated as a terrorist due to his Braille line they thought it was a bomb or somesuch thing.

Read on for more

Enrico Zini: Computing time offsets between EXIF and GPS

Computing time offsets between EXIF and GPS I like the idea of matching photos to GPS traces. In Debian there is gpscorrelate but it's almost unusable to me because of bug #473362 and it has an awkward way of specifying time offsets. Here at SoTM10 someone told me that exiftool gained -geosync and -geotag options. So it's just a matter of creating a little tool that shows a photo and asks you to type the GPS time you see in it. Apparently there are no bindings or GIR files for gtkimageview in Debian, so I'll have to use C. Here is a C prototype:
/*
 * gpsoffset - Compute EXIF time offset from a photo of a gps display
 *
 * Use with exiftool -geosync=... -geotag trace.gpx DIR
 *
 * Copyright (C) 2009--2010  Enrico Zini <enrico@enricozini.org>
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software
 * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
 */
#define _XOPEN_SOURCE /* glibc2 needs this */
#include <time.h>
#include <gtkimageview/gtkimageview.h>
#include <libexif/exif-data.h>
#include <stdio.h>
#include <stdlib.h>
static int load_time(const char* fname, struct tm* tm)
 
    ExifData* exif_data = exif_data_new_from_file(fname);
    ExifEntry* exif_time = exif_data_get_entry(exif_data, EXIF_TAG_DATE_TIME);
    if (exif_time == NULL)
     
        fprintf(stderr, "Cannot find EXIF timetamp\n");
        return -1;
     
    char buf[1024];
    exif_entry_get_value(exif_time, buf, 1024);
    //printf("val2: %s\n", exif_entry_get_value(t2, buf, 1024));
    if (strptime(buf, "%Y:%m:%d %H:%M:%S", tm) == NULL)
     
        fprintf(stderr, "Cannot match EXIF timetamp\n");
        return -1;
     
    return 0;
 
static time_t exif_ts;
static GtkWidget* res_lbl;
void date_entry_changed(GtkEditable *editable, gpointer user_data)
 
    const gchar* text = gtk_entry_get_text(GTK_ENTRY(editable));
    struct tm parsed;
    if (strptime(text, "%Y-%m-%d %H:%M:%S", &parsed) == NULL)
     
        gtk_label_set_text(GTK_LABEL(res_lbl), "Please enter a date as YYYY-MM-DD HH:MM:SS");
      else  
        time_t img_ts = mktime(&parsed);
        int c;
        int res;
        if (exif_ts < img_ts)
         
            c = '+';
            res = img_ts - exif_ts;
         
        else
         
            c = '-';
            res = exif_ts - img_ts;
         
        char buf[1024];
        if (res > 3600)
            snprintf(buf, 1024, "Result: %c%ds -geosync=%c%d:%02d:%02d",
                    c, res, c, res / 3600, (res / 60) % 60, res % 60);
        else if (res > 60)
            snprintf(buf, 1024, "Result: %c%ds -geosync=%c%02d:%02d",
                    c, res, c, (res / 60) % 60, res % 60);
        else
            snprintf(buf, 1024, "Result: %c%ds -geosync=%c%d",
                    c, res, c, res);
        gtk_label_set_text(GTK_LABEL(res_lbl), buf);
     
 
int main (int argc, char *argv[])
 
    // Work in UTC to avoid mktime applying DST or timezones
    setenv("TZ", "UTC");
    const char* filename = "/home/enrico/web-eddie/galleries/2010/04-05-Uppermill/P1080932.jpg";
    gtk_init (&argc, &argv);
    struct tm exif_time;
    if (load_time(filename, &exif_time) != 0)
        return 1;
    printf("EXIF time: %s\n", asctime(&exif_time));
    exif_ts = mktime(&exif_time);
    GtkWidget* window = gtk_window_new(GTK_WINDOW_TOPLEVEL);
    GtkWidget* vb = gtk_vbox_new(FALSE, 0);
    GtkWidget* hb = gtk_hbox_new(FALSE, 0);
    GtkWidget* lbl = gtk_label_new("Timestamp:");
    GtkWidget* exif_lbl;
     
        char buf[1024];
        strftime(buf, 1024, "EXIF time: %Y-%m-%d %H:%M:%S", &exif_time);
        exif_lbl = gtk_label_new(buf);
     
    GtkWidget* date_ent = gtk_entry_new();
    res_lbl = gtk_label_new("Result:");
    GtkWidget* view = gtk_image_view_new();
    GdkPixbuf* pixbuf = gdk_pixbuf_new_from_file(filename, NULL);
    gtk_box_pack_start(GTK_BOX(hb), lbl, FALSE, TRUE, 0);
    gtk_box_pack_start(GTK_BOX(hb), date_ent, TRUE, TRUE, 0);
    gtk_signal_connect(GTK_OBJECT(date_ent), "changed", (GCallback)date_entry_changed, NULL);
     
        char buf[1024];
        strftime(buf, 1024, "%Y-%m-%d %H:%M:%S", &exif_time);
        gtk_entry_set_text(GTK_ENTRY(date_ent), buf);
     
    gtk_widget_set_size_request(view, 500, 400);
    gtk_image_view_set_pixbuf(GTK_IMAGE_VIEW(view), pixbuf, TRUE);
    gtk_container_add(GTK_CONTAINER(window), vb);
    gtk_box_pack_start(GTK_BOX(vb), view, TRUE, TRUE, 0);
    gtk_box_pack_start(GTK_BOX(vb), hb, FALSE, TRUE, 0);
    gtk_box_pack_start(GTK_BOX(vb), exif_lbl, FALSE, TRUE, 0);
    gtk_box_pack_start(GTK_BOX(vb), res_lbl, FALSE, TRUE, 0);
    gtk_widget_show_all(window);
    gtk_main ();
    return 0;
 
And here is its simple makefile:
CFLAGS=$(shell pkg-config --cflags gtkimageview libexif)
LDFLAGS=$(shell pkg-config --libs gtkimageview libexif)
gpsoffset: gpsoffset.c
It's a simple prototype but it's a working prototype and seems to do the job for me. I currently cannot find out why after I click on the text box, there seems to be no way to give the focus back to the image viewer so I can control it with keys. There is another nice algorithm to compute time offsets to be implemented: you choose a photo taken from a known place and drag it on that place on a map: you can then look for the nearest point on your GPX trace and compute the time offset from that. I have seen that there are programs for geotagging photos that implement all such algorithms, and have a nice UI, but I haven't seen any in Debian. Are there any such softwares that can be packaged? If not, the interpolation and annotation tasks can now already be performed by exiftool, so it's just a matter of building a good UI, and I would love to see someone picking up the task.

10 July 2010

Colin Watson: debhelper statistics, redux

Apropos of my previous post, I see that dh has now overtaken CDBS as the most popular rules helper system of its kind in Debian unstable, and shows no particular sign of slowing its rate of uptake any time soon. The resolution of the graph is such that you can't see it yet, but dh drew dead level with CDBS on Thursday, and today 3836 packages are using dh as opposed to 3823 using CDBS.

John Goerzen: Radios

Those of you that follow me on twitter or identi.ca know that I ve been working on my amateur radio license. This started a few weeks ago when Jacob got excited about radios, and must have infected me too. I ve been studying and learning a bit. I had called a local ham (amateur radio operator) I know from church. He gave me one of his apparently newer radios, a 1981 2m Icom IC-22U 10W unit. That was great and got me all the more interested. Then last Friday, a couple of local hams came over to check out that radio and see what repeaters I could hit with it. They discovered a handheld 5W unit could hit at least one local repeater, and this 10W unit could too. One of them also had an HF rig in his pickup, and it was fun to stand out in the driveway and listen to conversations in Michigan and Utah. That evening erased any doubt in my mind about whether or not I would become a ham and it s a bit hard to believe it was only a week ago. I had been studying for the Technician-class amateur radio license, the lowest of three levels of licensing. One of the guys that came over Friday gave me a book to study for the General license, the intermediate level. It warned me to allow a month to prepare, and here I was planning to take the exams Tuesday. I picked it up Sunday, so really got 2 days of studying in. Tuesday I drove to Independence, KS, about 2.5 hours away for the exams. I went so far primarily because the Wichita exams weren t going to be offered until mid-August and I didn t want to wait that long. I found a fun group of people in Independence. I hadn t expected to have fun taking exams, but really, I did. We visited before the exams, and they wanted to know where I was from, of course. When it was time to take the exams, I guess I sort of surprised them by already having my FRN (FCC registration number) and photo copies of my IDs with me. One of them said, You re going to have no trouble with this, are you? I took the Technician exam, checked my work carefully, and turned it in to be graded. They checked it twice and had two people looking over it before they announced I passed with all 35 questions correct. I should note at this point that I was the only person taking the exams that night. Anyhow, on to the General exam. Same drill. It was harder, of course, and I turned it in to be graded. One of them looked at the test, looked at me, put on a grave face and said, Uh oh, not one right. I knew he was joking and they announced I passed THAT one with 100% correct as well. I hadn t expected that, and neither had they. That was what I planned to do, but they said that nobody had ever walked in, taken two tests, and had a score like that and pretty much insisted I try taking the Amateur Extra exam as well. I said I haven t studied for it at all and really doubt I could pass, but eventually went ahead and took that one too. I didn t pass, but I don t think they d have wanted to let me leave without trying. After the paperwork was done, they invited me to hang around and chat with their group that was meeting next door for a bit. I did, and drove home. Then was the frustrating part of the week: waiting for my license. I can t transmit until the FCC issues my license, even though I had passed the exam. My handheld radio arrived later in the week, and of course I still couldn t do much with it. Finally the FCC posted my license late yesterday so I was able to talk to people. It s been fun and I look forward to doing more. I was able to talk to people 55 miles away while driving, and with suitable equipment at home should be able to do much more than that. Several of the people I talked to were offering me tips. I ve never seen a group of people so eager to help out someone new. It s an amazing community and I think Jacob and Oliver will enjoy it one day too.

Enrico Zini: Searching OSM nodes in Spatialite

Searching OSM nodes in Spatialite Third step of my SoTM10 pet project: finding the POIs. I put together a query to find all nodes with a given tag inside a bounding box, and also a query to find all the tag values for a given tag name inside a bounding box. The result is this simple POI search engine:
#
# poisearch - simple geographical POI search engine
#
# Copyright (C) 2010  Enrico Zini <enrico@enricozini.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
#
from pysqlite2 import dbapi2 as sqlite
class PoiDB(object):
    def __init__(self):
        self.db = sqlite.connect("pois.db")
        self.db.enable_load_extension(True)
        self.db.execute("SELECT load_extension('libspatialite.so')")
        self.oldsearch = []
        self.bbox = None
    def set_bbox(self, xmin, xmax, ymin, ymax):
        '''Set bbox for searches'''
        self.bbox = (xmin, xmax, ymin, ymax)
    def tagid(self, name, val):
        '''Get the database ID for a tag'''
        c = self.db.cursor()
        c.execute("SELECT id FROM tag WHERE name=? AND value=?", (name, val))
        res = None
        for row in c:
            res = row[0]
        return res
    def tagnames(self):
        '''Get all tag names'''
        c = self.db.cursor()
        c.execute("SELECT DISTINCT name FROM tag ORDER BY name")
        for row in c:
            yield row[0]
    def tagvalues(self, name, use_bbox=False):
        '''
        Get all tag values for a given tag name,
        optionally in the current bounding box
        '''
        c = self.db.cursor()
        if self.bbox is None or not use_bbox:
            c.execute("SELECT DISTINCT value FROM tag WHERE name=? ORDER BY value", (name,))
        else:
            c.execute("SELECT DISTINCT tag.value FROM poi, poitag, tag"
                      " WHERE poi.rowid IN (SELECT pkid FROM idx_poi_geom WHERE ("
                      "       xmin >= ? AND xmax <= ? AND ymin >= ? AND ymax <= ?) )"
                      "   AND poitag.tag = tag.id AND poitag.poi = poi.id"
                      "   AND tag.name=?",
                      self.bbox + (name,))
        for row in c:
            yield row[0]
    def search(self, name, val):
        '''Get all name:val tags in the current bounding box'''
        # First resolve the tagid
        tagid = self.tagid(name, val)
        if tagid is None: return
        c = self.db.cursor()
        c.execute("SELECT poi.name, poi.data, X(poi.geom), Y(poi.geom) FROM poi, poitag"
                  " WHERE poi.rowid IN (SELECT pkid FROM idx_poi_geom WHERE ("
                  "       xmin >= ? AND xmax <= ? AND ymin >= ? AND ymax <= ?) )"
                  "   AND poitag.tag = ? AND poitag.poi = poi.id",
                  self.bbox + (tagid,))
        self.oldsearch = []
        for row in c:
            self.oldsearch.append(row)
            yield row[0], simplejson.loads(row[1]), row[2], row[3]
    def count(self, name, val):
        '''Count all name:val tags in the current bounding box'''
        # First resolve the tagid
        tagid = self.tagid(name, val)
        if tagid is None: return
        c = self.db.cursor()
        c.execute("SELECT COUNT(*) FROM poi, poitag"
                  " WHERE poi.rowid IN (SELECT pkid FROM idx_poi_geom WHERE ("
                  "       xmin >= ? AND xmax <= ? AND ymin >= ? AND ymax <= ?) )"
                  "   AND poitag.tag = ? AND poitag.poi = poi.id",
                  self.bbox + (tagid,))
        for row in c:
            return row[0]
    def replay(self):
        for row in self.oldsearch:
            yield row[0], simplejson.loads(row[1]), row[2], row[3]
Problem 3 solved: now on to the next step, building a user interface for it.

Jonathan McDowell: SPI 2010 AGM & Board Election

As SPI secretary I announced that nominations for the SPI board were open at the start of the month. The nomination period closes this Tuesday (13th July) with voting opening up on Thursday 15th. This year over half the board is up for election - 5 seats (currently held by Luk Claes, Joshua D. Drake, Bdale Garbee, Joerg Jaspert & Martin Zobel-Helas). So far I've received only 2 nominations, though I'm aware these things are often left to the last minute, so hopefully more will appear in the next few days. All anyone who wants to stand needs to do is drop secretary@spi-inc.org a (preferably PGP signed) email nominating yourself and providing a position statement (which will all be published once the nomination period is over).

Oh, and if you're a contributing SPI member please do remember to vote once voting is open!

Steve Kemp: New backported packages!

Since I'm I'm using real titles I guess I should make a real post, in which real things are mentioned. Unfortunately recently most of my time has been spent offline, doing things in and around Edinburgh. However I have done a few things which are possibly worthy of mention. My Lenny repository has been updated a little:
The Gimp
There's a slightly newer version of The Gimp available now, corresponding to a recent upload to unstable.
gtk-gnutella
Once again I was forced to update the backported gtk-gnutella package, as my previous one was too old to connect to the network.
itag
Finally I added a Lenny package for the itag software which is now essentially complete.
Of those things I had a lot of fun with the itag software. Partly because it allows me to horde my images in a way that I appreciate, but also because it made me go over some older images and be pleasantly suprised. My personal archive, ~/Images, is now just over 80Gb, and goes back about ten years. (Of course the older images were taken with random point and shoot digital cameras and each images is only a few hundred k in size. The newer images, saved at full-resolution, may be 5Mb each.) Otherwise I've been slowly deploying OpenLDAP in anger, which has been educational. I've got a minor problem to solve which is that (posix)group definitions don't seem to be working reliably, but otherwise I've got Apache authenticating against groups, SSH logins working, and the little brother database using the LDAP server as an address book. (Mail clients? mutt is the one true mail client. notmuchmail.org will be interesting when further developed, but everything else I'm going to ignore with my stubborn Yorkshire nature ;)

ObQuote: "Oh no no no, dead broad OFF THE TABLE!", from Shrek.

Enrico Zini: Importing OSM nodes into Spatialite

Importing OSM nodes into Spatialite Second step of my SoTM10 pet project: creating a searchable database with the points. What a fantastic opportunity to learn Spatialite. Learning Spatialite is easy. For example, you can use the two tutorials with catchy titles that assume your best wish in life is to create databases out of shapefiles using a pre-built, i386-only executable GUI binary downloaded over an insecure HTTP connection. To be fair, the second of those tutorials is called "An almost Idiot's Guide", thus expliciting the requirement of being an almost idiot in order to happily acquire and run software in that way. Alternatively, you can use A quick tutorial to SpatiaLite which is so quick it has examples that lead you to write SQL queries that trigger all sorts of vague exceptions at insert time. But at least it brought me a long way forward, at which point I could just cross reference things with PostGIS documentation to find out the right way of doing things. So, here's the importer script, which will probably become my reference example for how to get started with Spatialite, and how to use Spatialite from Python:
#!/usr/bin/python
#
# poiimport - import nodes from OSM into a spatialite DB
#
# Copyright (C) 2010  Enrico Zini <enrico@enricozini.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
#
import xml.sax
import xml.sax.handler
from pysqlite2 import dbapi2 as sqlite
import simplejson
import sys
import os
class OSMPOIReader(xml.sax.handler.ContentHandler):
    '''
    Filter SAX events in a OSM XML file to keep only nodes with names
    '''
    def __init__(self, consumer):
        self.consumer = consumer
    def startElement(self, name, attrs):
        if name == "node":
            self.attrs = attrs
            self.tags = dict()
        elif name == "tag":
            self.tags[attrs["k"]] = attrs["v"]
    def endElement(self, name):
        if name == "node":
            lat = float(self.attrs["lat"])
            lon = float(self.attrs["lon"])
            id = int(self.attrs["id"])
            #dt = parse(self.attrs["timestamp"])
            uid = self.attrs.get("uid", None)
            uid = int(uid) if uid is not None else None
            user = self.attrs.get("user", None)
            self.consumer(lat, lon, id, self.tags, user=user, uid=uid)
class Importer(object):
    '''
    Create the spatialite database and populate it
    '''
    TAG_WHITELIST = set(["amenity", "shop", "tourism", "place"])
    def __init__(self, filename):
        self.db = sqlite.connect(filename)
        self.db.enable_load_extension(True)
        self.db.execute("SELECT load_extension('libspatialite.so')")
        self.db.execute("SELECT InitSpatialMetaData()")
        self.db.execute("INSERT INTO spatial_ref_sys (srid, auth_name, auth_srid,"
                        " ref_sys_name, proj4text) VALUES (4326, 'epsg', 4326,"
                        " 'WGS 84', '+proj=longlat +ellps=WGS84 +datum=WGS84"
                        " +no_defs')")
        self.db.execute("CREATE TABLE poi (id int not null unique primary key,"
                        " name char, data text)")
        self.db.execute("SELECT AddGeometryColumn('poi', 'geom', 4326, 'POINT', 2)")
        self.db.execute("SELECT CreateSpatialIndex('poi', 'geom')")
        self.db.execute("CREATE TABLE tag (id integer primary key autoincrement,"
                        " name char, value char)")
        self.db.execute("CREATE UNIQUE INDEX tagidx ON tag (name, value)")
        self.db.execute("CREATE TABLE poitag (poi int not null, tag int not null)")
        self.db.execute("CREATE UNIQUE INDEX poitagidx ON poitag (poi, tag)")
        self.tagid_cache = dict()
    def tagid(self, k, v):
        key = (k, v)
        res = self.tagid_cache.get(key, None)
        if res is None:
            c = self.db.cursor()
            c.execute("SELECT id FROM tag WHERE name=? AND value=?", key)
            for row in c:
                self.tagid_cache[key] = row[0]
                return row[0]
            self.db.execute("INSERT INTO tag (id, name, value) VALUES (NULL, ?, ?)", key)
            c.execute("SELECT last_insert_rowid()")
            for row in c:
                res = row[0]
            self.tagid_cache[key] = res
        return res
    def __call__(self, lat, lon, id, tags, user=None, uid=None):
        # Acquire tag IDs
        tagids = []
        for k, v in tags.iteritems():
            if k not in self.TAG_WHITELIST: continue
            for val in v.split(";"):
                tagids.append(self.tagid(k, val))
        # Skip elements that don't have the tags we want
        if not tagids: return
        geom = "POINT(%f %f)" % (lon, lat)
        self.db.execute("INSERT INTO poi (id, geom, name, data)"
                        "     VALUES (?, GeomFromText(?, 4326), ?, ?)",
                (id, geom, tags["name"], simplejson.dumps(tags)))
        for tid in tagids:
            self.db.execute("INSERT INTO poitag (poi, tag) VALUES (?, ?)", (id, tid))
    def done(self):
        self.db.commit()
# Get the output file name
filename = sys.argv[1]
# Ensure we start from scratch
if os.path.exists(filename):
    print >>sys.stderr, filename, "already exists"
    sys.exit(1)
# Import
parser = xml.sax.make_parser()
importer = Importer(filename)
handler = OSMPOIReader(importer)
parser.setContentHandler(handler)
parser.parse(sys.stdin)
importer.done()
Let's run it:
$ ./poiimport pois.db < pois.osm 
SpatiaLite version ..: 2.4.0    Supported Extensions:
        - 'VirtualShape'        [direct Shapefile access]
        - 'VirtualDbf'          [direct Dbf access]
        - 'VirtualText'         [direct CSV/TXT access]
        - 'VirtualNetwork'      [Dijkstra shortest path]
        - 'RTree'               [Spatial Index - R*Tree]
        - 'MbrCache'            [Spatial Index - MBR cache]
        - 'VirtualFDO'          [FDO-OGR interoperability]
        - 'SpatiaLite'          [Spatial SQL - OGC]
PROJ.4 Rel. 4.7.1, 23 September 2009
GEOS version 3.2.0-CAPI-1.6.0
$ ls -l --si pois*
-rw-r--r-- 1 enrico enrico 17M Jul  9 23:44 pois.db
-rw-r--r-- 1 enrico enrico 37M Jul  9 16:20 pois.osm
$ spatialite pois.db
SpatiaLite version ..: 2.4.0    Supported Extensions:
        - 'VirtualShape'        [direct Shapefile access]
        - 'VirtualDbf'          [direct DBF access]
        - 'VirtualText'         [direct CSV/TXT access]
        - 'VirtualNetwork'      [Dijkstra shortest path]
        - 'RTree'               [Spatial Index - R*Tree]
        - 'MbrCache'            [Spatial Index - MBR cache]
        - 'VirtualFDO'          [FDO-OGR interoperability]
        - 'SpatiaLite'          [Spatial SQL - OGC]
PROJ.4 version ......: Rel. 4.7.1, 23 September 2009
GEOS version ........: 3.2.0-CAPI-1.6.0
SQLite version ......: 3.6.23.1
Enter ".help" for instructions
spatialite> select id from tag where name="amenity" and value="fountain";
24
spatialite> SELECT poi.name, poi.data, X(poi.geom), Y(poi.geom) FROM poi, poitag WHERE poi.rowid IN (SELECT pkid FROM idx_poi_geom WHERE (xmin >= 2.56 AND xmax <= 2.90 AND ymin >= 41.84 AND ymax <= 42.00) ) AND poitag.tag = 24 AND poitag.poi = poi.id;
Font Picant de la Cellera "amenity": "fountain", "name": "Font Picant de la Cellera" 2.616045 41.952449
Font de Can Pla "amenity": "fountain", "name": "Font de Can Pla" 2.622354 41.974724
Font de Can Ribes "amenity": "fountain", "name": "Font de Can Ribes" 2.62311 41.979193
It's impressive: I've got all sort of useful information for the whole of Spain in just 17Mb! Let's put it to practice: I'm thirsty, is there any water fountain nearby?
spatialite> SELECT count(1) FROM poi, poitag WHERE poi.rowid IN (SELECT pkid FROM idx_poi_geom WHERE (xmin >= 2.80 AND xmax <= 2.85 AND ymin >= 41.97 AND ymax <= 42.00) ) AND poitag.tag = 24 AND poitag.poi = poi.id;
0
Ouch! No water fountains mapped in Girona... yet. Problem 2 solved: now on to the next step, trying to show the results in some usable way.

9 July 2010

Benjamin Mako Hill: Memory

Today I started to tell a friend about something from dinner the night before. Except that she was at the dinner. And sitting at the same table! Even when prompted, I couldn't really remember! This does not warrant a blog post. Anybody who knows me well knows that my memory for these kinds of more mundane details is pretty porous. This kind of thing happens all the time. I'm writing this so that when I'm much older, and still forgetting things all the time, folks can use this as a reference point before concluding that senility is setting in. I'm afraid that everyone else will forget my forgetfulness!

Guido G nther: krb5-auth-dialog: Simple AFS plugin

I've just added a simple afs plugin to krb5-auth-dialog. It just calls either aklog or afslog after acquiring or renewing your Kerberos ticket to get you into your AFS cell. To activate it use
gconftool-2 --set --list-type=string --type=list /apps/krb5-auth-dialog/plugins/enabled [afs]
and restart krb5-auth-dialog. A package with the above enabled has been uploaded to Debian. Thanks to the test account in the global2000.at cell provided by J rg Herzinger I could test this easily and I have to say that AFS looks like a cool thing.

Mike Hommey: Iceweasel 4.0 beta 1 preliminary packages for Debian, and a nice trick

Since this blog is now syndicated on Planet Mozilla, a little background for its readers: Iceweasel is the name under which Firefox is distributed in Debian. This is an unfortunate detail due to copyright issues with the Firefox logos that have been solved recently. Work is under progress to get Firefox back in Debian (I do have good hope this will be possible). Anyways, I ve started to work on getting Iceweasel in shape for the 4.0 release in a few months. The packaging being in a good enough shape, I am hereby making available some preliminary packages (meaning there is still some more work needed for proper packaging) for the first beta release. The packages are available in a separate repository, which you need to add to your sources.list:
deb http://mozilla.debian.net/packages/ ./
Only x86 and x86-64 (amd64) packages are available for now. As far as I tested it, it works as well as the official Firefox 4.0beta1 binaries, though some xpcshell tests failed, and I haven t got reftests working yet. If you still wish to use a current version of iceweasel (i.e. non-beta), but want to try this beta, here is the nice trick: Instead of installing iceweasel from the above mentioned repository, Now you can start the Iceweasel beta with /some/directory/usr/bin/iceweasel. This trick should work with older versions of iceweasel, too.

Tim Retout: Toggle fullscreen in Emacs

Today I improved the function I was using to toggle full screen in GNU Emacs.
(defun toggle-fullscreen (&optional f)
  (interactive)
  (let ((current-value (frame-parameter nil 'fullscreen)))
       (set-frame-parameter nil 'fullscreen
                            (if (equal 'fullboth current-value)
                                (if (boundp 'old-fullscreen) old-fullscreen nil)
                                (progn (setq old-fullscreen current-value)
                                       'fullboth)))))
(global-set-key [f11] 'toggle-fullscreen)
The previous function would always restore the frame to "normal" state, even if Emacs was previously maximized. This one makes it behave more like, say, Epiphany or Totem. Apparently using GNOME Keyboard Shortcuts to set this to F11 globally would override custom behaviour from some programs. I've not found any examples of this, though. This would be easily extensible to disable the Emacs toolbar when fullscreen, for instance.

Christian Perrier: Launching DebConf10 Cheese and Wine party

Today, I officially launched the discussion to prepare Debconf attendees mind that we will have the 6th Cheese and Wine party at DebConf 10 in New York City. It has to be awesome. It will be.

Enrico Zini: Filtering nodes out of OSM files

Filtering nodes out of OSM files I have a pet project here at SoTM10: create a tool for searching nearby POIs while offline. The idea is to have something in my pocket (FreeRunner or N900), which doesn't require an internet connection, and which can point me at the nearest fountains, post offices, atm machines, bars and so on. The first step is to obtain a list of POIs. In theory one can use Xapi but all the known Xapi servers appear to be down at the moment. Another attempt is to obtain it by filtering all nodes with the tags we want out of a planet OSM extract. I downloaded the Spanish one and set to work. First I tried with xmlstarlet, but it ate all the RAM and crashed my laptop, because for some reason, on my laptop the Linux kernels up to 2.6.32 (don't now about later ones) like to swap out ALL running apps to cache I/O operations, which mean that heavy I/O operations swap out the very programs performing them, so the system gets caught in some infinite I/O loop and dies. Or at least this is what I've figured out so far. So, we need SAX. I put together this prototype in Python, which can process a nice 8MB/s of OSM data for quite some time with a constant, low RAM usage:
#!/usr/bin/python
#
# poifilter - extract interesting nodes from OSM XML files
#
# Copyright (C) 2010  Enrico Zini <enrico@enricozini.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
#
import xml.sax
import xml.sax.handler
import xml.sax.saxutils
import sys
class XMLSAXFilter(xml.sax.handler.ContentHandler):
    '''
    A SAX filter that is a ContentHandler.

    There is xml.sax.saxutils.XMLFilterBase in the standard library but it is
    undocumented, and most of the examples using it you find online are wrong.
    You can look at its source code, and at that point you find out that it is
    an offensive practical joke.
    '''
    def __init__(self, downstream):
        self.downstream = downstream
    # ContentHandler methods
    def setDocumentLocator(self, locator):
        self.downstream.setDocumentLocator(locator)
    def startDocument(self):
        self.downstream.startDocument()
    def endDocument(self):
        self.downstream.endDocument()
    def startPrefixMapping(self, prefix, uri):
        self.downstream.startPrefixMapping(prefix, uri)
    def endPrefixMapping(self, prefix):
        self.downstream.endPrefixMapping(prefix)
    def startElement(self, name, attrs):
        self.downstream.startElement(name, attrs)
    def endElement(self, name):
        self.downstream.endElement(name)
    def startElementNS(self, name, qname, attrs):
        self.downstream.startElementNS(name, qname, attrs)
    def endElementNS(self, name, qname):
        self.downstream.endElementNS(name, qname)
    def characters(self, content):
        self.downstream.characters(content)
    def ignorableWhitespace(self, chars):
        self.downstream.ignorableWhitespace(chars)
    def processingInstruction(self, target, data):
        self.downstream.processingInstruction(target, data)
    def skippedEntity(self, name):
        self.downstream.skippedEntity(name)
class OSMPOIHandler(XMLSAXFilter):
    '''
    Filter SAX events in a OSM XML file to keep only nodes with names
    '''
    PASSTHROUGH = ["osm", "bound"]
    TAG_WHITELIST = set(["amenity", "shop", "tourism", "place"])
    def startElement(self, name, attrs):
        if name in self.PASSTHROUGH:
            self.downstream.startElement(name, attrs)
        elif name == "node":
            self.attrs = attrs
            self.tags = []
            self.propagate = False
        elif name == "tag":
            if self.tags is not None:
                self.tags.append(attrs)
                if attrs["k"] in self.TAG_WHITELIST:
                    self.propagate = True
        else:
            self.tags = None
            self.attrs = None
    def endElement(self, name):
        if name in self.PASSTHROUGH:
            self.downstream.endElement(name)
        elif name == "node":
            if self.propagate:
                self.downstream.startElement("node", self.attrs)
                for attrs in self.tags:
                    self.downstream.startElement("tag", attrs)
                    self.downstream.endElement("tag")
                self.downstream.endElement("node")
    def ignorableWhitespace(self, chars):
        pass
    def characters(self, content):
        pass
# Simple stdin->stdout XMl filter
parser = xml.sax.make_parser()
handler = OSMPOIHandler(xml.sax.saxutils.XMLGenerator(sys.stdout, "utf-8"))
parser.setContentHandler(handler)
parser.parse(sys.stdin)
Let's run it:
$ bzcat /store/osm/spain.osm.bz2   pv   ./poifilter > pois.osm
[...]
$ ls -l --si pois.osm
-rw-r--r-- 1 enrico enrico 19M Jul 10 23:56 pois.osm
$ xmlstarlet val pois.osm 
pois.osm - valid
Problem 1 solved: now on to the next step: importing the nodes in a database.

Phil Hands: Talking to my MP about the Digital Economy Act

I had a constructive meeting with my Member of Parliament yesterday, in which he agreed to push this critique of the Digital Economy Act (the dire nonsense that Lord Mandelson forced through in the dying seconds of the previous UK parliamentary session, apparently at the behest of the copyright exploitation industry, who will presumably now owe him a few lucrative consultancies)If you're wanting to do something similar, feel free to use the content, or use it for inspiration, or just use the one-page PDF version if that's helpful to you in explaining the worthlessness of this law.Note, I know there are many other points that could be made, but I was restricting myself to one side of A4 (just) -- MPs cannot be expected to read more it seems.

Julian Andres Klode: Nokia/Intel/Google/Canonical openness and professionality in MeeGo, Android, Ubuntu

MeeGo (Nokia/Intel): Openness does not seem to be very important for Nokia and Intel. They develop their stuff behind closed doors and then do a large code drop (once dropped, stuff gets developed in the open). In terms of professionality, it does not look much better: If you look at the meego-dev mailing list, you feel like you are in some kind of a kindergarten for open source developers Things like HTML emails and top-posting appear to be completely normal for them, they don t even follow the basic rules for emails and they also appear to ignore any advice on this topic. Oh, and writing a platform in GTK+ while pushing Qt as the supported toolset is not a good idea. Android (Google): The situation with Android appears to be even worse. Google keeps an internal development tree and the only time the public sees changes is when a new release is coming and Google does a new code drop. Such a behavior is not acceptable. Ubuntu (Canonical): Canonical does an outstanding job on openness. Canonical employees develop their software completely in the open, and there are no big code drops. They basically have no competitive advantage and allow you to use their work for your own project before they use it themselves. Canonical employees basically behave like normal free software developers doing stuff in his free time, the only exception being that they are paid for it. Most Canonical employees also know how to behave on mailing lists and do not post HTML emails or do top-posting. There are of course also other people: The design process of the Ubuntu font on the other hand is a disaster, with the font being created using proprietary software running on Windows; for a free software company, that s not acceptable (such persons should probably not be hired at all). Furthermore, the quality of Ubuntu packages is often bad when compared to the quality of a Debian package (especially in terms of API documentation of Canonical-created software); a result of vastly different development cycles and different priorities. On a scale from 0 to 15 (15 being the best), Nokia/Intel receive 5 points ( D in U.S. grades), Google receives 3 points ( F in U.S. grades), and Canonical receives 12 points ( B in U.S. grades). Please note that those grades only apply for the named projects, the companies may behave better or worse in other projects. Final tips for the companies:
Filed under: General, Ubuntu

Alexander Reichle-Schmehl: Release Critical Bug report for Week 27

The unofficial rc bug tracker currently knows about the following bugs:
In Total:501
Affecting Squeeze:401
Squeeze only:95
Remaining to be fixed in Squeeze:306
Of these 306 bugs, the following tags are set:
Pending in Squeeze:16
Patched in Squeeze:49
Duplicates in Squeeze:38
Contrib or non-free in Squeeze:12
Claimed in Squeeze:5
Delayed in Squeeze:6
Can fixed in a security Update:35
Otherwise fixed in Squeeze:33
Ignoring all the above (multiple tags possible), 163 bugs need to be fixed by Debian Contributers to get Debian 6.0 Squeeze released. However, the with the view of the Release Managers, 310 need to be dealt with for the release to happen. Please see my older post for an explanation of the different numbers.

Julian Andres Klode: Build systems

In the past weeks, I was looking at several build systems. As it turned out, there is not a single sane generic build system out there. Autotools: Autotools are ugly, slow, and require an immense amount of code copies in the source tree. WAF: WAF is not as ugly as autools and it s faster and does not generate Makefiles or stuff like this. But it has serious issues: It requires one to copy it to the source tarball, has no stable API, and requires Python for building. Furthermore, support for unit testing is broken: It runs the unit tests, but does not abort the build process if the tests fail and does not display why the tests fail. CMake: The syntax is ugly, it generates Makefiles, and support for pkg-config seems to be very very basic. So how should a build system look like? So, why is there no sane build system?
Filed under: General

Petter Reinholdtsen: jXplorer, a very nice LDAP GUI

Since my last post about available LDAP tools in Debian, I was told about a LDAP GUI that is even better than luma. The java application jXplorer is claimed to be capable of moving LDAP objects and subtrees using drag-and-drop, and can authenticate using Kerberos. I have only tested the Kerberos authentication, but do not have a LDAP setup allowing me to rewrite LDAP with my test user yet. It is available in Debian testing and unstable at the moment. The only problem I have with it is how it handle errors. If something go wrong, its non-intuitive behaviour require me to go through some query work list and remove the failing query. Nothing big, but very annoying.

Next.

Previous.