Search Results: "spa"

1 October 2009

Daniel Kahn Gillmor: sexist behavior in the free software community

So not even 3 months out from RMS's sexist Gran Canaria virgins remarks, we have another powerful leader in the Free Software Community making sexist remarks in a talk to developers (this time, it's Mark Shuttleworth). It's a shame that these two people have said stupid things that hurt their causes and their communities by perpetuating an unfriendly environment for women. And it's a bigger shame that neither leader appears to care enough about their community to issue a sincere public apology for their screwup (if i'm wrong about this, please point me to the apology i've looked). These guys are in a situation which is nowhere near as hard as writing good software or managing complex technical projects: if you make a stupid mistake, own up to it, apologize, and try not to make similar mistakes in the future. Perhaps worst of all, are the remarkable number of unreasonably fucked-up comments on the blog posts discussing these unfortunate events. If you're in the habit of defending remarks like those made by RMS and Shuttleworth on the 'net, please take a minute and ask yourself a few questions: Please read the Geek Feminism wiki and blog. Even if you don't agree with everything on those sites (hey, it's a wiki! and a blog! you don't have to agree with everything!), people are at least trying to address the problem of sexism in our community there. Engage constructively and don't hide or ignore problems! UPDATE: I've asked the site admin to remove some trollish comments. If you want to use the comments on this blog to announce something like "i'd like for free software to be a boys-only club", that's fine, i'd like to know who you are so i can adjust my perception of you accordingly. Anything advocating or excusing rape and/or murder is way over the line and is not welcome here. i'm sure you can find some other place to troll. Tags: community, conduct, feminism, free software, sexism

Gerfried Fuchs: Times Are Changing

Since I returned from this year's debconf quite a lot things have changed in my life. One thing I knew way before debconf already, I switched jobs. I was working for a pretty long time (and also through great times) for Silver Server, for well over 6 years. Let's see what I can do in and for my new job. I'm still only pretty short there yet and I being to understand what's going on (and what could need improvements in some workflows) but it's too early to see that more precisely. On a small side-note, the job change also required me to switch my mobile phone number because the old one was a company's number. I now have a private one that I don't fear of losing anymore. If you subtract the number 3933309527644 from my old one you have my new one. On the other hand you can always ask me for it in case you had my old one and wonder. ;) Besides my mobile phone number other smaller things have changed over time too. Most of you noticed that I'm neither using my @ist.org nor my @debian.org address anymore because they did carry the wrong nick. And given that everyone was switching their gnupg keys I did so too. Like before I again created separate keys for private usage and for Debian usage. I'm not really following the procedure of sending out signing requests to people who have signed my old key(s), personally I don't follow them myself so I don't expect others to do. They are though cross-signed with the old ones just to be on the save side to keep the WoT not losing connections. Also I finally got around to set up my own ejabberd server and did create my a new jabber ID that I also plan to keep. It is the same as my private mailaddress as you can see linked above in the key for private usage. ;) One of the reasons for the server is by the way also a testbed for the packaging of ejabberd which will get moved into team maintenance with the next upload which will be the next upstream version, 2.1. But the biggest and most important change in my life is that I will have to rethink my spare time spending quite a lot because there will be something demanding a lot of time starting in about march next year. An extremely fortunate change and almost as happy one even though it means that I won't be able to attend next year's debconf in NY, but I hope the people whom I told that I will be there aren't too disappointed by this announcement. Sorry! :)

30 September 2009

John Goerzen: Sen. Brownback s Visit

Since I asked for suggestions on the Brownback town hall, I figured I ought to report what happened. He was scheduled to spend about 30 minutes in it; he probably spent a total of 40 or 45. He arrived a few minutes early, went around the room shaking hands and asking each person their name (including me). It was polite on the one hand, and a little odd on the other, as he was occasionally reaching over chairs to reach people far away. Anyhow, he stood up front and had a little speech to start with. He spent the first 5-10 minutes talking about local sports teams. It might have been a nice icebreaker if it was 30-60 seconds, but it just felt weird by the time it was over. Everybody knew he was here to talk about other things and had very limited time. He then launched into the economy, talking about keeping taxes down and making sure they re low in Kansas for the recovery. He touched on energy, supporting wind energy (which led to his most memorable comment of the morning: This state blows, and it s got lots of wind, too. ) He also came out in favor of coal, oil, and gas. Basically he s pro-energy. Can t go wrong with that, eh? He rode our prototype all-electric ZTR mower, and praised it quite a bit. He touched on health care briefly, saying that he favors incremental changes to bring down costs. He thought health savings accounts and high-deductible health plans (such as one option at my employer with a $2500 deductible) were good innovations. He then opened it up for questions. There was a brief awkward moment when nobody raised their hand, so I did. I started by saying where I agreed with him. I told him I agreed that health care is way too expensive, that I agree that we need liability reform, and said that we pay almost as much each month in health care as we do in all federal taxes combined. I was going to continue, but he interrupted, and said, Are you serious? Yes. 80% as much in fact. Do you have an illness in your family? My wife has asthma, a thyroid disorder, and we had a baby this year. That s it. Wow. So I continued. I told him I was concerned about the 46 million uninsured Americans, that they weren t getting the care they needed. I pointed out that many of them wouldn t be able to afford that $2500 deductible in the HDHP, let alone regular doctor visits. And I said that it is our moral and even religious duty to take care of the least of these when it comes to health care. He answered by first saying that nobody goes without care that s the law because anybody can get care in any emergency room in the country. Then he spent several minutes describing how he thinks the government can t manage anything, and thus a single-payer system wasn t right for us. After a few minutes of this, he asked if that answered my question. I said that I don t care how the 46 million people get care, whether it s a public option or subsidies for private companies, just that they get it. I also acknowledged that ERs must provide necessary stabilizing care , but pointed out that 20,000 Americans die each year due to inadequate health care and reminded him that ERs are not required to provide diagnostic or preventative services, or long-term treatment of things like cancer. For the first, but not the last, time of the day, at that, he got really quiet, sort of hung his head, and said something like, Yeah, I agree. It is hard on the least of these. I wish we could do more. And moved on to the next question. At this point, it got weird. The next question was a guy that stood up, saying how we ve got to put God back into the constitution. He said he s lived long enough to see the moral decline in this country, and it s going to hell faster than we know. We need to be God-fearing again, he said. Brownback, in reply, again did the quiet hang-his-head thing and said, I agree. Then moved on. We then had a question about state s rights, nullification , and the 10th amendment. Though it was obvious some issue was underneath it, I couldn t figure out just what it was. Brownback, predictably, thinks that the federal government has grown way too powerful. (Ironic for someone that supported the Patriot Act, but hey.) Then one other question, and as he was going to wrap it up, a yelled question from the back: What s good about Kansas? Brownback was caught a bit off-guard, but launched into talking about aerospace, and how he hopes to lure wealthy retirees from California to settle here. All in all, an interesting experience. More respectful than I feared, more scary than I imagined, and about as genuine as I expected a politician at that level gets.

Enrico Zini: Lessons learnt from Oracle

Lessons learnt from Oracle Lesson number 1: "how to handle temporary files".
$ rm tp*
$ proc code=cpp lines=yes dba_vm.pc 
Pro*C/C++: Release 10.2.0.1.0 - Production on Wed Sep 30 11:10:00 2009
Copyright (c) 1982, 2005, Oracle.  All rights reserved.
System default option values taken from: /usr/local/oracle/10.2.01/db_1/precomp/admin/pcscfg.cfg
$ echo $?
0
$ ls tp*
tpoHRjc8  tpqkU4Cp  tpY5Eo4G
$ 
Today we learn this: for every successful compilation of a single source file, it is a Good Thing to leave three temporary files around as a tribute to the Holy Trinity who bestowed upon us the grace of seeing our prayers fulfilled and our work rewarded. For those illiterate of us who do not want to learn from the Market Leaders, and do not want to put a rm -f tp* in their makefiles just to avoid in the future to see hours of work on tpl_support.cc disappear at the first invocation of make, here's the Yokel's Wrapper:
#!/bin/sh -ue
if [ $# -ne 2 ]
then
        echo "Usage: $0 infile outfile" >&2
        exit 1
fi
DIR= mktemp -d 
cleanup()  
        rm -rf "$DIR"
 
trap cleanup EXIT
cp "$1" "$DIR"
(
        cd $DIR
        proc code=cpp lines=yes "$1"
        mv  basename "$1" .pc .c "$2"
)
mv "$DIR/$2" .
exit 0

Romain Beauxis: About free software and yellow pages...

Even if it has happened in the past that I could support a pragmatic approach in the context of free software and media technologies, there is something that offenses me in the context of free audio players. In the world of audio streaming servers on for net radios, there are two majors competitors. In the one hand, Shoutcast, the commercial proprietary streaming server released by Nullsoft, which is also the editor of Winamp, the popular media player. On the other hand, Icecast, which started as an open source implementation of the protocol used by Shoutcast, but was rewritten in its version 2 release to support the free codecs from xiph. Now, appart from the respective quality of the two servers, the real competition takes place in another place: availability of the radios using this or that server. For that purpose, both have implemented Yellow Pages. Shoutcast's yellow pages are here, whereas Icecast/Xiph's yellow pages are there. The great thing with yellow pages is their integration into the audio player. Then, the user can browse the list of radios, and test them. It is very good both for convenience no need to download a playlist or enter manually a complicated URL and advertisement a single place to list all possible radios. The yellow pages from Shoutcast are very popular. They are integrated in Winamp of course, but also in many other popular players, in the PSP, in the iPhone, etc... The Shoutcast yellow pages are reserved for shoutcast users. Of course, the protocol has been reversed it is kind of dummy but if your radio get caught registering to Shoutcast's yellow pages while not using their proprietary software, your radio gets blacklisted and you loose a whole lot of listeners (and money too). For that reason, many net radios want to use Shoutcast as their broadcasting server. From the point of view of Nullsoft, it seems a good call. At least, they play their game and make their money out of it, so let's put this appart. But, what about the free software world ? The obvious idea would be that the free software projects should push altogether for the use of free alternatives, hence pushing for a support of Icecast's yellow pages in the free players. Of course, free software being also a matter of choice, it should be possible to also support Shoutcast's yellow pages. After all, both protocols are very simple, so it is no big deal to implement.. Well, unfortunately, that is not exactly what happens.. Guess what, Amarok and VLC only implement Shoutcast's yellow pages... Given that VLC is a massively popular audio player that spreads among a lot of plateforms, I find it quite a pity that they do not do something to encourage its users to use free alternatives to the proprietary Shoutcast yellow pages, and similarly for Amarok. The adoption of free alternatives only works with massive demand, and we should not hope that proprietary softwares help in any way, but, at least, free softwares should be a bit more aware of what they're advocating when they choose to implement only the proprietary support...

Dirk Eddelbuettel: RInside release 0.1.1, and a fresh example

Last week's 0.1.0 release of RInside, and the first to have been published on CRAN, still had some issues with builds and use on OS X. Thanks to testing and fixes by Jan de Leeuw, Jeff Horner and particularly Simon Urbanked, things are said to be better now with the new release 0.1.1 which went onto CRAN yesterday. So no new features, but fixes to the main Makefile as well as the Makefile for the examples directory, some minor fixes and editing for the examples. I also added a file THANKS to show some appreciation for the various patches and fixes I have been receiving -- they are appreciated! However, today I committed a new example to SVN archive at R-Forge. It is based on this thread on r-devel. Abhijit Bera tries to do this in C, but to me his questions provide rather clear motivation for showing how much simpler things can be via C++ and the Rcpp classes along with RInside. Using a small example, the task was to pass a weight vector to a portfolio solver from the Rmetrics package fPortfolio and to then access the computed solution. The original poster struggled with access from C to the S4 classes used by fPortfolio and could not set the weights. But when using RInside, we simply pass a C++ vector of weights down to R, solve the problem and pass a solution vector back using the handy evaluation of R expressions:
// -*- mode: C++; c-indent-level: 4; c-basic-offset: 4;  tab-width: 8; -*-
//
// Another simple example inspired by an r-devel mail by Abhijit Bera
//
// Copyright (C) 2009 Dirk Eddelbuettel and GPL'ed
#include "RInside.h"                    // for the embedded R via RInside
#include "Rcpp.h"                       // for the R / Cpp interface used for transfer
#include <iomanip>
int main(int argc, char *argv[])  
    try  
        RInside R(argc, argv);          // create an embedded R instance
        SEXP ans;
        std::string txt = "suppressMessages(library(fPortfolio))";
        if (R.parseEvalQ(txt))          // load library, no return value
            throw std::runtime_error("R cannot evaluate '" + txt + "'");
        txt = "lppData <- 100 * LPP2005.RET[, 1:6]; "
	  "ewSpec <- portfolioSpec(); "
	  "nAssets <- ncol(lppData); ";
        if (R.parseEval(txt, ans))      // prepare problem
            throw std::runtime_error("R cannot evaluate '" + txt + "'");
	const double dvec[6] =   0.1, 0.1, 0.1, 0.1, 0.3, 0.3  ; // choose any weights you want
	const std::vector<double> w(dvec, &dvec[6]);
	R.assign( w, "weightsvec");	// assign STL vector to R's 'weightsvec' variable
	txt = "setWeights(ewSpec) <- weightsvec";
        if (R.parseEvalQ(txt))		// evaluate assignment
            throw std::runtime_error("R cannot evaluate '" + txt + "'");
	txt = "ewPortfolio <- feasiblePortfolio(data = lppData, spec = ewSpec, constraints = \"LongOnly\"); "
	  "print(ewPortfolio); "
	  "vec <- getCovRiskBudgets(ewPortfolio@portfolio)";
        if (R.parseEval(txt, ans))      // assign covRiskBudget weights to ans
            throw std::runtime_error("R cannot evaluate '" + txt + "'");
	RcppVector<double> V(ans);      // convert SEXP variable to an RcppMatrix
	R.parseEval("names(vec)", ans);	// assign columns names to ans
	RcppStringVector names(ans);
	for (int i=0; i<names.size(); i++)  
	  std::cout << std::setw(16) << names(i) << "\t"
		    << std::setw(11) << V(i) << "\n";
         
      catch(std::exception& ex)  
        std::cerr << "Exception caught: " << ex.what() << std::endl;
      catch(...)  
        std::cerr << "Unknown exception caught" << std::endl;
     
    exit(0);
 

Chris Lamb: GeoDjango and the UK postcode database

This post describes how to play with the UK postcode database that recently turned up on WikiLeaks using GeoDjango. You will need the following pieces of free software:
  1. PostgreSQL (MySQL does not provide sufficient geospacial features) I will be using Postgres 8.4.1; please adjust your paths for your version and/or distributor.
  2. PostGIS spacial database extensions I will be using version 1.4.0.
  3. Django I will be using version 1.1. This version includes the GeoDjango framework.
  4. GEOS geometry engine I will be using version 3.0.0 from the libgeos-c1 package in Debian unstable.
You will also need a copy of the leaked postcode data. Usage of this data is almost certainly a copyright violation.
Getting started First, create a Django project and application for this example:
$ django-admin startproject ukpostcodes
$ cd ukpostcodes
$ ./manage.py startapp postcodes
Next, edit settings.py:
  • Configure DATABASE_*. You must use the postgresql_psycopg2 PostgreSQL database driver.
  • Add django.contrib.gis & ukpostcodes.postcodes to INSTALLED_APPS.
If you have not already done so, you must install PostGIS in your database:
$ sudo -u postgres createlang plpgsql -d <databasename>
$ sudo -u postgres psql <databasename> -f /usr/share/postgresql/8.4/contrib/postgis.sql
$ sudo -u postgres psql <databasename> -f /usr/share/postgresql/8.4/contrib/spatial_ref_sys.sql
Modelling a postcode Now define a simple model to store imported postcodes:
from django.contrib.gis.db import models
class Postcode(models.Model):
    name = models.CharField(max_length=8, db_index=True)
    location = models.PointField()
    objects = models.GeoManager()
  • We import models from django.contrib.gis.db instead of django.db.
  • We override the default objects manager with GeoDjango's GeoManager() so we can perform spacial queries.
Save this model in postcodes/models.py. We can now create this model in our database using:
$ ./manage syncdb
Importing postcode data We will use a management command to import the data. First, create the necessary modules with:
$ mkdir -p postcodes/management/commands
$ touch postcodes/management/commands/__init__.py
$ touch postcodes/management/__init__.py
Next, save the following code in postcodes/management/commands/import_postcodes.py:
import sys
import csv
from django.contrib.gis.geos import Point
from django.core.management.base import BaseCommand
from postcodes.models import Postcode
class Command(BaseCommand):
    def handle(self, \*args, \*\*options):
        Postcode.objects.all().delete()
        count = 0
        for row in csv.reader(sys.stdin):
            try:
                name = row[0].upper().strip().replace('  ', ' ')
                location = Point(map(float, row[13:15]))
            except ValueError:
                print "I: skipping %r" % row
                continue
            Postcode.objects.create(name=name, location=location)
            count += 1
            if count % 10000 == 0:
                print "Imported %d" % count
Finally, we can import the data using:
$ bunzip2 /path/to/uk-post-codes-2009.bz2   ./manage.py import_postcodes
This takes approximately one hour on my computer. If you plan to import the data multiple times, consult the Django documentation for faster methods of importing initial data.
Quering the postcode database We can now query our postcode database. First, let's find the postcode for Buckingham Palace:
>>> from postcodes.models import Postcode
>>> palace = Postcode.objects.get(name='SW1A 1AA')
>>> palace
<Postcode: SW1A 1AA>
Next, let's find all postcodes within 1 mile. For this, we will need to construct a tuple representing the area we want to search in.
>>> from django.contrib.gis.measure import Distance
>>> area = (palace.location, Distance(mi=1))
>>> Postcode.objects.filter(location__distance_lte=area)
[<Postcode: SE1 7BF>, <Postcode: SE1 7SG>, <Postcode: SE1 7JA>, <Postcode: SE1 7PB>,
 <Postcode: SE1 7PD>, <Postcode: SW1A 1DW>, <Postcode: SW1A 1EA>, <Postcode: SW1A 1EE>,
 <Postcode: SW1A 1EF>, <Postcode: SW1A 1EG>, '...(remaining elements truncated)...']
>>> Postcode.objects.filter(location__distance_lte=area)
7024
To find "neighbour" postcodes, we can pass a Point object to the .distance(..) method on our queryset this adds a field we can then sort on:
>>> Postcode.objects.distance(palace.location).order_by('distance')[:4]
[<Postcode: SW1A 1AA>, <Postcode: SW1E 6JP>, <Postcode: SW1E 6LA>, <Postcode: SW1E 6WG>]
Putting it all together Let's pretend we are making an app to find pizza from an arbitrary user-submitted postcode. (We will assume we have parsed the user's postcode and have found its Postcode object, but we will continue to use Buckingham Palace here.) First, define the following model in postcodes/models.py and re-run ./manage.py syncdb:
class PizzaJoint(models.Model):
    name = models.CharField(max_length=200)
    postcode = models.ForeignKey(Postcode)
    objects = models.GeoManager()
    def __unicode__(self):
         return self.name
Now, add some local pizza places:
>>> def add_pizza_joint(name, loc):
...     code = Postcode.objects.get(name=loc)
...     PizzaJoint.objects.create(name=name, postcode=code)
...
>>> add_pizza_joint('Pizza Hut', 'SW1E 6SQ')
>>> add_pizza_joint('Pizza Express', 'SW1E 5NA')
>>> add_pizza_joint('Pizza On The Park', 'SW1X 7LY')
If we were not using GeoDjango, we would have to be content with simply matching the postcode:
>>> PizzaJoint.objects.filter(postcode=palace)
[]
However, we can now find the cloest pizza places:
>>> PizzaJoint.objects.distance(palace.location,
            field_name='postcode__location').order_by('distance')
[<PizzaJoint: Pizza Hut>, <PizzaJoint: Pizza Express>, <PizzaJoint: Pizza On The Park>]
The .distance(..) method also sets a distance attribute on all the objects returned; this makes it easy to show the actual distance to the user:
>>> for x in PizzaJoint.objects.distance(palace.location,
                     field_name='postcode__location').order_by('distance'):
...    print "%s (%d metres away)" % (x, x.distance.m)
...
Pizza Hut (499 metres away)
Pizza Express (509 metres away)
Pizza On The Park (902 metres away)
For more details, see the GeoDjango documentation.

27 September 2009

Joey Hess: additions and subtractions

Hauled this stove into the yurt today, across a creek that was flooded yesterday. Putting a stove in a space changes/refocuses it so much.

John Goerzen: How To Record High-Definition MythTV Files to DVD or Blu-Ray

I ve long had a problem. Back on January 20, I took the day off work to watch the inauguration of Barack Obama. I saved the HD video recordings MythTV made of the day (off the ATSC broadcast), intending to eventually save them somehow. I hadn t quite figured out how until recently, so there they sat: 9 hours of video taking up about 60GB of space on my disk. MythTV includes a program called mytharchive that will helpfully transcode your files and burn a DVD from them. But it helpfully will transcode the beautiful 1920 1080i down to DVD resolution of on a good day 720 480. I couldn t have that. Background My playback devices might be a PC, the MythTV, or the PlayStation 3. Of these, I figured the PS3 was going to be hardest to accommodate. ATSC (HD) broadcasts in the United States are an MPEG Transport Stream (TS). Things are a bit complicated, because there may be errors in the TS due to reception problems, the resolution and aspect ratio may change multiple times (for instance, down to SD for certain commercials). And, I learned that some ATSC broadcasts are actually 1920 1088 because the vertical resolution has to be a multiple of a certain number, and those bottom 8 pixels shouldn t be displayed. Adding to the complexity, one file was 7 hours of video and about 50GB itself. I was going to have to do quite some splitting to get it onto 4.7GB DVD+Rs. I also didn t want to re-encode any video, both for quality and for time reasons. Attempts So, I set out to try and figure out how to do this. My first approach was the sledgehammer one: split(1). split takes a large file and splits it on byte or line boundaries. It has no knowledge of MPEG files, so it may split them in the middle of frames. I figured that, worst case, I could always use cat to reassemble the file later. Surprisingly, both mplayer and xine could play back any of these files, but the PS3 would only play back the first part. I remembered this as an option if all else failed. Next, I tried avidemux. Quite the capable program and I thought I could use it to cut my file into DVD-sized bits. But I couldn t get it to let me copy the valid MPEG TS into another MPEG TS it kept complaining of incompatible formats, but wouldn t tell me in what way they were incompatible. I could get it to transcode to MPEG4, and produce a result that worked on the PS3, but that wasn t really what I was after. Then, I tried mpgsplit. Didn t recognize the MPEG TS as a valid file, and even when I used a different tool to convert to MPEG PS, acted all suspicious as if it bought the MPEG from a shady character on a grungy street corner. dvbcut I eventually wound up using dvbcut to split up the ATSC (DVB) recordings. It understood the files natively and did exactly what I wanted. Well, *almost* exactly what I wanted. It has no command-line interface and didn t have a way to split by filesize, but I calculated that about 35 minutes of the NBC broadcast and 56 minutes of the PBS broadcast could fit on a single DVD+R. It worked very, very nicely. The resulting files tested out well on both the PS3 and the Linux box. So after that, I wrote up an index.txt file to add to each disc and a little shell scripting later and I had a directory for each disc. I started burning them with growisofs. Discs 1 and 2 burned well, but then I got an error like this:
File disc06/VIDEO/0930-inaug-ksnw-06.mpg is larger than 4GiB-1.
-allow-limited-size was not specified. There is no way do represent this file size. Aborting.
Eeeeepp. So apparently the ISO 9660 filesystem can t represent files bigger than 4GB. My files on disc 1 had represented multiple different programs, and stayed under that limit; and disc 2 s file was surprisingly just a few KB short. But the rest of them weren t. I didn t want to have to go back and re-split the data to be under 4GB. I also didn t want to waste 700MB per disc, or to have to make someone change video files every 15 minutes. So I decided to investigate UDF, the filesystem behind Blu-Ray discs. mkisofs couldn t make a pure UDF, only a hybrid 9660/UDF disc that risked compatibility issues with big files. There is a mkudffs, but it doesn t take a directory of its own. So I wrote a script to do it. Note that this script may fail with dotfiles or files with spaces in them:
#!/bin/bash
set -e
if [ ! -d "$1" -o -e "$2" -o -z "$2" ]; then
   echo "Syntax: $0 srcdir destimage [volid]"
   echo "destimage must not exist"
   exit 5
fi
if [ " id -u " != "0" ]; then
   echo "This program must run as root."
fi
EXTRAARGS=""
if [ ! -z "$3" ]; then
   EXTRAARGS="--vid=$3"
fi
# Get capacities at http://en.wikipedia.org/wiki/DVD+R as of 9/27/2009
SECSIZE=2048
# I'm going to set it a few lower, than capacity 2295104 just in case.
# Must be at least one lower than the actual size for dd to do its thing.
# SECTORS=2295103
SECTORS=2295000
echo "Allocating image..."
dd if=/dev/zero "of=$2" bs=2048 "seek=$SECTORS" count=1
echo "Creating filesystem..."
mkudffs --blocksize=2048 $EXTRAARGS "$2"
mkdir "$2.mnt"
echo "Populating..."
mount -o rw,loop -t udf "$2" "$2.mnt"
cp -rvi "$1/"* "$2.mnt/"
echo "Unounting..."
umount "$2.mnt"
rmdir "$2.mnt"
echo "Done."
That was loosely based on a script I found on the Arch Linux site. But I didn t like the original script, because it tried to do too much, wasted tons of time writing out 4.7GB of NULLs when it could have created a sparse file in an instant, and was interactive. So there you have it. HD broadcast to playable on PS3, losslessly. Note that this info will also work equally well if you have a Bluray drive.

26 September 2009

Gunnar Wolf: The bad and the worse; Representative democracies' minimums

Martin rants about the German electoral system. From his rant, I'll pick up only two points And I'll try to connect with Toxicore's excelent (Spanish) blog post, where he quotes political analist Denise Dresser. Dresser has made a great point: Our probably-imposed, legitimacy-impaired president Felipe Calder n has requested the society to talk good about Mexico, to project a positive image of the country. Dresser says, yes, there is a lot of good to talk about the country, and we should emphasize on its richness and beauty, invite people to come and visit, to know what the country is really like. But at the same time, it is our duty to talk bad about the bad areas and decisions of our government, as that is the best (if not the only) thing many of us can do to really get things to happen That is what we can do to push our country's good things forward, to make the country sustainable, to pull attention towards what needs (such as the very very deplorable cases of censorship, human rights violation, ecosystem predation we have seen in the last years). Anyway... What did I want to comment about Martin's post? He criticizes Germany's law requiring a 5% quota for a party to have parliamentary representation. In Mexico, the minimum is 2%. Most people agree, though, that it is too low, and that we should push to increase it. Why? Because the money that is spent in supporting the party system. In Mexico, when a political party fails to get 2% of the vote, it is basically disbanded and it is very hard for it to regroup, to compete again. Many people believe we should aim to a political system with as few political parties as possible (such as the semi-democratic system they have in the USA). I strongly prefer the system found in most European (and even many South American) countries where there is a real wealth of ideological positions represented, and where governments have to be formed by agreeing to form coalitions, as it is almost impossible for them to get full majority. I would much rather see Mexico march towards a parliamentary-based political system, away from the presidential one. Of course, that is almost impossible to expect. With the current political system, we are bound to have forever few monolithic, meaningless political parties. We will likely converge on three blocks, following the current three major blocks (leftoid PRD, centroid PRI, rightoid PAN). They are different in some important senses, yes, but in general they are much the same. I don't hold any hopes to ever see something like the Pirate Party appearing in our system...

Christoph Haas: This should work fine nowadays (about vino, Compiz and wrong optimism)

I ve been back from KDE (4.2!) to Gnome and with the mighty powers of Compiz my desktop is the envy of my Windows coworkers and still geeky functional. Well, somehow functional. For a while I wondered why I can t properly use VNC to remote-control my desktop from home. The connection works and I see the desktop but nothing seems to happen when I click or type. Now I figured out the desktop in the VNC client window is just not refreshed while my mouse and keyboard actions really do what they were supposed to on the VNC server. This is my sad tale of how I learned that a regression may mean driving the motorcycle to work in the middle of the night after cursing VNC. :) After a little searching through the lazyweb I found out that it s a bug reported on Ubuntu already. Apparently Compiz doesn t use the XDAMAGE extension properly. Somehow when using the graphics adapter s magic 3D powers X11 loses control of what sections are changed I m not an X11 guru. At least the client doesn t know when to redraw areas of the screen that have changed. The suggested workarounds were pretty ridiculous. Start another x11vnc session without compiz and connect to that. Very funny. I want to control my running desktop not create a new one. So I switched off compiz at work and voila VNC works again. Some eye-candy and shortcuts are gone but remote controlling the desktop is far more important while im on call. But I quickly missed Compiz. The missing eye-candy wasn t that bad. But metacity s window placement algorithm sucks as badly as back in 2002. As if I can hear it whisper: I have no idea where to put that stupid window. You tell me. I researched a little further and somehow the bug is known to Gnome, too. And other users suffering from the problem are pointed to the VirtualGL page of Wikipedia. Apparently the problem was fixed for several people until a regression was reported where things failed again. Seems like the vino developers removed the piece of code that detected the use of 3D window managers as a 2008 christmas present. Their reason: This should work fine nowadays. Hell, no, it doesn t! Somehow I downgraded vino to version 2.24.1-3 and ran the vino server manually (/usr/lib/vino/vino-server sm-disable) to see if it prints any error message. Indeed it does:
Composite detected, disabling XDamage extension.
This is a workaround while XDamage extension does not work correctly with 3d desktops.
Hopefully it will work at next vino release.
This time it worked well, too . I upgraded to the current version 2.28.0-1 (as in unstable by today) and suddenly my VNC connection had the same problems again. Aha! Let s try the Lenny version 2.22.2-1. Works, too. This has taken me hours to debug. Finally I know I can report a bug against the vino package. I wonder how long it would have taken a mortal to figure that out. :( [Sorry to my sponsorees who's package sponsoring requests were delayed again due to this mess.]

Pablo Lorenzzoni: Ruby versus Python

This is not another rant to praise one in spite of the other (an everybody knows I love Ruby, so it would not be impartial), but sometimes people seems to live in another world and do things for the wrong reasons. I just read this blog post by Kanwei Li in which he gives 2 or 3 reasons he ditched Ruby in favor of Python. First of all, both are great languages and, although I favor Ruby, I use Python for some projects and they are not all that different. Of course, everyone is free to choose which language one favors, but Kanwei seems to be ditching Ruby out of not knowing much about it, or out of preferring one style over the other His first reason is that in Python white spaces matter. I used to think this is just a matter of style, but every now and then mandatory alignment hurts me (just try to put together a code generator and you ll notice it). Although my code is always correctly aligned, I like that it s done so because I want it that way, and not because some language demands it. Rants and more rants have been written about Python s mandatory alignment (or other languages lack of it), and I am not going through all of it Just I don t think it s a good reason to ditch Ruby After, he makes a big deal out of Ruby s ternary if. As written by him, he prefers

#python
if len(a) > 0:
        v = a[0]
        a = a[1:]
else:
        v = None
over Ruby s ternary if

#ruby
v = a.empty? ? a.shift : nil
Hey! Come on Ruby s ternary if is not mandatory It was copied from C just as a syntax sugar. You can do without it, just as in Python:

#ruby
if ! a.empty?
    v = a.shift
else
    v = nil
end
Better yet! you can use if s return as v value:

#ruby
v = if ! a.empty?
    a.shift
else
    nil
end
How beautiful is that! Python lacked ternary if for a long time, and when it finally acquired one via PEP 308 its syntax was made different from every other language! Although I don t think that is a problem, some people might think it would be better not reinventing the wheel. Next, Kanwei goes over a famous problem of Ruby: the lack of a sum method for Array. I admit it s strange, but that is completely coherent: Ruby s Arrays are ordered collection of objects and not mathematical arrays. How do you sum objects that are not numbers? Many different people will have many different answers to that, so Ruby leaves this decision for the programmer and provides basic methods to deal with collections of anything (that can be used to apply sum to numbers, if wished). So, in Ruby you have to use Array#inject to perform a sum:

[1,2,3].inject(0)    sum, value  sum + value  
Array#inject (actually Enumerable#inject) was borrowed from Smalltalk and allows you to loop through an array, building up an accumulator value as you go. When it s done, the final value of this accumulator is returned. Very useful for combining array elements, whether by summing them, building up a pretty display string, whatever. In the example above, I am initializing the accumulator with 0. If you use Array to mathematical operations and you want your arrays to work that way, you can always add a sum method to Array class:

class Array
    def sum
        self.inject(0)  sum, value  sum + value 
    end
end
Maybe it would be better if you just use Arrays as containers (as it was intended to) and implement that sum inside your own class I completely agree with Reg Braithwaite here. Kanwei also mention Python is faster than Ruby. That is true, but was more true some time ago. First of all, Python is older and has had more time to improve its speed. Ruby, ITOH, just now acquired a good VM and improvements to it finally can run parallel to improvements in the language itself, so I am expecting this to be less true every release. Python is already not getting much faster between releases, unlike Ruby (the differences between 1.8.7 and 1.9.1 are really impressive!). IMHO this is not a good reason to choose one instead of the other: if you really need speed, go for C :-) Now this is something I find interesting Kanwei has mentioned: Python is more production ready . He argues that Google is using it, so it must be good. Well I cannot argue against that: Google is really using Python. But IBM, Oracle, EA, Cisco, Siemens, etc are using Ruby so that is just a matter of preferring one or another company. Both are production ready I agree, though, that Ruby 1.9.1 has many differences from 1.8.7, and that that may be seen as some inconsistency, but Python also has changed a lot since its 2.0 version, for that matter. And the changes to Ruby brought many benefits I think they worth it. At last, Kanwei compares Python and Ruby docstrings. Here I also have to agree with him: Ruby docstrings sucks. Actually that s why everybody uses rdoc instead (and that is much more powerful than Python s docstrings). Again, I don t think that is reason enough to ditch Ruby (actually, the existence of rdoc, rubygems & friends should bring people to Ruby instead), but that is a matter of personal taste. Surely, Kanwei s reasons were easy to argue against. There are areas were Python shines much more than Ruby (and vice-versa), but those Kanwei mentioned are not among them. I think both languages are powerful enough, and both are way better than Perl or PHP, so either one you choose would be fine. Better if you don t have to choose and use both ;). If you have to, ITOH, pay more attention on how you feel while coding in each one, and not to some cheap reasons such as above. If you are a programmer, what matters most is that you ll spend a lot of time coding with any given language let that be something pleasant then.

Stefano Zacchiroli: RC bugs of the week - week 4

RCBW - week #4 RC bugs squashed this week by yours truly: You're welcome to join the initiative!, in doing so please tag the blog post as "debian" and "rc".

25 September 2009

Julian Andres Klode: cupt and how to write package managers


cupt is a new package manager written in Perl by Eugene V. Lyubimkin, who previously contributed to APT. And more than all, the project makes no sense at all. First of all, there is a language issue. Implementing a package manager in Perl has some major drawbacks. One of the features of APT was it being written in a lower-level language (i.e. C++ which really is below Perl), making it possible to write applications like synaptic and python bindings which in turn lead to applications like gnome-app-install or Ubuntu s new Software Store. Furthermore, writing a package manager in Perl means that Distributions such as Emdebian might not be able to use it since they have excluded Perl due to its space requirements. This becomes even more important considering that cupt depends on even more perl libraries. This means that cupt will never be able to replace APT. Secondly, a package manager should not be designed specifically for one distribution. This is another major drawback of cupt and other package managers such as yum or zypper. The smart package manager, written in Python and funded by Canonical Ltd. is an example for a distribution-neutral package manager. Now let s take a look at package management in modern distributions. Usually we have two levels of package managers, the first being tools like dpkg and rpm which take care of installing and removing the packages and level-2 package managers implementing dependency resolvers and package retrieval from remote locations. Recently, distributions started to add a third layer named PackageKit, which shall provide distribution-independent package management user interfaces. The project was well-received by RPM-based distributions, but failed in Debian-based distributions due to not supporting debconf. Furthermore, adding a third layer just increases the possibility of problems. The right way to do package management is a distribution-independent level-2 package manager written in C. The smart project shows us that this is possible although itself fails to meet the lower-level language (C) requirement. That s why I decided to write a package manager in Vala, a GObject-based language which gets converted to C and then compiled. If successful, this project will be able to replace most of the current level-2 package managers and will also provide the same distribution-independence as provided by a level-3 package manager such as PackageKit. It is also easy to create binding for other programming language such as Python or Perl thus enabling application developers to choose the language they like most. The core of this project is a vendor-neutral library, temporarily called libapt (as the project is called APT2 for now). This library contains all the code which is not specific to a vendor i.e. file retrieval, dependency resolver, caches, etc and is then enhanced by several vendor-specific plugins, each implementing a PackageManager (interface to the distribution s level-1 package manager) and a Repository (well, repositories from which you can download packages) interface. We could even enhance the vendor-independent interface to include more details of a repository. Most repositories nowadays consist of 4 components: A meta index (Release files for Debian, repomd.xml for Fedora/openSUSE/etc.), a package index (e.g. Packages files for Debian, *primary.xml.gz on Fedora, etc. ), a source index (e.g Sources files in Debian) and a files index (e.g. Contents-*.gz for Debian). I took a look at the repository formats of Slackware, openSUSE and Fedora and it seems that this concept can be applied to all of them. So maybe all we need are distribution dependent parsers for those files. One of the most important issues with APT is its use of mmap() for the cache. Using mmap() makes it hard to grow the cache, which is sometimes needed. We see a lot of bug reports from people with too small cache sizes. We can circumvent this problem by utilizing an embedded database like SQLite for this, but we would probably loose some speed and it may be harder to maintain a flexible API. We should see what the best option is here, both ways are possible since Vala 0.7.6 includes my patches for adding mmap(), ftruncate(), mremap() and some other functions. An idea to circumvent the mmap() issue is gathering statistics about the relation between the number of repositories and the size of the cache and then using a value which is slightly above the average statistical value. The project is not very mature yet, it only includes basic library functions for downloading files and parsing configuration files, etc. You can find the (MIT licensed) code at http://git.debian.org/?p=users/jak/apt2.git. I also have some local code for repository management and multi-threaded file fetching, but it s just not ready to be merged yet. Posted in APT2, Debian

Julian Andres Klode: The APT2 project


I just started working on a replacement for APT written in Vala and called APT2 (I know, the name could be better). The main idea behind the project is to create a library for working with Debian repositories and packages, and on top of this library a few applications. This is different from APT because the project focuses on the library part, whereas APT is primarily focused on the application part. I chose Vala for this task because it has an easy syntax and allows me to use the features provided by GLib and Co. really easily. GLib provides many functions needed for APT like file access, unicode handling, hashsums creation. It also provides us with signals which are useful for implementing progress reporting. Another project, called cupt tries to provide an APT replacement in Perl. Apart from the fact that I don t like Perl, it also has the disadvantage of being too big to ship on embedded devices and is not useable from other languages such as Python or C/C++. Yet another project, smart provides a different package manager written in Python. While I like Python, it can t be used from languages such as Perl or C/C++ and is too large. APT2 will only require GLib, Gee and libarchive, which require about 2MB of space; about 1/10 of Perl s size. The core of APT2 is a library called libapt . At the moment, this library provides classes for reading tag files (TagFile&TagSection) and an incomplete configuration class based on libgee s HashMap (to ease coding, it will probably be changed to a special data structure like those in apt-pkg). The next step will be implementing a parser for configuration files (currently we only support in-memory objects) and writing the Acquire system. The planned Acquire system uses GModule to modularize the support for different URI schemes. Each module provides a worker class which implements one or more URI schemes. The first of these modules will provide a worker using GIO, which deals with local file access, samba shares, FTP, SFTP, WebDAV and various other protocols, including HTTP until a replacement has been written (since we don t want to force gvfs-backends). The workers communicate with the parent Acquire object using signals and can cause the whole acquisition to be aborted by emiting an abort signal (or similar). The whole project has just been started and may take some months until it becomes useful (if it becomes useful at all). The code can be found at http://git.debian.org/?p=users/jak/apt2.git and is licensed under the MIT license (see http://www.opensource.org/licenses/mit-license.php). It is a secondary project and will not affect my work on apt (not much yet) and python-apt (0.8 series). Posted in APT2, Debian

Julian Andres Klode: APT2: config parser testing


If you have an amd64 system, install the apt2 package from deb http://people.debian.org/~jak/debian/ unstable/ and run the apt2-config command. Make sure that the parser reports no errors, otherwise send me an email or leave a comment here. One known exception is that all values must be quoted in the configuration file, I have no plans to fix this (probably just deprecate unquoted strings in APT instead). The parser is not as strict as cupt s parser, but it gives you more help if something wents wrong. We also ignore most semicolons for now (they will be turned into warnings or errors later on). It is using GScanner from GLib for parsing the files. apt2-config replicates the functionality of apt-config. It currently do not read the configuration files in the correct order, so don t expect apt2-config dump to produce exactly the same output as apt-config dump . To build APT2 yourself, you need a patch for waf from http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=548329. Posted in APT2

Gunnar Wolf: Honduras: .hn NIC attacked/intervened by the de-facto government authorities

I was requested to forward this information to as wide an audience as possible. Possibly two months ago the legality/legitimacy of the actions carried out by the Hondurean armed forces, which captured a democratically elected president and without a judicial order or trial process forced him out of the country, starting a de-facto government, was something questionable. Each day, however, it becomes clearer and clearer the Hondureans are suffering a represive military-backed system which cannot be expected to fulfill as a trustable entity to conduct fair, credible elections. I got this message from a Hondurean friend (of course, whose identity I am not divulging) denouncing the government's invasion of the .hn domain name registry, which is handled by the Sustainable Development Network (Red de Desarrollo Sustentable RDS-HN). The National Telecomunications Comission (Comisi n Nacional de Telecomunicaciones, CONATEL) demands all domain name registration under the .hn top-level domain (TLD) to be suspende, and all the lists and databases regarding said TLDs to be handed over, detailing the IP ranges and the responsibles. They did this under the argument that RDS-HN is an Internet Service Provider (which it is not Being a registrar means they are responsible for the well-keeping of public information and of handling a public good, the .hn TLD, not that they provide any kind of regulated service to individuals or organizations), with military personnel disguised as civilians (and who refused to identify themselves). If you are interested, please read further on the text I received straight from my Hondurean contacts (Spanish) (or its unaccurate but often helpful automated translation to English, done through Google Translate) Even though this information is normally accessible via WHOIS and similar services (this only states clearly nobody in CONATEL was able to do what I just did legally and anonymously from my personal workstation), they did it in such a fashion in order to scare the operators and the society. Honduras is going through a very hard process. Whatever happens there will likely impact on the future reactions to the most retrograd and powerful sectors of society in the rest of Latin America. We do our best (even if as non-Hondureans living outside Honduras it only means raising our voices) to avoid the risk of our region going back to the sad, cruel and bloody 1970s history. [update] My friend Mave, who works at NIC Chile, sent as a comment to this post LACTLD's official stand on this regard (Spanish. English version also available). LACTLD (Latin American and the Caribbean ccTLD's Organization) clearly backs RDS-HN and condemns the illegal government's actions.
AttachmentSize
Detailing the intervention attempt of RDS-HN by the de-facto government agents (Spanish)8.61 KB

MJ Ray: Strange Spammer Sightings

I ve seen a couple of the conversation-spams described in Misspelled nemesis club: A new twist on spam? It looks like they exploit the typically weak moderation practices of many software discussion mailing lists. Do you think they are human-powered? Are they caught out if you set default_member_moderation (new list member postings are moderated) or equivalent in the list software? Google and Yahoo Groups seem to be suffering worst. Do they even have such an option? I ve also caught some strange behaviour on one of the SMTP servers we run for a customer. Does anyone know what virus or worm or other nasty authenticates but gives itself away by sending RCPT TO:<MISSING*EMAIL*ADDRESS> to SMTP servers?

Christine Spang: principle of most surprise

A python example in ipython:
In [1]: for i in range(10):
...:     print "i in loop:", i
...:
...:
i in loop: 0
i in loop: 1
i in loop: 2
i in loop: 3
i in loop: 4
i in loop: 5
i in loop: 6
i in loop: 7
i in loop: 8
i in loop: 9
In [2]: print "i out of loop:", i
i out of loop: 9
This bit me last night while writing some code for a digital communications lab assignment. I typed the wrong variable name, which was from an inner loop when I meant to use the element from the outer loop. Is there actually a sane reason for a loop variable not to go out of scope when the loop ends? Tell me there's a good reason for it. It took me completely by surprise.

24 September 2009

Christine Spang: FSF Women in Free Software Mini Summit

I was one of the participants invited to the FSF's Women in Free Software mini-summit which happened on the same day as Software Freedom Day this year. That was this past Saturday, September 19th. I was feeling hesitant going in, given the controversy over "why would the FSF run a closed event about such an important topic" and just a general burnt out feeling about the subject on my part. Honestly, I've tended to avoid being much of a feminist in the past and simply hope that people will stop bickering and leave me to my code, especially in the aftermath following well-publicised incidents. This is despite the fact that the thing that got me involved with free software originally was Debian Women, back in late 2004/early 2005. I needed a hook, and yet I haven't felt confident enough to reach out to others now that I'm in. I see now that there are valid reasons for having an event that is purposefully small. We all fit in a nice conference room at the FSF office, the atmosphere was very intimate, we got to learn everyone's names (and even remember them). I met some great local free software people who I hadn't met before. Food was take out from My Thai, an excellent vegan restaurant in Boston's Chinatown. While it sucks to be exclusive, in my opinion the small size really had an effect on what we got done and how we felt at the end. The official minutes from the meeting are online here, and there's a picture and brief blog post from Deborah here. The picture was taken using Cheese! I'm in the middle wearing my ever-popular Best Practical "my free software runs your company" t-shirt. There are only a few things I want to highlight myself.
  1. It was a great idea to invite someone who is involved in freedom movements but is not necessarily heavily involved in free software in particular. Hillary brought fresh insights and helped us draw parallels and come up with ideas that I don't think we would have thought of otherwise. It's easy to get used to parts of a community as being "normal" and I am so happy that we have allies who can show us where we've internalised or just have learned to ignore sexist parts of the community. Women in free software are already in free software and we need to learn to reach out to others who aren't in already.
  2. Cooperative power. There are groups of people that just aren't attracted by FOSS marketing that challenges you to "prove yourself the best" or similar. Just because someone doesn't like coding at 3am doesn't mean he doesn't like coding. If we want to succeed at our mission, we need to stop thinking win-lose and start thinking win-win. This applies to being more inclusive in general, not just for women. It also applies to valuing contributions from those who don't code. We need them too.
  3. There are times when I don't speak up and I should. There are times when I don't blog (or participate in discussions via other media) because I don't feel like dealing with potential community backlash. I am very careful about not stepping on peoples' toes, because I've seen other people get trampled on and am not particularly excited about experiencing that myself. While I'm trying to cultivate courage in myself, it was good to have a reminder that it's not just me.
The current plan is to hold a bigger event that is open to all in the spring. I am so excited about the momentum we are building! I think the biggest thing that I took out of this is that there are still things wrong with the community and that I shouldn't be afraid to speak up and be an activist. Watching people like Deborah and Hillary talk perfectly seriously about making very long-term plans and reaching parity was incredibly empowering. We can do so much better. The time to do it is now.

Next.

Previous.