As part of preparing xulrunner 1.9.2 (which would lead to Iceweasel 3.6), I have gone through all the patches currently applied to the latest xulrunner release, to keep track of what should be upstreamed, what needed feedback, what patches I needed to update in upstream bugzilla, etc. As I have talked about our differences with upstream on a few occasions already, and as I was doing this grunt work anyway, I thought it would be nice to make the results public. First, a few reference numbers (excluding changes to generated configure files):

• Changes between upstream 1.9.0.1 and Debian release 1.9.0.1-1: 85 files changed, 1163 insertions(+), 644 deletions(-)
• Changes between upstream 1.9.0.16 and Debian release 1.9.0.16-1: 86 files changed, 1177 insertions(+), 634 deletions(-)
• Changes between upstream 1.9.1 and Debian release 1.9.1-1: 70 files changed, 770 insertions(+), 363 deletions(-)
• Changes between upstream 1.9.1.6 and Debian release 1.9.1.6-1: 76 files changed, 796 insertions(+), 380 deletions(-)
• Changes between upstream 1.9.1.5 and upstream 1.9.1.6: 165 files changed, 3094 insertions(+), 1387 deletions(-)
• Changes between upstream 1.9.1 and upstream 1.9.1.6: 1263 files changed, 64322 insertions(+), 67376 deletions(-) (though more than half of that relates to upstream bump of sqlite3 and the vorbis stack)

A few observations on the above data:

• There are less changes between upstream and Debian than between two consecutive upstream versions.
• Debian changes only account for changes to the xulrunner package, i.e. it excludes changes to Iceweasel itself, or to nspr or nss, but these are far less important in number.
• The number of changes applied decreases when switching to a new upstream branch. This is mostly due to patches getting applied upstream.
• Debian patches on a given branch are pretty stable. In other words, most of the work is done on the first release of the branch.

The latter used to be true, but is not anymore: xulrunner 1.9.1.6-2 added a lot of changes and is now up to: 98 files changed, 1107 insertions(+), 517 deletions(-). What happened ? will you ask me. A lot happened: the test suite, attempts at correctly installing C++ headers and IDL files, bug fixes. Work has also been done such that make clean/distclean doesn t leave stuff around and that xulrunner can be properly built twice in a row. Differences between 1.9.1.6-1 and 1.9.1.6-2 look like this: 31 files changed, 341 insertions(+), 167 deletions(-). Some were stolen from bugzilla (13 files changed, 162 insertions(+), 70 deletions(-)):

• https://bugzilla.mozilla.org/show_bug.cgi?id=458631
• https://bugzilla.mozilla.org/show_bug.cgi?id=519468
• https://bugzilla.mozilla.org/show_bug.cgi?id=525047

Many others were sent:

• Test suite issues (2 files changed, 3 insertions(+), 4 deletions(-)):
  • https://bugzilla.mozilla.org/show_bug.cgi?id=543441
  • https://bugzilla.mozilla.org/show_bug.cgi?id=543438
• make clean/distclean issues (10 files changed, 28 insertions(+), 15 deletions(-)):
  • https://bugzilla.mozilla.org/show_bug.cgi?id=541767
  • https://bugzilla.mozilla.org/show_bug.cgi?id=541768
  • https://bugzilla.mozilla.org/show_bug.cgi?id=541769
  • https://bugzilla.mozilla.org/show_bug.cgi?id=541770
  • https://bugzilla.mozilla.org/show_bug.cgi?id=541774
  • https://bugzilla.mozilla.org/show_bug.cgi?id=543464
  • https://bugzilla.mozilla.org/show_bug.cgi?id=543469
• Other issues (8 files changed, 112 insertions(+), 49 deletions(-)):
  • https://bugzilla.mozilla.org/show_bug.cgi?id=535990
  • https://bugzilla.mozilla.org/show_bug.cgi?id=538933
  • https://bugzilla.mozilla.org/show_bug.cgi?id=542468
  • https://bugzilla.mozilla.org/show_bug.cgi?id=542789
  • https://bugzilla.mozilla.org/show_bug.cgi?id=542999

A few others were filed without a patch, as I felt a proper upstreamable patch needed discussion (3 files changed, 16 insertions(+), 3 deletions(-)):

• https://bugzilla.mozilla.org/show_bug.cgi?id=538929
• https://bugzilla.mozilla.org/show_bug.cgi?id=542749

A few changes are related to the way we build and how we run the test suite (5 files changed, 35 insertions(+), 18 deletions(-)):

• Disable python-xpcom tests for now.
• Disable javaxpcom tests at build time when DEB_NO_JAR is unset.
• Synchronize config/rules.mk and js/src/config/rules.mk for check-sync-dirs.py (we used to localize the changes required in each config/rules.mk, but as we now run <core>make check they need to be synchronized)</core>
• Workaround for https://bugzilla.mozilla.org/show_bug.cgi?id=455238
  

Finally, a few were related to the Debian changes history (2 files changed, 1 insertions(+), 26 deletions(-)):

• Remove enable-system-lcms check, which somehow resisted merges with upstream
• Properly clean xulrunner/installer/*.system.conf

Going through the rest of the changes between upstream and the latest Debian release, we can split in 5 categories of changes:
Those that were picked from upstream, or sent and eventually applied in 1.9.2 or trunk (12 files changed, 68 insertions(+), 41 deletions(-)):

• https://bugzilla.mozilla.org/show_bug.cgi?id=331779 (this one was applied on Fennec)
• https://bugzilla.mozilla.org/show_bug.cgi?id=442196
• https://bugzilla.mozilla.org/show_bug.cgi?id=444918
• https://bugzilla.mozilla.org/show_bug.cgi?id=481350
• https://bugzilla.mozilla.org/show_bug.cgi?id=491617
• https://bugzilla.mozilla.org/show_bug.cgi?id=491646
• https://bugzilla.mozilla.org/show_bug.cgi?id=501241
• https://bugzilla.mozilla.org/show_bug.cgi?id=502369
• https://bugzilla.mozilla.org/show_bug.cgi?id=510040
• https://bugzilla.mozilla.org/show_bug.cgi?id=514151

Those that were picked from bugzilla or sent there, but are not applied upstream for various reasons (29 files changed, 138 insertions(+), 209 deletions(-)):

• https://bugzilla.mozilla.org/show_bug.cgi?id=331776 (the part that remains in Debian needs to be sent in a new bug as this one is closed)
• https://bugzilla.mozilla.org/show_bug.cgi?id=349985
• https://bugzilla.mozilla.org/show_bug.cgi?id=356011
• https://bugzilla.mozilla.org/show_bug.cgi?id=428765
• https://bugzilla.mozilla.org/show_bug.cgi?id=434190 (we actually don t have a patch for this one, but a workaround is in bz#486681 below)
• https://bugzilla.mozilla.org/show_bug.cgi?id=435683
• https://bugzilla.mozilla.org/show_bug.cgi?id=440908
• https://bugzilla.mozilla.org/show_bug.cgi?id=444440
• https://bugzilla.mozilla.org/show_bug.cgi?id=444477
• https://bugzilla.mozilla.org/show_bug.cgi?id=486681
• https://bugzilla.mozilla.org/show_bug.cgi?id=501300
• https://bugzilla.mozilla.org/show_bug.cgi?id=506224
• https://bugzilla.mozilla.org/show_bug.cgi?id=530534

Those that were reported but for which I wanted feedback for a proper fix, yet a patch was applied in Debian because this was necessary (6 files changed, 22 insertions(+), 13 deletions(-)):

• https://bugzilla.mozilla.org/show_bug.cgi?id=445128
• https://bugzilla.mozilla.org/show_bug.cgi?id=501434
• https://bugzilla.mozilla.org/show_bug.cgi?id=528687

Those that are yet to be sent to bugzilla (15 files changed, 111 insertions(+), 54 deletions(-)):

• xulrunner-config: Give more appropriate cflags and libs
• Install applications in /usr/local/lib instead of /usr/lib
• Unhide release notes link in about: but only if app.releaseNotesURL is defined
• In about:, Don t put an about:blank link when there is no vendorURL defined
• Don t use static strings when setting environment variables
• Exec instead of uselessly forking in xulrunner launcher script
• Disable optimization on alpha for the url-classifier component (this will probably wait for the test suite failures on various architectures to be fixed)
• Build js shell and xpcshell against system libreadline
• Don t export js_SetTraceableNativeFailed, which is only used internally
• Increase stability and performance on mips and mipsel

And finally, the changes that are either Debian specific or not deemed upstreamable (30 files changed, 460 insertions(+), 77 deletions(-) ; this can sound huge, but see the details below):

• Feature changes:
  • Disable APNG support when system libpng doesn t support it (6 files changed, 71 insertions(+), 10 deletions(-) ; mostly #ifdef ing parts of the code related to animation)
  • Use application.ini from where the xulrunner-stub lies (1 files changed, 14 insertions(+), 3 deletions(-) ; This one may be dropped and replaced by something totally different, possibly upstreamed)
  • Don t register plugins if the MOZILLA_DISABLE_PLUGINS environment variable is set (1 files changed, 25 insertions(+), 0 deletions(-) ; very useful to isolate plugin-related crashes ; I don t think it s worth upstreaming considering the upcoming plugin-in-separate-process work)
  • Remove (un )registering system (1 files changed, 0 insertions(+), 43 deletions(-) ; xulrunner versions that are installed by a package manager don t need to be registered, since the package does it already ; a variant that conditionally removes it may be upstreamable)
• Preferences changes:
  • Set javascript.options.showInConsole (1 files changed, 1 insertions(+), 0 deletions(-))
  • Set DPI to system settings (1 files changed, 1 insertions(+), 1 deletions(-))
• Cosmetic changes:
  • Install loading_16_grey.gif in classic.jar, and add an override for loading16.png (1 files changed, 2 insertions(+), 0 deletions(-) ; Since we don t support APNG, we replace the most used one by its (ugly) GIF equivalent)
  • Add links for about:bugs and about:README.Debian in about: (3 files changed, 19 insertions(+), 0 deletions(-) ; I d like to make that something living in debian/extra-stuff)
• Integration changes:
  • Add another preferences directory for applications: defaults/syspref (1 files changed, 2 insertions(+), 0 deletions(-))
  • Add an rpath to libpyloader.so and _xpcom.so (2 files changed, 2 insertions(+), 0 deletions(-) ; allows python xpcom to work out of the box)
  • Forward-port nsIBadCertListener from 1.8 (4 files changed, 266 insertions(+), 1 deletions(-) ; This is necessary for at least galeon and kazehakase, because the new interface is useless for embedding applications)
• Build-related changes:
  • Allow to override the PYTHON_SO variable (1 files changed, 1 insertions(+), 1 deletions(-))
  • Put the crmf library before the NSS libraries (1 files changed, 1 insertions(+), 1 deletions(-))
  • Allow to build java jar files in 2 pass (2 files changed, 8 insertions(+), 2 deletions(-))
  • Avoid libxpcom being excluded from linked libraries by -Wl, as-needed (1 files changed, 2 insertions(+), 0 deletions(-))
  • Ignore system libjpeg, libpng and zlib version checking (1 files changed, 3 insertions(+), 3 deletions(-))
  • Force to not use -fshort-wchar (1 files changed, 1 insertions(+), 1 deletions(-))
  • Add soversion to libmozjs (2 files changed, 19 insertions(+), 2 deletions(-) ; useful for couchdb, libjavascript-perl and others)
  • Don t build example component (1 files changed, 0 insertions(+), 1 deletions(-))
  • Don t install system profile (1 files changed, 0 insertions(+), 2 deletions(-))

Note, the numbers in the various diff stats obviously don t add up, since some changes overlap. All that information doesn t account the few additional components added in debian/filemonitor and debian/extra-stuff, that account for nearly 600 lines (excluding licensing boilerplate), but I ll talk about these in another post. This analysis also helped spotting another harmless merge error (the first one was spotted a few weeks ago and just left the enable-system-lcms around, without any effect): a variable replacement in a Makefile that doesn t have any effect, since the 2 variables have the same value. Only a pointless 1 line change.

Romain uploaded our first release of RProtoBuf to CRAN yesterday. RProtoBuf provides bindings for GNU R to the Google Protobuf implementation. Google Protobuf is (and I quote) a way of encoding structured data in an efficient yet extensible format that is used for almost all internal RPC protocols and file formats at Google. RProtoBuf had a funny start. I had blogged about the 12 hour passage from proof of concept to R-Forge project following the ORD session hackfest in October. What happened next was as good. Romain emailed within hours of the blog post and reminded me of a similar project that is part of Saptarshi Guha's RHIPE R/Hadoop implementation. So the three of us--Romain, Saptarshi and I---started emailing and before long it becomes clear that Romain is both rather intrigued by this (whereas Saptarshi has slightly different needs for the inner workings of his Hadoop bindings) and was able to devote some time to it. So the code kept growing and growing at a fairly rapid clip. Til that stopped as we switched to working feverishly on Rcpp to both support the needs of this project, and to implement ideas we had while working on this. That now lead to the point where Rcpp is maturing in terms of features, so we will probably have time come back to more work on RProtoBuf to take advantage of the nice templated autoconversions we now have in Rcpp. Oddly enough, the initial blog post seemed to anticipate changes in Rcpp. Anyway -- RProtoBuf is finally here and it already does a fair amount of magic based of code reflection using the proto files. The Google documentation has a simple example of a 'person' entry in an 'addressbook' which, when translated to R, goes like this:

R> library( RProtoBuf )                      ## load the package
R> readProtoFiles( "addressbook.proto" )     ## acquire protobuf information
R> bob <- new( tutorial.Person,              ## create new object
+   email = "bob@example.com",
+   name = "Bob",
+   id = 123 )
R> writeLines( bob$toString() )              ## serialize to stdout
name: "Bob"
id: 123
email: "bob@example.com"
R> bob$email                                 ## access and/or override
[1] "bob@example.com"
R> bob$id <- 5
R> bob$id
[1] 5
R> serialize( bob, "person.pb" )             ## serialize to compact binary format

There is more information at the RProtoBuf page, and we already have a draft package vignette, a 'quick' overview vignette and a unit test summary vignette. More changes should be forthcoming as Romain and I find time to code them up. Feedback is as always welcome.

So I finally got off my lazy butt and started the paperwork to get a formal recognition of studies equivalence for a undergraduate studies (Licenciatura en ingenier a de software) via CENEVAL's Acuerdo 286 (licenciatura). Part of the paperwork involves filling the form I photographed and attached to this node. This is the utmost example of ber-redundant paperwork... Where it requires my personal data, some of the fields are:

CURP

Clave nica de Registro Poblacional, Unique Populational Registration Key. This is a string composed by:

• WOIG: First letter and first vowel after the first letter of my first family name, first letter of my second family name, first letter of my given name
• 760427: birth date, yymmdd
• H: Sex (H = male, M = female, I guess)
• DF: State or federative entity I was born in
• LSN: First consonant after the first letter of the first and second family name and of the given name
• 03: Deambiguation digits

So far, so good.

Age

Well, cannot it be deduced from the CURP? It has my full birth date!

Sex

Well, cannot it be deduced from the CURP? It is a specific field in it

Birth date

Man, you already have it literally on top of the field!

Nationality

This tramit is only for Mexicans, so... what's the point?

Birth place

The CURP states already the state or federative entity I was born... But I'll accept this one, as not all states have only one city as DF does

Quite a nice catalog of redundancies. As a cherry on top of the cake, the phone number states I should write my long distance phone code (LADA historically, Larga Distancia Autom tica, Automatic Long Distance) only if I am in any of the states. I can only ask myself why... Anyway... Lets continue filling paperwork. Grah. Hopefully I will be able to get my papers... somewhen in the next half century.

Cropping images with GDAL I am working to get a better integration between Meteosatlib and GDAL. A nice aspect of GDAL is that it allows to create read/write drivers around two functions: Open and CreateCopy. Open opens a datset read only, and to implement that all you have to do is to implement read access to your data using the GDALDataset interface. To implement CreateCopy, all you have to do is to create a new file reading information from a GDALDataset; then call Open on it. This means that there is no need to support incremental updates, and that all the data required to create a new file is readily available. This simplifies matters a lot. GDAL provides some interesting image manipulation functions, that can work over just these two calls. The way it does it is by exploiting the concept of a virtual dataset, which wraps an existing dataset but changes some parameters on the fly. This means that you can wrap a read only dataset with a virtual dataset that transforms it somehow, and then pass the virtual dataset to CreateCopy to save the transformed image in a format of your choice. On top of that, for many transformations you do not need to create your own virtual datasets, but you can use the functions provided by the VRT GDAL driver. One can learn a lot on how to use VRTDataset by reading the source code for gdal_translate. By doing that I could come up with this code for cropping an image, which is an interesting VRTDataset example.

GDALDataset* crop(GDALDataset* poDS, int xoff, int yoff, int xsize, int ysize)
 
        VRTDataset *poVDS = (VRTDataset*)VRTCreate(xsize, ysize);
        // Copy dataset info
        const char* pszProjection = poDS->GetProjectionRef();
        if (pszProjection != NULL && strlen(pszProjection) > 0)
                poVDS->SetProjection(pszProjection);
        double adfGeoTransform[6];
        if (poDS->GetGeoTransform(adfGeoTransform) == CE_None)
         
                // Adapt the geotransform matrix to the subarea
                adfGeoTransform[0] += xoff * adfGeoTransform[1]
                        + yoff * adfGeoTransform[2];
                adfGeoTransform[3] += xoff * adfGeoTransform[4]
                        + yoff * adfGeoTransform[5];
                poVDS->SetGeoTransform(adfGeoTransform);
         
        poVDS->SetMetadata(poDS->GetMetadata());
        // Here I also copy metadata from my own domain
        char **papszMD;
        papszMD = poDS->GetMetadata(MD_DOMAIN_MSAT);
        if (papszMD != NULL)
                poVDS->SetMetadata(papszMD, MD_DOMAIN_MSAT);
        for (int i = 0; i < poDS->GetRasterCount(); ++i)
         
                GDALRasterBand* poSrcBand = poDS->GetRasterBand(i + 1);
                GDALDataType eBandType = poSrcBand->GetRasterDataType();
                poVDS->AddBand(eBandType, NULL);
                VRTSourcedRasterBand* poVRTBand = (VRTSourcedRasterBand*)poVDS->GetRasterBand(i + 1);
                poVRTBand->AddSimpleSource(poSrcBand,
                                xoff, yoff,
                                xsize, ysize,
                                0, 0, xsize, ysize);
                poVRTBand->CopyCommonInfoFrom(poSrcBand);
                // Again, I copy my own metadata
                papszMD = poSrcBand->GetMetadata(MD_DOMAIN_MSAT);
                if (papszMD != NULL)
                        poVRTBand->SetMetadata(papszMD, MD_DOMAIN_MSAT);
         
        return poVDS;
 

This function wraps a dataset with a virtual dataset that crops it. Just pass the resulting dataset to GDALCreateCopy to save it in the format that you need.

Agile Business Analysts? I have recently been fleshing out my thoughts on the role of business analysts in an Agile team. This is something I have historically addressed on a case-by-case basis with clients, but when thinking about it last week at a client site, I realized that I do have a general take on how the role and responsibilities of a business analyst change when a group moves to using an Agile process. Per the usual, I ll be using Scrum as the example process where needed. The Classical Roles Historically the business analyst has had two primary roles. In a process where cross-functional communication and collaboration are minimized, these roles are essential. What little communication that takes place between the business and the software development teams purporting to serve them passes almost exclusively through one or more business analysts. I view the business analyst in a phased development approach as having two roles; Translator and Gatekeeper. The Translator The business analyst is primarily tasked with taking the needs of the business and translating them into a written document that is then handed to the software developers and testers. This role s necessity is based upon some fundamental assumptions. First, programmers and testers are on the whole, too socially retarded to actually talk to the business persons and find out what they want. Second, the business is primarily viewed as being unable to focus its thought long enough to tell the programmers and testers what they need. Since a key value of an Agile approach to software development is to have individuals and their interactions be the driving force rather than processes an tools (see The Agile Manifesto), the obliteration of this role is a primary objective. The Gatekeeper Business analysts are also viewed the gatekeepers for requirements in classical, phased software process approaches. This is a somewhat unfair role, since they rarely have the authority to prevent business or technology changes, like a meter maid trying to stop an armed terrorist. It s even more unnatural since change always happens, so it s like the meter maid being assigned to an anti-terrorist unit with no additional training, authority, or weapons. The Agile Roles Some of the more extreme practitioners of Agile methodologies say that business analysts have no role in an Agile team. I disagree, provided the role of the business analyst is allowed to evolve. The roles I ve seen them take on as valuable members of an Agile team are Facilitator, Historian, and Journalist. The Facilitator A high degree of collaboration and interaction take place between business and technology in Agile Software Development. The business analyst can serve a critical need in these interactions, facilitating communication between business and technology and making sure that critical areas are being covered in an interaction. This can be a more fulfilling experience for an analyst, since they often talk to one side, then go to other to pass on the information, all the while thinking, Man, this would be simpler if you were both in the room with me at the same time. The Historian The business analyst s documentation skill is excellent for capturing significant information that often gets lost in a team of purely technical people. I have seen some great uses of business analysts in this area. One is to document the system that the team actually builds (not the big, up-front imagined system that is covered in a phased design stage). Another is capturing key technological and architectural decisions and the context in which they were made, so that when a group revisits certain items asking, Why in the world did we decide to do that? , they have the means to be reminded or informed why a particular path was chosen. The Journalist The business analyst can also be the team s journalist, making sure the latest information makes it out to all interested parties. One creative approach I ve seen at a client site is a business analyst who has a project blog. He posts entries after each meeting between the business and technology, documenting what new user stories were created (even scans in the story cards!), a summary of the discussions held, and documents any key decisions that might have been made. Oh, and he provides an excellent executive summary at the top that s suitable for anyone s review, right up to the CEO. Tell me you wouldn t love to have that guy in your Agile team.

Getting to X After having prepared a bunch of libraries, the idea was to get X in shape. I d like to thank Julien Cristau very much for his support all along the way, since I knew nothing about X. Basically the plan was to get:

• xserver-xorg-core-udeb: X itself.
• xserver-xorg-input-evdev-udeb: for input, as you probably guessed.
• xserver-xorg-video-fbdev-udeb: for output, as you also probably guessed.

Not much to say about the former, it only pulled two additional libraries compared to what was previously udeb-ified: libaudit0 and libselinux1. Since security modules aren t the primary concerns in this exercise, --enable-xselinux quickly became --disable-xselinux and those dependencies went away. Both drivers were trivial to tweak, nothing really worth noting there. Once those 3 additional packages added to localudebs/ and to pkg-lists/local, the mklibs tool really wasn t happy with them during the rebuild of the netboot image, since the drivers are X plugins, meaning they need symbols which aren t defined in other libraries. Example:

No library provides non-weak EventSwapVector

Since the idea is to get something working rather than getting something perfect at once, d-i s packages/mklibs/src/mklibs.py (installed as /usr/bin/mklibs) was told to stop raising an exception, and to print a warning message instead. Rebuilding an image was then working again. It took some time, but finally it got noticed those packages were actually all installed in the initramfs, so there was no need to try and load/install them manually during the installation process, everything was there already, and it was sufficient to switch to VT2 to see the /usr/bin/Xorg binary. Some packages were still needed: xkb-data, and x11-xkb-utils. The latter also pulled libxkbfile1, but well, an additional library doesn t sound very scary. At this point, one has to try and run Xorg. The result was a black screen, and no means of getting back to another terminal, be it inside Qemu or VirtualBox. Troubleshooting X First idea: adding the openssh-server-udeb package to the image, and start sshd from there. But then one has to go through the missing config file, tweaking privilege separation, generating a host key, setting a password, etc. And for whatever reasons, it wasn t possible to access it. That s where the DebianInstaller/NetworkConsole page on the Debian Wiki came into play: it includes a few steps to get a working sshd within d-i, painlessly. So, X was actually starting properly, but getting a black screen is now the default. One wants X -retro to get the well-known old behaviour. From udevadm info --export-db, it looked like input stuff were missing, and indeed, udev-udeb doesn t include /lib/dev/input_id and /lib/udev/rules.d/60-persistent-input.rules. Since scp seems broken, wget from the host did the trick followed by chmod +x for the input_id file, otherwise that doesn t work so well. But still, there were some missing evdev stuff. And indeed evdev wasn t to be seen in lsmod, nor available to be modprobe d. That s where it became a little dirtier: wget and udpkg on mouse-modules-2.6.30-2-amd64-di, followed by depmod to make sure, and then modprobe was OK. A little udevadm trigger run to ensure udev was fully aware of the new situation, and restarting X led to a moving cursor there, as well as a working keyboard, since it was possible to switch between terminals! Prove it! Alright, I m proudly providing you with a screenshot: (Really, I ve never been so happy to see this fracking black-and-grey-grid screen. I kid you not!) If we forget about the very few network-console-related questions answered to get an sshd running, please note that the installation process is still just starting, the very first debconf question is still unanswered. Now, patching udev again so that udev-udeb includes the two mentioned files was almost easy (details aren t important anyway), while pulling evdev.ko module automatically was trivial, just a matter of adding such a line to a packages list (I picked pkg-lists/netboot/amd64.cfg):

mouse-modules-$ kernel:Version 

Which means one can now, out of the box:

• Boot the ISO, select Install.
• Wait for the first question to appear.
• Switch to another terminal, and activate it.
• Run Xorg -retro there.

Next step Rework the Pango/Cairo/Gtk stack to use X11 instead of DirectFB.

Producing udebs How to patch That s actually quite trivial. Most packages from Josselin s 1st and 2nd lists are libraries, and generating udebs for libraries is easy. Basically it boils down to declaring a new binary package:

• Duplicate the libfooN entry in debian/control.
• Add the -udeb suffix to the Package line.
• Add a Package-Type: udeb line.
• Make sure Section is debian-installer.
• (Maybe) make sure Priority is only optional.

Now, populating it might depend on the considered source package, but thankfully, pkg-x11 packages are quite sane, using debhelper s dh_install. So duplicating debian/libfooN.install into debian/libfooN-udeb.install is usually sufficient. It s a bit trickier when dh_movefiles is used, but calling install and cp -r after the dh_movefiles call should be sufficient. Next step, tweak shlibs for libfooN. Without that step, a libbarM package built against libfooN would get libfooN in Depends, which is not appropriate. The idea is to get libfooN-udeb instead. Trivial: add --add-udeb=libfooN-udeb to the dh_makeshlibs call. Possible issues at that step:

• debian/shlibs or debian/libfooN.shlibs is used instead of dh_makeshlibs. Trivial fix: feed its contents to dh_makeshlibs through its -V argument, and delete that file. Caveat: when it comes to applying a patch removing that file, make sure it s actually removed, and not just empty. I m going to file a bug about that, the default behaviour with empty files seems a bit buggy.
• Several libraries are shipped: one can t pass several --add-udeb arguments. Solution: use several dh_makeshlibs -p$pkg -V$depends --add-udeb=$pkg-udeb call. (I thought it was needed for libx11 but I can t read, actually: libxcb1 libx11-xcb1.)

Various checks to perform:

• Make sure debian/libfooN/DEBIAN/shlibs is correct: it should have a udeb line. Example:

  libfoo N libfooN (>= some-version)
  udeb: libfoo N libfooN (>= some-version)
  

• Make sure the generated udebs have no non-udeb packages mentioned in Depends. Since I m still at the proof-of-concept stage, I ve iterated when I encountered such additional packages, without checking what was actually needed. The idea is to get rid of extra stuff once the final needs figured out. Of course, one has to track down the dependencies to get the packages incrementally built and installed (for the updated shlibs files).

Results (These files might be moved around during the following steps, but this post shall be kept up-to-date and point to them at all time.) Legend:

• Red box: package needing a patch.
• Green box: package already udeb-aware.
• Label with [n]: package patched at step N.
• Label with a *: warning when patching, shlibs file to be removed.

Graphs are scaled down to 75%, click for full size. Step 1:

• Patches: udebs-1/diffs/
• Udebs: udebs-1/udebs/
• Notes: as explained above, libx11-xcb1 is probably not needed.
• Dependency graph:
  

Step 2:

• Patches: udebs-2/diffs/
• Udebs: udebs-2/udebs/
• Dependency graph:
  

Adding udebs to a d-i image Basic d-i image General guidelines (see d-i documentation for detailed installation, needed packages, etc.):

svn co svn://svn.debian.org/svn/d-i/trunk/ debian-installer-trunk
cd debian-installer-trunk/installer/build
make build_netboot

If that breaks (e.g. due to missing packages), make sure there s a similar entry in sources.list.udeb:

deb http://yourmirror/debian squeeze main/debian-installer

If everything goes well, the generated image should be available as dest/netboot/mini.iso, and tmp/netboot/udeb.list should contain the list of embedded udebs. To give it a try:

qemu-system-x86_64 -cdrom dest/netboot/mini.iso

Actually adding udebs Drop all additional udebs under localudebs/. Needed Packages and Packages.gz should be generated on the fly (but beware, secure apt is then disabled, even for non-local udebs). Now, tell d-i we want some more packages. If all of them are wanted, one can do that:

for i in localudebs/*.udeb; do \
  dpkg-deb --showformat='$ Package \n' -W $i; \
done > pkg-lists/local

(Update: Thanks to Guillem for pointing out a better and cleaner way than dpkg --info $i grep Package: awk ' print $2 ') Generate a new image (note the rebuild part of the target):

make rebuild_netboot

Make sure they got added by looking in tmp/netboot/udeb.list and profit! What s next Future steps include:

• Making sure those udebs are actually usable.
• Reworking Pango/Cairo/Gtk udebs.
• Taking a look into X itself.

Stay tuned.

Yesterday, and about nine days after release 0.7.3 of Rcpp (a set of R / C++ interface classes), Romain and I released version 0.7.4. It has been uploaded to CRAN and Debian, and mirrors should already have new versions. As before, my local page is also available for downloads and some more details. The release once again combines a number of necessary fixes with numerous new features:

• Building on OS X did not support multi-arch, and we are grateful for Simon who once again came to the rescue. Things should be fine now. The big take-away is that under no circumstances, include either a file configure or src/Makefile if you want multi-arch builds for free. As Rcpp is effectively a library to be used by other packages, this mattered.
• We added a file NEWS from which I include the relevant section below.
• Much more code re-organisation and enhancement making passage of various C++ types even easier -- see the NEWS entry below.
• More unit tests, now including ones for the 'old Rcpp API'.

Post-release, I also reworked the doxygen setup slightly so that all examples are now browseable, and the whole documentation is now searchable as well. Lastly, we had a remaining Windows build issue. Also, Brian Ripley and Uwe Ligges kindly sent us a small patch supporting the new Windows 64-bit builds using the new MinGW 64-bit compiler for Windows -- so release 0.7.5 may follow in due course. The NEWS file entry for release 0.7.4 is as follows:

0.7.4	2010-01-30
    o	matrix matrix-like indexing using operator() for all vector 
    	types : IntegerVector, NumericVector, RawVector, CharacterVector
    	LogicalVector, GenericVector and ExpressionVector. 
    o	new class Rcpp::Dimension to support creation of vectors with 
    	dimensions. All vector classes gain a constructor taking a 
    	Dimension reference.
    o	an intermediate template class "SimpleVector" has been added. All
    	simple vector classes are now generated from the SimpleVector 
    	template : IntegerVector, NumericVector, RawVector, CharacterVector
    	LogicalVector.
    o	an intermediate template class "SEXP_Vector" has been added to 
    	generate GenericVector and ExpressionVector.
    o	the clone template function was introduced to explicitely
    	clone an RObject by duplicating the SEXP it encapsulates.
    o	even smarter wrap programming using traits and template
        meta-programming using a private header to be include only
        RcppCommon.h
    o 	the as template is now smarter. The template now attempts to 
    	build an object of the requested template parameter T by using the
    	constructor for the type taking a SEXP. This allows third party code
    	to create a class Foo with a constructor Foo(SEXP) to have 
    	as<foo> for free.
    o	wrap becomes a template. For an object of type T, wrap<t> uses
    	implicit conversion to SEXP to first convert the object to a SEXP
    	and then uses the wrap(SEXP) function. This allows third party 
    	code creating a class Bar with an operator SEXP() to have 
    	wrap for free.
    o	all specializations of wrap :  wrap<double>, wrap< vector<double> >
    	use coercion to deal with missing values (NA) appropriately.
    o	configure has been withdrawn. C++0x features can now be activated
    	by setting the RCPP_CXX0X environment variable to "yes".
    o	new template r_cast<int> to facilitate conversion of one SEXP
    	type to another. This is mostly intended for internal use and 
    	is used on all vector classes
    o	Environment now takes advantage of the augmented smartness
    	of as and wrap templates. If as<foo> makes sense, one can 
    	directly extract a Foo from the environment. If wrap<bar> makes
    	sense then one can insert a Bar directly into the environment. 
    	Foo foo = env["x"] ;  /* as<foo> is used */
	Bar bar ;
	env["y"] = bar ;      /* wrap<foo> is used */    	
    o	Environment::assign becomes a template and also uses wrap to 
    	create a suitable SEXP
    o	Many more unit tests for the new features; also added unit tests
        for older API

As always, even fuller details are in the ChangeLog on the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

How a window manager with tiling layouts makes a 1024x600 resolution livable. I spend a lot of time in front of a screen, and for the past year I have used only my netbook. No desktops, no laptops. Just a cheap $200 computer that is nearly a kid's toy. The most challenging thing about this is dealing with the 1024x600 screen resolution. When I moved to this screen from something with many more pixels, it at first seemed very cramped. After a year, I think that the most important part of coping with the small screen is the flexability and customizability of layouts provided by the Xmonad window manager. In this post I will explain a few layouts I have developed for fitting specific tasks onto the netbook screen. (I assume that you know maybe 0.1% of Haskell (about what I do), and can read Haskell code without having a brain aneurysm.) web browsing What the netbook is designed for. You just want a web browser, taking up the full screen, and with its own tabs. So far, so easy: that's Xmonad's Full layout. But, you sometimes want to see two websites side-by-side. This layout accomplishes that, allowing the sizes to be adjusted as needed. It also uses Xmonad's Magnifier to zoom the smaller window when it's focused, which is useful if you briefly need to see more of a web site.

mySplit = magnifiercz' 1.4 $ Tall nmaster delta ratio
    where
        -- The default number of windows in the master pane
        nmaster = 1
        -- Percent of screen to increment by when resizing panes
        delta   = 3/100
        -- Default proportion of screen occupied by master pane
        ratio   = 60/100

web development If you're doing web development, you'll want probably one big web browser window, but also with a nice wide terminal on the same screen, in order to see web server logs. A simple way to do this is to mirror Xmonad's Tall layout by 90 degrees.

myWide = Mirror $ Tall nmaster delta ratio
    where
        -- The default number of windows in the master pane
        nmaster = 1
        -- Percent of screen to increment by when resizing panes
        delta   = 3/100
        -- Default proportion of screen occupied by master pane
        ratio   = 80/100

The myWide layout uses the full screen width for a single terminal, and splits the width when there are more. This is sufficient for viewing logs and doing minor things at the shell prompt, in between testing the result in the web browser. Of course a terminal can be temporarily moved to the master area by pressing mod-return, if you need it to be larger. chat For chat, you want to put the buddy list on the side of the screen, and use the rest for conversation windows. Xmonad's IM module takes care of allocating a sidebar on the screen for the buddy window. The rest of the screen can be occupied by any layout you choose. I like to have the conversation windows be as wide as possible, and typically only want to see two conversations at a time. But sometimes I might have a dozen visible. A good layout to cope with those needs is Grid.

myChat' l = withIM size roster l
    where
        -- Ratio of screen roster will occupy
        size = 1%5
        -- Match roster window
        roster = Title "Buddy List"
myChat = myChat' Grid

The withIM layout puts the buddy list on the left; I prefer it on the right for some reason, so I tweaked my layout to do that. All it took was using the reflectHoriz layout modifier to get a mirror image of the layout. Then I reflect the inner layout back to its normal orientation. Being able to throw in a few function calls and mutate a layout like that is where Xmonad shines.

myChat' l = reflectHoriz $ withIM size roster $
            reflectHoriz $ l

logs I have a dedidated workspace that I use to tail logs, and as a place to send long-running tasks (such as compiles). The layout for this needs to keep windows wide, to see whole logged lines, but they need only be five or so lines tall. And it's convenient to have one bigger window with the small ones below. Xmonad has a perfect layout for this, called Dishes, because it's sorta like a stack of plates.

myDish = limitWindows 5 $ Dishes nmaster ratio
    where
        -- The default number of windows in the master pane
        nmaster = 1
        -- Default proportion of screen occupied by other panes
        ratio = 1/5

Notice that the screenshots above also have Procmeter in a sidebar on the right. I (ab)used the withIM layout to do that:

myLogs' l = reflectHoriz $ withIM procmeterSize procmeter $
              reflectHoriz $ l
    where
        -- Ratio of screen procmeter will occupy
        procmeterSize = 1%7
        -- Match procmeter
        procmeter = ClassName "ProcMeter3"
myLogs = myLogs' myDish

coding Everything above was easy. Doing coding (or writing) on a small screen is where it gets hard. When I'm coding I want to have one window that is exactly 80 columns wide, and as tall as possible, where I do the main development. Then I need a minimum of two other windows also visible: one action window for running tests and the like, and one to view references (emails, man pages, other files, etc). I may need to view any of these windows fullscreen at any time, of course. Often, deep into something, I will accumulate many other reference and action windows. But three seems to be the magic, minimum number for me; use of screen or anything else doesn't reduce it; if I can't see all three at once, I will waste time and concentration flipping back and forth. The Xmonad layout I've developed to handle this is based on the handy FixedColumn layout, which automatically keeps the master window 80 columns wide. I modify this by adding the Magnifier, so that my action and reference windows "pop out" over the master window when focused.

myCode = limitWindows 3 $ magnifiercz' 1.4 $ FixedColumn 1 20 80 10

The 1.4 was carefully tuned, to magnify a window to be big enough to be usable for editing, reading, or command-line, while not obscuring too much of the other two windows. And again I use limitWindows, to only show three windows, in order to keep them all as big as possible. Other windows are hidden offscreen, but I can rotate to them with alt-tab as needed. reading ebooks When I'm reading an ebook, I want to have the reader in the middle of the screen. But a widescreen netbook like mine is too wide to comfortably read text all the way across, so it's nice to have margins. (In the margins, I put terminals, or notes.) The ThreeColumns module has a layout that does just that.

myBook = ThreeColMid nmaster delta ratio
    where
        nmaster = 1
        -- Percent of screen to increment by when resizing panes
        delta   = 3/100
        -- Default proportion of screen occupied by master pane
        ratio   = 2/3

putting it all together I join all my layouts together, so Xmonad will allow switching between them. These three combinations all have the same layouts, only ordered differently, so that different workspaces can have different default layouts.

codeFirst  = myCode    myWide    myGrid    myDish
dishFirst  = myDish    myCode    myWide    myGrid
gridFirst  = myGrid    myDish    myCode    myWide

Xmonad layouts can be decorated with things like title bars. To save horizontal space, I leave off the title bars on layouts where they are not needed to disambiguate windows. Two helper functions can be applied to add or remove titles (and also cause the layout to avoid the Gnome panel.)

withTitles l = noFrillsDeco shrinkText myTheme (desktopLayoutModifiers l)
noTitles l = desktopLayoutModifiers l

I have semi-dedicated workspaces; one for each of the above activities. To assign a layout to a workspace, I use the PerWorkspace module.

perWS = onWorkspace "logs" (withTitles $ myLogs' dishFirst) $
        onWorkspace "web"  (noTitles   $ (mySplit   myWide)) $
        onWorkspace "chat" (noTitles   $ myChat' gridFirst) $
        onWorkspace "book" (noTitles   $ myBook) $
                           (withTitles $ codeFirst)

Finally, I allow toggling between the currently selected layout and fullscreen mode, and use smartBorders to avoid displaying borders when there is only one window onscreen.

myLayout = smartBorders $ toggleLayouts Full perWS

Xmonad layout optimised for the small screen of a netbook: Done! (For now...)

---

---

PS: Thanks to #xmonad regulars for always having the answer I need up their sleeves.

Some what belatedly (hey I'm travelling!) I've changed the layout of my main website http://www.enc.com.au/ so it now has black panels and a little pop-up. It's all in the name of T he Great Australian Internet Blackout which is a protest about the upcoming Australian Governments Internet Filtering.

As per usual with this sort of thing, the proposed solution will not even meet its rather vague goals, unless you count filtering a 1000 or so websites a "goal". While the numbers vary, the best estimates is its way over 100 to 200 million websites. So that 1000 is 0.0005, or 1 in 200,000.

The filter is alternatively going to filter reported refused classification material or perhaps "other stuff"; they'll get back to you on the second one. For the first it works that somone finds a website of sufficient level of classification, reports it to the government and it gets filtered. And how many times have you come across RC websites, just by wandering around? That's not R rated websites, its not porn, its not even reasonably exotic porn, but stuff that's downright nasty.

The government, as per usual, has come out with the tired old protect the children rant. The filter doesn't stop spam, it doesn't stop weirdos on facebook or MSN chatting up your teenage daughter or son, it filters websites that, unless you or your child is REAL unlucky you will never see; ever.

So why am I against it? Surely filtering RC level websites is great right? And to be honest if that is all this ever was going to be, it's still a complete waste of my (and if you are an Australian Internet user) your money, but once the filtering is in, there is going to be a line outside the relevant ministers door to filter anything else anyone wants to filter, for example:

• Games that are available outside Australia, if they have not been classified here then they shouldn't be available here
• Torrent trackers, because the industry would no doubt tell the government that the only thing they're good for is illegal download of movies or music
• Websites about certain topics that some people would rather not be seen, there is a large mountain of stuff that fits in here depending which crank has their own barrow to push

This sort of filtering while initially seeming ok, will get worse, much much worse. There is not a real strong push from the general public about filtering RC websites, most people see and hate spam much more. If you are an Australian citizen I ask you to visit the The Great Australian Internet Firewall website and write to your local member of parliament.

Related articles by Zemanta

• Australian 'Net filter to block video games, too (arstechnica.com)

I have lately been reading Homer s epic poems: first The Iliad, and now I am nearly done with The Odyssey. I figure there isn t anyone alive today that believes that Zeus literally caused thunder in answer to a prayer, or that Athene really transformed Ulysses between having a youthful and an aged physical appearance at a whim. Despite our understanding that these poems don t reflect a literal truth, we still find meaning and truth in them. It is for this reason that they are read by high school and college students all over the world. This same reason drives our reading of more modern plays and novels everything from King Lear to Catcher in the Rye. We learn something of the author s world, something about our world, and if we are truly lucky, a deeper understanding of the universal truths of human life. And it is with that preface that I suggest that the Old Testament or parts of it, at least ought to be read in the same manner. Modern Christianity speaks of a loving, caring God, one who is deeply concerned for the wellbeing of all. Under this understanding, forgiveness is more desired than retribution, and helping the week is better than enslaving them. How then can one square that with a literal reading of the Old Testament? This was a key question I asked over a span of perhaps 15 years. I was perplexed that the God of Love ought to turn someone into a pillar of salt for turning her head the right way, that almost all life on earth might be extinguished by a flood, that slavery is condoned and regulated, and all sorts of people being stoned to death, animals killed for no reason. In short, the God of the Torah, at least, didn t seem to me to be even the same person as the God the Church talks about. I raised this question with many people, and there was even a seminar on it at a convention I went to in 2001. The answers I got usually were of one of two types: 1) God is beyond our comprehension, and this is one of the mysteries we will never understand because that s just the way it is; or 2) the arrival of Jesus changed things, and it s impossible for a modern person to fully appreciate the laws as they existed prior to that. These are really two sides of the same stick: they re both saying, Yep, that s odd. But we have to believe that the Bible is inerrant and literally true, so we just have to accept the mystery and move on. Except I m not so good at accepting mysteries and moving on. It strikes me as odd that nobody even mentioned the third option: that some of the stuff in the Old Testament is, to be blunt, made up. This even though I have come to learn later that some of those people probably believed this to be the most correct explanation. Now, that doesn t mean it has no value or that is doesn t show us truth. Romeo & Juliet was made up, but we learn from it. A typical example of this is the creation myth. There are some that are very defensive about it, perhaps thinking that it weakens their religion to admit it might not be literally true. To me, I find that insisting upon its literal truth weakens the religion; can we not see how a piece of literature speaks to us today and leave it at that? Need we say that Shakespeare s Julius Caesar is an inferior play because it is a work of historical fiction? The position I suggest here is not some crazy nutjob position. Wikipedia has a concise overview of some of the scholarship surrounding these ideas. I now count myself as somewhat inspired by Homer to read the Old Testament in the same way that I read Homer: as a story that can speak to us today, one that inspired a nation in captivity and after, and launched perhaps the most amazing religious movement in history. I only wish that more people would admit the possibility of a non-literal reading of the Bible. This return to an earlier era of Christianity is, in my mind, the only way that Christianity can maintain its relevance in this age. Update: A note I received suggests I ought to make a bit of a clarification. I am not bothered by the fact that people have differing opinions about the historicity of Genesis. I m all for putting all the opinions out there for sure. I think that really the concern over whether Genesis is literally true or not is mostly irrelevant. I have no problem with Christians that find Genesis to be literally true. What I m lamenting is the attitude that you re not Christian if you re not sure that Genesis is literally true or saying anything else about Genesis undermines Christianity. I believe neither of those statements, and would really rather that we collectively got past the creationism vs. evolution debate already. Update 2: It appears that my use of a bit of technical language has caused some confusion. A creation myth can be defined as a supernatural story or explanation that describes the beginnings of humanity, earth, life, and the universe (cosmogony), often as a deliberate act by one or more deities. It is a category of explanations. Simply calling the Genesis story a creation myth is an act of categorization only, and doesn t imply anything about its accuracy or value.

Of course, one would expect me to react to this post. Apparently, from my cumulated experience, joking about funny translations is a very widespread game among german-speaking developers (of course, as everybody knows, us French do translate everissing and no single Frainche DiDi will make such fun abaouttel10n wouhorke, aha). I'd say: more generally widespread among people .who jargonize about computing so often in their daily work that they often forget that not everybody, even in a country where people are as clever about English as German/Austrian/Swiss folks are (we all know that 100% population in these places speak a very perfect English) do understand what "downgrade" means. Seriously, "funny" translations are our (translators) daily nightmare. We hate them because we know people hate us (or make jokes about our work) when they face them. The example taken by Patrick is a nice case as "downgrade" is tricky for all of us. For instance, French translators have chosen "revenir la version pr c dente" as "bring back to former version". This is about the only "right" way to say what's being done ("d grader" is another option but that has a negative connotation). Of course, I often *hear* people (incl. me) say "downgrader" which is an horrible neologism....and of course doesn't say anything to people who are clueless in English. English is, unfortunately, often a very compact language. This is our daily pain. So, as German translation of "to downgrade", what do *you* propose, Patrick? "Downgradieren"? Tricky, eh? Taken from APT for "downgrade":

• German: deaktualizieren
• French: revenir la version pr c dente
• Spanish: desactualizar
• Italian: retrocedere
• Danish: nedgraderingen (hope this is the inifitive, just guessing)
• Dutch: degraderen
• etc.

Hint: you can see this with "apt-get source aptitude" and look in po/ for you favorite language. That sounds funny to you? Fine. But as usual in free software, please report.....and patches welcome...

As a german I'm used to strange translations in computer context.
I saw it back when I was using Microsoft products and I regulary stumble upon it on Debian systems. But whats actually kind of funny:Debian is outstanding in that regard.
An example:
Debian has that "wonderful" package manpages-de which provides a german manpage for ps.
It contains a formulation that is so inaccurate and not really funny:

"--cumulative Daten von toten Kindern einbeziehen (als Summe zusammen mit den Eltern)"

For the English only speakers: That can be roughly translated to "Collect data from dead children
(summed with their parents)". For me this somehow sounds like we are a butcher OS and so I reported #495441 but nobody cared about it yet.

Today I had another WTF on german translations because when I wanted to upgrade my system I read the following sentence:

"Sind Sie sich sicher, dass Sie die oben genannten Pakete installieren bzw. aufr sten wollen?"

Cold war, anyone?

Well its a classical case of using a word-by-word translation instead of a meaning-orientied translation. Its a fact that 'upgrade' can be translated to 'aufr sten' (as done above) but in that case its just not appropriate. As a matter of fact most german people I'm aware of would actually confuse the meaning with a meaning such as in a military context (therefore the "Cold war, anyone?" text above). And just btw. if we would rely on computing power to translate it back into English we would get:

Are you sure itself that you want to install and/or rig the packages specified above?

So it seems that Babelfish has a similar pereception.

Chris Smart writes about the latest money making schemes for OS distributors, Canonical is getting paid by Yahoo to make them stop using Google as the default Firefox search engine [1]. I think this is OK, the user can easily change it back if desired and it allows them to pay the salaries of more employees who contribute code back to upstream projects. Above are sections of my Webalizer output related to my blog which show the data transfer use of search.msn.com (Bing presumably) which is 50% greater than that of Google and Yahoo combined. Why does MSN need to do 455MB of transfer so far this month to scan my blog when Google gets the job done with 189M and Yahoo only takes 109M? Also judging by the referrals Bing is only 3% as much use to me as Google. Above is a sample of the Webalizer output from www.coker.com.au, MSN is using 525MB of data to scan the site which contains about 1.2G of static files that change very rarely. A Russian malware site seems to be downloading it three times a month, and Google only takes 35MB of data transfer to scan the site (which is probably still excessive). If Bing was a quality search engine that returned appropriate results then this could be forgiven. But however it is a very poor search engine that returns bad results. For example if you query Google or Yahoo for bonnie++ you will get an entire page of search results concerning my Bonnie++ benchmark, and those results are ordered in a sensible way. If you ask Bing then the first four results concern Bonnie (three women and a plant) and most of the first page don t concern my benchmark. Some time ago I had blocked MSN from scanning a server that I ran. The server in question had all the web servers for my domain plus quite a few other small domains. The total MSN data transfer was 3G per month which was almost half the data allowance for the server in question (data plans in Australia suck that s why my web servers are hosted in Germany now), so it was a question of whether to allow normal operation of the business or MSN searches. With Microsoft not running a popular search engine (then or now) it was an easy decision. I think that anyone who accepts money from Microsoft/Bing is doing their users a mis-service. Bing is simply an inferior search engine, it gets bad results and imposes excessive costs on service providers. Yahoo however seems to be a reasonable service, not as good as Google for the web hosters but not too bad. I wonder what would happen if Yahoo offered some sponsorship money to the Debian project in exchange for being the default search engine. I m sure it would be dramatic.

• [1] http://blog.christophersmart.com/2010/01/27/yahoo-pays-canonical-to-switch-firefox-away-from-google/

I always thought Gammu to be workaround for phones whose vendors do not provide software for your favorite operating system. We try to do our best job to work with variety of phones, even though this is hard and never finished job. However today I was really surprised to find out that some phone vendor built their official application for managing phone on Gammu. The application is of course closed source and available only for Windows, but this does not change anything on the fact they use it. They seem to have modified code slightly, adding some new functions and removing others, separated the code to several DLLs, however I'm 100% sure it is Gammu as they use some unique (read awful) things we have in our code. I'm not going to disclose their name right now as I want to give them fair amount of time for reaction on my asking for sources. But you will for sure find more information later in my blog. PS: Gammu is of course not the only free software used inside this application, you can find more there (and you will find there more than I did if you will look longer than few minutes I spent on examining rest of code).

I'm glad you asked, since I've been pondering this for a while. $subject is my new project slogan. Now I'm not sure whether we can actually use it, because a) it's stolen from Debian, and b) another (commercial, proprietary) database product already uses the "universal database" line.

I have come to appreciate that the "universality" of a software proposition can be a killer feature. For example, Debian GNU/Linux, the "universal operating system", might not be the operating system that is the easiest to approach or use, but once you get to know it, the fact that it works well and the same way on server, desktop, and embedded ensures that you never have to worry about what operating system to use for a particular task. Or Python, it's perhaps not the most geeky nor the most enterprisy programming language, but you can use it for servers, GUIs, scripting, system administration, like few other languages. It might as well be the "universal programming language". A lot of other software is not nearly universal, which means that whenever you move into a new area, you have to learn a lot of things from scratch and cannot easily apply and extend past experiences. And often the results are then poor and expensive.

The nice thing about PostgreSQL is that you never have to worry about whether to use it, because you can be pretty sure that it will fit the job. Even if you don't care whether something is "open source" or "most advanced". But it will fit the job. The only well-known exception is embedded databases, and frankly I think we should try to address that.

For a long time we had a bug against the archive asking us to export a file similar to the Maintainers index, just for the Uploaders (aka Co-Maintainers). Back then it wasn t that easy, but this changed. A while ago I also got a patch from Emilio Pozuelo Monfort against the first bug asking for it (#120262 !, and so went to integrate it. Of course this told us about a deficit in daks code parsing the Uploaders field. No, it is not good to solely split on a comma, when we have at least one Maintainer with a comma in their name field. We now split on the end of a mail address, followed by a comma. This simple idea is from Alexander Reichle-Schmehl, I was always thinking of a more complicated parser for it But heck, why? Yes, this can easily go wrong, if people, for whatever reason, add a space in there. But it only harms them, not us, so I don t think it is a big problem. Let us help you shot in your own foot. :) The new file will be exported as Uploaders.gz right besides the Maintainers.gz we already have on every mirror in the indices/ directory, starting with the next dinstall run (that will start in 2 hours and some 42 minutes, at time of writing this entry). Until then you can find a version at my space, to poke around and find bugs. A good example is right at the top - a52dec. Bug already filed, they missed a comma between two Uploaders. Feel free to hunt for your own bugs. :) And while I am writing already: Are you working with data generated out of the Debian archive, but have to generate the data yourself? (Like people had with the Uploaders index in the past, for example). Feel free to ask for it to be generated during dinstall. If the data is available (or can be made) and makes sense to have for more than just a personal pet project, we are happy to merge patches, help with it, provide it. (And really, asking cost nothing, we can t do more than deny it).

It is well known that apt has an issue when it comes to resolving circular dependencies. Therefore, Debian bug reporters have set out to eradicate circular dependencies from the archive. This does, however, add significant bloat to the actual packages, and I am questioning why this is really necessary. For example, aide. aide has an aide-common package, depending on aide aide-binary. aide-binary is a virtual package by aide, aide-dynamic, and aide-xen. All three providers of aide-binary depend on aide-common again, which is a circular dependency. If I now follow the request posed in #545852 and remove aide-common s dependency on aide aide-binary, I ll have to cater for the case of the aide-binary bein absent in each and every helper script included in aide-common, which would drive the script s complexity up to a new level. I really hate that idea. Is it really necessary to work around apt s shortcomings by eliminating the case that apt doesn t handle well from the archive, and driving up other packages complexity (and the possibility of bugs in this added complexity)?

It seems that Sun doesn't care much about getting bugs fixed in Java.This bug for example causes rendering artifacts in Apache Batik, and is very visible with many SVG files. It causes circles to be rendered as approximated diamonds. It has been reported 9 years ago (the first time, there duplicates).I understand that there are both more important bugs, and that one must avoid introducing new bugs when fixing bugs. But there should be little dependencies on a broken circle rendering routine, so please just fix this cosmetic bug, too. One of the reports is even staged "Fix understood" ...A more important issue with Sun Java (known since 2005) is this bug, which effectively breaks Java IPv4 networking on Debian unstable now (which recently changed the IPv6-to-IPv4 fallback behaviour). So far, Sun has rated this as "request for enhancement". WTF?Sure, you can work around the bug easily - change /etc/sysctl.d/bindv6only.conf to use the value of 0 instead to re-enable IPv4 fallback - but after all, IPv4 networking is pretty much an essential Java feature.

So, as I'm not always sitting directly in front of machines that I'm working on I've been using various tools to get temporary network connections between completely seperate network blocks, for example, sitting on wireless outside of my home, connecting back in to the desktop at home, and then connecting to the virtual machines on that desktop - the virtual machines are on their own private network, which is bridged and then routed through the desktop. The desktop machine has an ipv6 tunnel setup using tinc to my main VPS. That in turn has a tunnel setup to he.net's ipv6 network... I then have my laptop configured to also use tinc to connect to my VPS, so lots of ipv6 over ipv4, really. From that I can then get a 'direct' ssh connection between my laptop and my desktop (OK, so it's actually going over 2 ipv6 over ipv4 tunnels to get there, but it "looks" direct), so, that gets me as far as the external network on that box. From here, we can run a vde_switch which is connected to a tun/tap interface that's part of the internal bridge, once we've got that running, I bring up another vde_switch on my laptop attached to another tun/tap interface purely for that network with a static ip configure on that interface... using the magic of dpipe I then connect the two vde_switches together using vde_plug, and as if by magic, I then have access to the internal networking of my virtual machines, meaning that I can now ssh directly in to the virtual machines without having to do anymore interesting ssh bouncing and port forwarding. It's a little sick, but it really does make remote working a lot easier to do, it's a vpn of sorts, I suppose, as all traffic is being thrown over ssh pipes or tinc (which in turn is encrypted using x509 certificates)... probably not the quickest of networks, but very very nice all the same.