This spring I read a course on Python in Vilnius University, Department of Mathematics and informatics. The course was not mandatory, which I suspect is one of the reasons why my students were quite a bright and motivated bunch. I began the course with a brief introduction to the Python language and the standard library, then dedicated a few lectures to Django and in the remainder I covered various libraries (GUI toolkits, Pygame) and techniques (testing, debugging, optimization). One of the major tasks of the course (and the deciding factor in the final valuation) was to develop a project using Python. The students were given complete freedom over the topic of the project and the tools used, as long as Python was involved. The quality of the result varied, of course; had I given concrete tasks, the low bar would definitely have been higher. However, I am happy with how things turned out. Probably the best-executed project was 1000 Online, an online multiplayer card game (see rules). It was developed by Vytautas Karpavi ius. Vytautas apparently had prior experience with Python and Django, nevertheless, the quality of the app (and the code behind it) is impressive. Among the other projects, Gegute.lt, a Lithuanian clone of Reddit is still up and running. It's nothing too fancy, but looks quite clean for a university project. Other projects perhaps were not perfectly executed, but were very interesting conceptually. Jonas Keturka developed a bot (BitBucket) for the online strategy game Travian. Marius Damarackas developed a web-based guess-the-song game. He used YouTube for the song material and Last.fm (IIRC) to make you guess among similar songs, so that it would not be too easy. Andrius Chamentauskas developed a Puyo Pop clone (GitHub) with PyGame, complete with multiplayer and an AI. Paulius Budzinskas worked towards his course paper by developing a bacteria movement modelling and visualisation app (GitHub; Lithuanian only). David Abdurachmanov developed a spell-checker bot for WikiAnswers.com, with a really baroque and sophisticated architecture. I believe he has applied to Google Summer of Code with that project; I wonder how that is working out. For reference, you can find links to all student repositories on GitHub/BitBucket in this list. Beware: some of them may be documented/commented in Lithuanian. Another interesting effect of the course was that people started using Python in other related courses, for example in Mathematical Modelling and Numeric Methods. I even heard that the lecturer was impressed and expressed interest in learning Python. Another lecturer, this time a statistician, was interested in teaching and using SAGE at lab practice. Feeling such 'ripples' going through the department sure felt nice. By the way, the university also offers lectures on Ruby, read by Saulius Grigaitis. I took some inspiration from him for the course form and content.

I discovered this while doing automated testing of upgrades from Debian Lenny to Squeeze. A few packages in Debian still got circular dependencies, and it is often claimed that apt and aptitude should be able to handle this just fine, but some times these dependency loops causes apt to fail. An example is from todays upgrade of KDE using aptitude. In it, a bug in kdebase-workspace-data causes perl-modules to fail to upgrade. The cause is simple. If a package fail to unpack, then only part of packages with the circular dependency might end up being unpacked when unpacking aborts, and the ones already unpacked will fail to configure in the recovery phase because its dependencies are unavailable. In this log, the problem manifest itself with this error:

dpkg: dependency problems prevent configuration of perl-modules:
 perl-modules depends on perl (>= 5.10.1-1); however:
  Version of perl on system is 5.10.0-19lenny2.
dpkg: error processing perl-modules (--configure):
 dependency problems - leaving unconfigured

The perl/perl-modules circular dependency is already reported as a bug, and will hopefully be solved as soon as possible, but it is not the only one, and each one of these loops in the dependency tree can cause similar failures. Of course, they only occur when there are bugs in other packages causing the unpacking to fail, but it is rather nasty when the failure of one package causes the problem to become worse because of dependency loops. Thanks to the tireless effort by Bill Allombert, the number of circular dependencies left in Debian is dropping, and perhaps it will reach zero one day. :) Todays testing also exposed a bug in update-notifier and different behaviour between apt-get and aptitude, the latter possibly caused by some circular dependency. Reported both to BTS to try to get someone to look at it.

David Byrne gave an interesting TED talk about how changes to architecture drove changes to musical styles [1]. I think he does stretch the point a little. To a certain extent people develop the most complex instruments and the largest music halls that can be supported by the level of technology in their society people with a hunter-gatherer civilisation play drums because they can build them and can carry them. The NY Times has an interesting article about paternity leave in Sweden [2]. The Swedish government pays for a total of 13 months leave that can be split between parents for every child. Of those 13 months 2 months can only be taken by the father and that is likely to increase to a minimum of 4 months of paternity leave after the next election. Dan Meyer gave an interesting TEDX talk about how the current math curriculum in the US (as well as Australia and lots of other countries that do the same thing) is totally wrong [3]. His main point is that maths problems should be based on real-world use cases where not all needed data is immediately available and there is also useless data that must be discarded. He believes that the most important thing is developing mathematical problem solving skills basically the things that I did for fun when I was in primary school are skills that need to be taught to high-school students The Atlantic magazine has an amusing article by Daniel Byman and Christine Fair about the incompetent Islamic terrorists [4]. In Afghanistan half the suicide bombers kill only themselves and the US government has a lot of evidence of Taliban soldiers practicing bestiality and collecting porn. Islamic extremist groups are staffed by people who are bad soldiers and bad Muslims. Jon Masters wrote an interesting post titled What Would Jesus Buy about ethical purchasing decisions [5]. Jon references The Church of Stop Shopping which isn t a real religious organisation but a street theatre activist group. ZeroHedga has an insightful article comparing corporations and the US government to street gangs [6]. The conclusion is that when gangs take over a neighbourhood everyone has to join a gang for their own protection. Hillel Cooperman gave an interesting TED talk about being obsessed with Lego [7]. He compares Lego fans to Furries and makes a good case for this comparison. Marian Bantjes gave an interesting TED talk about her graphic art / graphic design work [8]. I ve never seen anything quite like this. Business Insider has an interesting article about oil cleanup, it seems that most people who worked on the Exxon Valdez disaster are now dead [9], s opposed to most people who worked in almost every other occupation at that time who are either still working or enjoying their retirement. The current gulf disaster is bigger, will require more workers for the cleanup, and can be expected to have a higher death toll. Some people claim that measures to reduce oil efficiency will impact the economy, how will millions of people who are chronically ill for the rest of their lives impact the economy? The NY Times has an interesting article on circle lenses [10], contact lenses designed to make the eyes look larger. It s illegal to sell contact lenses in the US without a prescription, but the latest trend is for women to buy them online in a variety of colors. The FDA should probably approve them, it would be better to have the quality controls you expect from a medical supply company instead of having people rely on Malaysian mail-order companies for the safety of their eyes. Don Marti has written an interesting article about the economic decline in the US, he suggests making pension funds invest in local jobs [11]. Companies are supposed to act on behalf of their stock-holders, but US companies often have the majority of their stock owned by the pension funds of workers but they act on behalf of a small number of rich people who own a minority of the stock. Don s article was inspired by Andy Grove s article in Bloomberg about the stagnation in technological development that has been caused by off-shoring the manufacturing [12]. Neil Brown has completed a test release of a new Linux software RAID feature for arrays with multiple redundancy that have bad sectors [13]. When a disk gets a bad sector the current behavior is to kick it out of the array, if you have two such errors on a 3 disk RAID-1 or a RAID-6 array then you lose all redundancy and are at risk of catastrophic failure even though in most cases both disks will still mostly work. With this patch some regions of the disk may be excluded but it can provide redundancy for other stripes. Thanks Neil for your great work here, and all your previous work over the last 10+ years! The RSPCA has a new campaign titled Close the Puppy Factories [14]. Dogs are kept in very poor conditions and forced to churn out puppies for their entire lives to supply pet stores. The RSPCA recommends that people buy puppies from registered dog breeders (not registered dog breeding companies ) and ask to see the dog s parents. They also recommend not buying from classified adverts or pet stores. Animal shelters have to euthenise huge numbers of unwanted animals, you can buy a pet dog or cat from an animal shelter for a small fee that covers the expenses related to housing and spaying it and save that animal from being euthenised! Maureen Dowd criticises the Catholic Church properly in an article for the New York Times [15]. The Catholic Church officially regards ordaining a woman and raping a child to be equally bad offenses. Frank Rich wrote an interesting column for the New York Times about Mel Gibson [16]. He describes the destruction of Mel Gibson s reputation as a symptom of changes in the culture in the US and also links it to the fall of Ted Haggard (who supported Gibson s most notorious movie The Passion of the Christ).

• [1] http://www.ted.com/talks/david_byrne_how_architecture_helped_music_evolve.html
• [2] http://www.nytimes.com/2010/06/10/world/europe/10iht-sweden.html
• [3] http://www.ted.com/talks/dan_meyer_math_curriculum_makeover.html
• [4] http://www.theatlantic.com/magazine/archive/2010/05/the-case-for-calling-them-nitwits/8130/
• [5] http://www.jonmasters.org/blog/2010/06/27/what-would-jesus-buy/
• [6] http://www.zerohedge.com/article/guest-post-corporate-entities-modern-day-street-gangs
• [7] http://www.ted.com/talks/hillel_cooperman_legos_for_grownups.html
• [8] http://www.ted.com/talks/marian_bantjes_intricate_beauty_by_design.html
• [9] http://www.businessinsider.com/warning-to-gulf-cleanup-workers-almost-every-crew-member-from-the-1989-exxon-valdez-disaster-is-now-dead-2010-6
• [10] http://www.nytimes.com/2010/07/04/fashion/04lenses.html
• [11] http://zgp.org/~dmarti/business/grove-on-jobs/
• [12] http://www.bloomberg.com/news/2010-07-01/how-to-make-an-american-job-before-it-s-too-late-andy-grove.html
• [13] http://lwn.net/Articles/391208/
• [14] http://www.closepuppyfactories.org/
• [15] http://www.nytimes.com/2010/07/18/opinion/18dowd.html
• [16] http://www.nytimes.com/2010/07/18/opinion/18rich.html

Normally, it's pretty easy in Rails to take advantage of the simple i18n tools provided. You just write I18n.t("some_key") and it'll translate that into whichever language. Today, however, I discovered what seems like a problem. If you do I18n.t(""), it returns a hash of all possible translations! And if you give it a nil, it returns an error because that's not a key in the DB. The first behavior doesn't sound correct at all, and the second seems dubious as well. I'm thinking about fixing it like so:

  def translate(key, options =  )
    return nil if key.nil?
    return "" if key == ""
    locale = options.delete(:locale)   I18n.locale
    backend.translate(locale, key, options)
  rescue I18n::ArgumentError => e
    raise e if options[:raise]
    send(@@exception_handler, e, locale, key, options)
  end

Is there any reason not to do this?

My project. This post assumes that the reader has read my last post. This weekly report is late, because I was too interested in the project to stop working on it to write. Next week it have to be earlier, since I m going to New York on Sunday, and will have to take a time from the project. First of all, I need to say that I thought all problems were gone in my last post, but I noticed after a lot of testing that there is a segfault (or memory corruption or etc) that happens about one of five times I ran the real/maillist benchmark. I tried a lot of combinations of parameters to the RTS (-C0 -A4k -i0 -DS and others) to make it behave deterministically, and it did, without segfaults. I could not reproduce the segfault deterministically, because all of these parameters made the segfault disappear. My old debugging technique, of printing a log to see where the program has went before the segfault happened, also is causing the segfault to disappear, so this one is being much harder to debug than the last ones. I m still looking at it. In a moment of this week I thought of pulling the latests patches from GHC, which created the need of rebuilding the whole system. I probably rebuilt with a different mk/build.mk, so I got some very distorted benchmark results when comparing before executions from before this build and after them. Because this distortion only made itself visible in a benchmark where I ve introduced some improvements which I was very positive about, I naively thought that the -46% results were actually real. After a reasonable disbelief of my mentor, I remembered the complete rebuild and thought that was the cause. I reran the pre-rebuild benchmarks, and got the updated results. This whole week I kept on benchmarking. At first I was using my usual system, but I noticed some very unexpected results, so I decided to run them in single-user mode, running nothing in parallel. In this condition, I belive the distortions are minimized. I organized my changes in sequential patches to the repository, to make it easier to measure. It takes a long time to run nofib, specially gc/gc_bench. While I organized the code, I notice the segfault was gone. I don t know still what was causing it, but I m not worrying very much, since it s gone. I m planning to write a bigger report next week with the complete results of the benchmarks.

I spent the last two days at the BSP in Munich thanks to everyone for inviting & feeding me, & especially to Stefan for hosting me again!

Unfortunately the weather was not good enough for an excursion to a Biergarten but the local chefs at the LiMux office delighted us with self-made spaghetti salmone & self-rolled sushi, & we spent quite some time behind our laptops.

The main results can be seen on the wiki page; my personal conclusion is that the list of remaining RC bugs contains mostly "hard stuff" (e.g. architecture-, hardware-, or desktop-specific bugs). Probably some packages will need to be kicked out of squeeze & some bugs might get a squeeze-ignore tag ... To put it positively: I have a gut feeling that the freeze is getting closer :)

This is a followup on my previous work on merging all the computer related LDAP objects in Debian Edu. As a step to try to see if it possible to merge the DNS and DHCP LDAP objects, I have had a look at how the packages pdns-backend-ldap and dhcp3-server-ldap in Debian use the LDAP server. The two implementations are quite different in how they use LDAP. To get this information, I started slapd with debugging enabled and dumped the debug output to a file to get the LDAP searches performed on a Debian Edu main-server. Here is a summary. powerdns Clues on how to set up PowerDNS to use a LDAP backend is available on the web. PowerDNS have two modes of operation using LDAP as its backend. One "strict" mode where the forward and reverse DNS lookups are done using the same LDAP objects, and a "tree" mode where the forward and reverse entries are in two different subtrees in LDAP with a structure based on the DNS names, as in tjener.intern and 2.2.0.10.in-addr.arpa. In tree mode, the server is set up to use a LDAP subtree as its base, and uses a "base" scoped search for the DNS name by adding "dc=tjener,dc=intern," to the base with a filter for "(associateddomain=tjener.intern)" for the forward entry and "dc=2,dc=2,dc=0,dc=10,dc=in-addr,dc=arpa," with a filter for "(associateddomain=2.2.0.10.in-addr.arpa)" for the reverse entry. For forward entries, it is looking for attributes named dnsttl, arecord, nsrecord, cnamerecord, soarecord, ptrrecord, hinforecord, mxrecord, txtrecord, rprecord, afsdbrecord, keyrecord, aaaarecord, locrecord, srvrecord, naptrrecord, kxrecord, certrecord, dsrecord, sshfprecord, ipseckeyrecord, rrsigrecord, nsecrecord, dnskeyrecord, dhcidrecord, spfrecord and modifytimestamp. For reverse entries it is looking for the attributes dnsttl, arecord, nsrecord, cnamerecord, soarecord, ptrrecord, hinforecord, mxrecord, txtrecord, rprecord, aaaarecord, locrecord, srvrecord, naptrrecord and modifytimestamp. The equivalent ldapsearch commands could look like this:

ldapsearch -h ldap \
  -b dc=tjener,dc=intern,ou=hosts,dc=skole,dc=skolelinux,dc=no \
  -s base -x '(associateddomain=tjener.intern)' dNSTTL aRecord nSRecord \
  cNAMERecord sOARecord pTRRecord hInfoRecord mXRecord tXTRecord \
  rPRecord aFSDBRecord KeyRecord aAAARecord lOCRecord sRVRecord \
  nAPTRRecord kXRecord certRecord dSRecord sSHFPRecord iPSecKeyRecord \
  rRSIGRecord nSECRecord dNSKeyRecord dHCIDRecord sPFRecord modifyTimestamp
ldapsearch -h ldap \
  -b dc=2,dc=2,dc=0,dc=10,dc=in-addr,dc=arpa,ou=hosts,dc=skole,dc=skolelinux,dc=no \
  -s base -x '(associateddomain=2.2.0.10.in-addr.arpa)'
  dnsttl, arecord, nsrecord, cnamerecord soarecord ptrrecord \
  hinforecord mxrecord txtrecord rprecord aaaarecord locrecord \
  srvrecord naptrrecord modifytimestamp

In Debian Edu/Lenny, the PowerDNS tree mode is used with ou=hosts,dc=skole,dc=skolelinux,dc=no as the base, and these are two example LDAP objects used there. In addition to these objects, the parent objects all th way up to ou=hosts,dc=skole,dc=skolelinux,dc=no also exist.

dn: dc=tjener,dc=intern,ou=hosts,dc=skole,dc=skolelinux,dc=no
objectclass: top
objectclass: dnsdomain
objectclass: domainrelatedobject
dc: tjener
arecord: 10.0.2.2
associateddomain: tjener.intern
dn: dc=2,dc=2,dc=0,dc=10,dc=in-addr,dc=arpa,ou=hosts,dc=skole,dc=skolelinux,dc=no
objectclass: top
objectclass: dnsdomain2
objectclass: domainrelatedobject
dc: 2
ptrrecord: tjener.intern
associateddomain: 2.2.0.10.in-addr.arpa

In strict mode, the server behaves differently. When looking for forward DNS entries, it is doing a "subtree" scoped search with the same base as in the tree mode for a object with filter "(associateddomain=tjener.intern)" and requests the attributes dnsttl, arecord, nsrecord, cnamerecord, soarecord, ptrrecord, hinforecord, mxrecord, txtrecord, rprecord, aaaarecord, locrecord, srvrecord, naptrrecord and modifytimestamp. For reverse entires it also do a subtree scoped search but this time the filter is "(arecord=10.0.2.2)" and the requested attributes are associateddomain, dnsttl and modifytimestamp. In short, in strict mode the objects with ptrrecord go away, and the arecord attribute in the forward object is used instead. The forward and reverse searches can be simulated using ldapsearch like this:

ldapsearch -h ldap -b ou=hosts,dc=skole,dc=skolelinux,dc=no -s sub -x \
  '(associateddomain=tjener.intern)' dNSTTL aRecord nSRecord \
  cNAMERecord sOARecord pTRRecord hInfoRecord mXRecord tXTRecord \
  rPRecord aFSDBRecord KeyRecord aAAARecord lOCRecord sRVRecord \
  nAPTRRecord kXRecord certRecord dSRecord sSHFPRecord iPSecKeyRecord \
  rRSIGRecord nSECRecord dNSKeyRecord dHCIDRecord sPFRecord modifyTimestamp
ldapsearch -h ldap -b ou=hosts,dc=skole,dc=skolelinux,dc=no -s sub -x \
  '(arecord=10.0.2.2)' associateddomain dnsttl modifytimestamp

In addition to the forward and reverse searches , there is also a search for SOA records, which behave similar to the forward and reverse lookups. A thing to note with the PowerDNS behaviour is that it do not specify any objectclass names, and instead look for the attributes it need to generate a DNS reply. This make it able to work with any objectclass that provide the needed attributes. The attributes are normally provided in the cosine (RFC 1274) and dnsdomain2 schemas. The latter is used for reverse entries like ptrrecord and recent DNS additions like aaaarecord and srvrecord. In Debian Edu, we have created DNS objects using the object classes dcobject (for dc), dnsdomain or dnsdomain2 (structural, for the DNS attributes) and domainrelatedobject (for associatedDomain). The use of structural object classes make it impossible to combine these classes with the object classes used by DHCP. There are other schemas that could be used too, for example the dnszone structural object class used by Gosa and bind-sdb for the DNS attributes combined with the domainrelatedobject object class, but in this case some unused attributes would have to be included as well (zonename and relativedomainname). My proposal for Debian Edu would be to switch PowerDNS to strict mode and not use any of the existing objectclasses (dnsdomain, dnsdomain2 and dnszone) when one want to combine the DNS information with DHCP information, and instead create a auxiliary object class defined something like this (using the attributes defined for dnsdomain and dnsdomain2 or dnszone):

objectclass ( some-oid NAME 'dnsDomainAux'
    SUP top
    AUXILIARY
    MAY ( ARecord $ MDRecord $ MXRecord $ NSRecord $ SOARecord $ CNAMERecord $
          DNSTTL $ DNSClass $ PTRRecord $ HINFORecord $ MINFORecord $
          TXTRecord $ SIGRecord $ KEYRecord $ AAAARecord $ LOCRecord $
          NXTRecord $ SRVRecord $ NAPTRRecord $ KXRecord $ CERTRecord $
          A6Record $ DNAMERecord
    ))

This will allow any object to become a DNS entry when combined with the domainrelatedobject object class, and allow any entity to include all the attributes PowerDNS wants. I've sent an email to the PowerDNS developers asking for their view on this schema and if they are interested in providing such schema with PowerDNS, and I hope my message will be accepted into their mailing list soon. ISC dhcp The DHCP server searches for specific objectclass and requests all the object attributes, and then uses the attributes it want. This make it harder to figure out exactly what attributes are used, but thanks to the working example in Debian Edu I can at least get an idea what is needed without having to read the source code. In the DHCP server configuration, the LDAP base to use and the search filter to use to locate the correct dhcpServer entity is stored. These are the relevant entries from /etc/dhcp3/dhcpd.conf:

ldap-base-dn "dc=skole,dc=skolelinux,dc=no";
ldap-dhcp-server-cn "dhcp";

The DHCP server uses this information to nest all the DHCP configuration it need. The cn "dhcp" is located using the given LDAP base and the filter "(&(objectClass=dhcpServer)(cn=dhcp))". The search result is this entry:

dn: cn=dhcp,dc=skole,dc=skolelinux,dc=no
cn: dhcp
objectClass: top
objectClass: dhcpServer
dhcpServiceDN: cn=DHCP Config,dc=skole,dc=skolelinux,dc=no

The content of the dhcpServiceDN attribute is next used to locate the subtree with DHCP configuration. The DHCP configuration subtree base is located using a base scope search with base "cn=DHCP Config,dc=skole,dc=skolelinux,dc=no" and filter "(&(objectClass=dhcpService)( (dhcpPrimaryDN=cn=dhcp,dc=skole,dc=skolelinux,dc=no)(dhcpSecondaryDN=cn=dhcp,dc=skole,dc=skolelinux,dc=no)))". The search result is this entry:

dn: cn=DHCP Config,dc=skole,dc=skolelinux,dc=no
cn: DHCP Config
objectClass: top
objectClass: dhcpService
objectClass: dhcpOptions
dhcpPrimaryDN: cn=dhcp, dc=skole,dc=skolelinux,dc=no
dhcpStatements: ddns-update-style none
dhcpStatements: authoritative
dhcpOption: smtp-server code 69 = array of ip-address
dhcpOption: www-server code 72 = array of ip-address
dhcpOption: wpad-url code 252 = text

Next, the entire subtree is processed, one level at the time. When all the DHCP configuration is loaded, it is ready to receive requests. The subtree in Debian Edu contain objects with object classes top/dhcpService/dhcpOptions, top/dhcpSharedNetwork/dhcpOptions, top/dhcpSubnet, top/dhcpGroup and top/dhcpHost. These provide options and information about netmasks, dynamic range etc. Leaving out the details here because it is not relevant for the focus of my investigation, which is to see if it is possible to merge dns and dhcp related computer objects. When a DHCP request come in, LDAP is searched for the MAC address of the client (00:00:00:00:00:00 in this example), using a subtree scoped search with "cn=DHCP Config,dc=skole,dc=skolelinux,dc=no" as the base and "(&(objectClass=dhcpHost)(dhcpHWAddress=ethernet 00:00:00:00:00:00))" as the filter. This is what a host object look like:

dn: cn=hostname,cn=group1,cn=THINCLIENTS,cn=DHCP Config,dc=skole,dc=skolelinux,dc=no
cn: hostname
objectClass: top
objectClass: dhcpHost
dhcpHWAddress: ethernet 00:00:00:00:00:00
dhcpStatements: fixed-address hostname

There is less flexiblity in the way LDAP searches are done here. The object classes need to have fixed names, and the configuration need to be stored in a fairly specific LDAP structure. On the positive side, the invidiual dhcpHost entires can be anywhere without the DN pointed to by the dhcpServer entries. The latter should make it possible to group all host entries in a subtree next to the configuration entries, and this subtree can also be shared with the DNS server if the schema proposed above is combined with the dhcpHost structural object class. Conclusion The PowerDNS implementation seem to be very flexible when it come to which LDAP schemas to use. While its "tree" mode is rigid when it come to the the LDAP structure, the "strict" mode is very flexible, allowing DNS objects to be stored anywhere under the base cn specified in the configuration. The DHCP implementation on the other hand is very inflexible, both regarding which LDAP schemas to use and which LDAP structure to use. I guess one could implement ones own schema, as long as the objectclasses and attributes have the names used, but this do not really help when the DHCP subtree need to have a fairly fixed structure. Based on the observed behaviour, I suspect a LDAP structure like this might work for Debian Edu:

ou=services
  cn=machine-info (dhcpService) - dhcpServiceDN points here
    cn=dhcp (dhcpServer)
    cn=dhcp-internal (dhcpSharedNetwork/dhcpOptions)
      cn=10.0.2.0 (dhcpSubnet)
        cn=group1 (dhcpGroup/dhcpOptions)
    cn=dhcp-thinclients (dhcpSharedNetwork/dhcpOptions)
      cn=192.168.0.0 (dhcpSubnet)
        cn=group1 (dhcpGroup/dhcpOptions)
    ou=machines - PowerDNS base points here
      cn=hostname (dhcpHost/domainrelatedobject/dnsDomainAux)

This is not tested yet. If the DHCP server require the dhcpHost entries to be in the dhcpGroup subtrees, the entries can be stored there instead of a common machines subtree, and the PowerDNS base would have to be moved one level up to the machine-info subtree. The combined object under the machines subtree would look something like this:

dn: dc=hostname,ou=machines,cn=machine-info,dc=skole,dc=skolelinux,dc=no
dc: hostname
objectClass: top
objectClass: dhcpHost
objectclass: domainrelatedobject
objectclass: dnsDomainAux
associateddomain: hostname.intern
arecord: 10.11.12.13
dhcpHWAddress: ethernet 00:00:00:00:00:00
dhcpStatements: fixed-address hostname.intern

One could even add the LTSP configuration associated with a given machine, as long as the required attributes are available in a auxiliary object class.

Things not to do if you want to be a good person:

• Complain privately to another person about vis public behavior
• Attempt to mediate between members of a group of which ve is also a member
• Defend anyone in a position of power for abusing that power
• Emotionally blackmail a group with a threatened resignation

One of my readers writes in that ve thinks that an egomaniac I may have mentioned previously would get upset about a perceived lack of proper attribution because ve views vis bit of code like vis offspring. This causes me to immediately have flashbacks of poor parenting in public. Many times I have seen adults pay no attention to their children who are misbehaving, perhaps smearing chocolate on a stranger's dress or gnawing on a stranger's leg, and if such a victim either attempts surrogate parenting by kicking ver in the teeth or petitioning the parent to do something about ver, said parent will become very angry and indignant that anyone would impugn vis skills or dedication. So when I see some coder not patch vis buggy code for over three years, yet still find enough time to be a sanctimonious prick and think that ve has the right to tell people how ve should be treated and how the code should be modified by others, I think there are better ways to view that metaphor. On the other hand, I don't know any parents who neglect their kid for 3 years and then still want to actively direct childrearing, unless perhaps they're paying a governess.

Today I improved the function I was using to toggle full screen in GNU Emacs.

(defun toggle-fullscreen (&optional f)
  (interactive)
  (let ((current-value (frame-parameter nil 'fullscreen)))
       (set-frame-parameter nil 'fullscreen
                            (if (equal 'fullboth current-value)
                                (if (boundp 'old-fullscreen) old-fullscreen nil)
                                (progn (setq old-fullscreen current-value)
                                       'fullboth)))))
(global-set-key [f11] 'toggle-fullscreen)

The previous function would always restore the frame to "normal" state, even if Emacs was previously maximized. This one makes it behave more like, say, Epiphany or Totem. Apparently using GNOME Keyboard Shortcuts to set this to F11 globally would override custom behaviour from some programs. I've not found any examples of this, though. This would be easily extensible to disable the Emacs toolbar when fullscreen, for instance.

I had a constructive meeting with my Member of Parliament yesterday, in which he agreed to push this critique of the Digital Economy Act (the dire nonsense that Lord Mandelson forced through in the dying seconds of the previous UK parliamentary session, apparently at the behest of the copyright exploitation industry, who will presumably now owe him a few lucrative consultancies)If you're wanting to do something similar, feel free to use the content, or use it for inspiration, or just use the one-page PDF version if that's helpful to you in explaining the worthlessness of this law.Note, I know there are many other points that could be made, but I was restricting myself to one side of A4 (just) -- MPs cannot be expected to read more it seems.

MeeGo (Nokia/Intel): Openness does not seem to be very important for Nokia and Intel. They develop their stuff behind closed doors and then do a large code drop (once dropped, stuff gets developed in the open). In terms of professionality, it does not look much better: If you look at the meego-dev mailing list, you feel like you are in some kind of a kindergarten for open source developers Things like HTML emails and top-posting appear to be completely normal for them, they don t even follow the basic rules for emails and they also appear to ignore any advice on this topic. Oh, and writing a platform in GTK+ while pushing Qt as the supported toolset is not a good idea. Android (Google): The situation with Android appears to be even worse. Google keeps an internal development tree and the only time the public sees changes is when a new release is coming and Google does a new code drop. Such a behavior is not acceptable. Ubuntu (Canonical): Canonical does an outstanding job on openness. Canonical employees develop their software completely in the open, and there are no big code drops. They basically have no competitive advantage and allow you to use their work for your own project before they use it themselves. Canonical employees basically behave like normal free software developers doing stuff in his free time, the only exception being that they are paid for it. Most Canonical employees also know how to behave on mailing lists and do not post HTML emails or do top-posting. There are of course also other people: The design process of the Ubuntu font on the other hand is a disaster, with the font being created using proprietary software running on Windows; for a free software company, that s not acceptable (such persons should probably not be hired at all). Furthermore, the quality of Ubuntu packages is often bad when compared to the quality of a Debian package (especially in terms of API documentation of Canonical-created software); a result of vastly different development cycles and different priorities. On a scale from 0 to 15 (15 being the best), Nokia/Intel receive 5 points ( D in U.S. grades), Google receives 3 points ( F in U.S. grades), and Canonical receives 12 points ( B in U.S. grades). Please note that those grades only apply for the named projects, the companies may behave better or worse in other projects. Final tips for the companies:

• Nokia/Intel: Teach your employees how to behave in email discussions.
• Nokia/Intel/Google: Don t do big code drops, develop in the open. If someone else can create something awesome using your work before you can, you are on the right way. Competitive Advantage must not exist.
• Canonical: Fire those who used Windows to create the Ubuntu font and restart using free tools.
• All: Document your code.

Filed under: General, Ubuntu

Since my last post about available LDAP tools in Debian, I was told about a LDAP GUI that is even better than luma. The java application jXplorer is claimed to be capable of moving LDAP objects and subtrees using drag-and-drop, and can authenticate using Kerberos. I have only tested the Kerberos authentication, but do not have a LDAP setup allowing me to rewrite LDAP with my test user yet. It is available in Debian testing and unstable at the moment. The only problem I have with it is how it handle errors. If something go wrong, its non-intuitive behaviour require me to go through some query work list and remove the failing query. Nothing big, but very annoying.

My project. This post assumes that the reader has read my last post. I m posting this weekly report earlier this week because there are too many things to tell already. I ve found the reason behind the segfault I m looking for so much time. The last problem, which is the only I know exactly when was fixed, because the programs started working, was related to allocating two times in the same region of memory. This happened because in each major GC the list of free line groups is generated again, but my old code was still allocating in the same line group of the last generation. So the last part of the line group, which was not yet used, would be a part of a line group in generated in the new collection, and it will be used for allocation two times: one in the allocation of the current line group, and another when this new line group starts being allocated. The implementation of the allocation of memory in lines is not very complicated, but it has some details that should be paid attention, and that were the cause of most trouble last weeks, and still need improvement. Initially I was allocating one object per line, just to see if it would work. As it didn t, I kept on improving the approach until I could find the problem. The next attempt was by setting ws->todo_free and ws->todo_lim in alloc_for_copy() in rts/sm/Evac.c. I think this is not ideal, because I didn t want the code to become too inconsistent with the way memory was allocated using these pointers before my changes. So I created new variables, line_free and line_lim, at first in the gen_workspace ws, the same place that todo_free and todo_lim are, but because of the last problem I described in the previous paragraph I changed it to generation. I m still not sure about where to place these pointers, this is something that can be improved. Another problem that I took a long time to understand is that the object need to be scavenged after being allocated. When it was allocated in todo_free, it was being scavenged by scavenge_block() in rts/sm/Scav.c, because the block in which it s in, todo_bd, is scavenged by this function. As I didn t wanted to the whole block where the free line group is to get scavenged again, I didn t want to send it to this function. So I thought about creating a way to scavenge only part of a block, that is, the space in the free line group that was allocated. This is still a valid idea, but I noticed that it was easier to use the mark stack. So I mark the object that is allocated in the line and push it in the mark stack. The main problem with this approach it s only possible to allocate in lines during major GCs, since only in this kind of GCs the mark stack is active. This is certainly the place where I can make more improvement. The patch of these changes and another one for the sanity checking explained in the last post. I m now benchmarking these changes with nofib, to see how much it affects the performance.

The X Strike Force needs you! (redux) This time, the focus is on the xserver-xorg-video-intel driver. KMS and UMS are going to be used heavily below, one may want to check the mode-setting wikipedia article and the modesetting page on X.org. Intel follows a 3-month cycle for its video driver releases. Some comments on the last ones:

• 2.9: What we ve had in Debian for some time already. Works mostly fine, supports both KMS and UMS. What we ve planned to release with for some time already.
• 2.10: First version with ripped-off UMS support. Not a significant improvement.
• 2.11: Apparently (still) having a bunch of regressions from 2.9, which is why we didn t switch to it.
• 2.12: Many regressions are supposed to be fixed, meaning it could compete with 2.9, which is why we need you.

It d be nice to have users install 2.12 from experimental and see how it goes from here. If regressions are found, we should be able to try hard(er) to get them fixed, so that we can move 2.12 to unstable and release with it. If there are too many of them, and if they re not going to be fixed on time, we might stick with 2.9. Pulling the driver from experimental should be quite easy: libdrm* packages are going to be upgraded, but that s mostly it (meaning the server itself will stay the same, and rolling back should be as easy as downgrading the driver and the libdrm* packages). A note on KMS and UMS: a benefit of sticking with 2.9 was being able to switch back to UMS in case KMS wasn t usable for some cards. Unfortunately, popular/well-spread cards like i8xx aren t well supported yet, that s why Chris Wilson (upstream) is considering introducing UMS support again for that kind of blacklisted cards. This way, it should be possible to move to 2.12 (including Chris work) without leaving some users behind. Feedback appreciated as follows:

• If 2.9 used to work and if 2.12 doesn t: please file a bug upstream.
• If 2.12 works while 2.9 didn t, reportbug -k or debian-x@.

Thanks already!

A new version 0.8.3 of Rcpp is now CRAN and in Debian. It comes about three weeks after the 0.8.2 release. And even though we promised to concentrate on documentation, it contains a raft of new features:

• The addition of what we dub Rcpp sugar: some syntactic sugar based on clever use of expression templates that lets us write C++ expression as neatly and compactly as vectorised R expressions (while getting C++ speed!!). More on that below.
• New classes Date and Datetime with internal representations just like R's Date and POSIXct classes, plus vector versions DateVector and DatetimeVector which behave just like STL vectors. With that, the new API is now feature-complete compared to the older 'classic' API.
• Rcpp Modules (our 'not unlike Boost::Python' feature introduced in version 0.8.1) can now expose public data members
• A new API class InternalFunction which can expose C++ functions even without R modules.

The main thing here is Rcpp sugar for which we also have a new (seventh !!) vignette Rcpp-sugar. As a quick example, consider this simple C++ function that takes two vectors from R and creates a new one conditional on the relative values:

export "C" SEXP foo( SEXP xs, SEXP ys)  
    Rcpp::NumericVector x(xs), y(ys);
    int n = x.size();
    Rcpp::NumericVector res( n );
    double xd = 0.0, yd = 0.0 ;
    for( int i=0; i<n; i++) 
        xd = x[i];
        yd = y[i];
        if( xd < yd ) 
            res[i] = xd * xd ;
          else  
            res[i] = -( yd * yd);
         
     
    return res ;
 

Now, if you use R, you really want to writes this more compactly. And now you can, thanks to Rcpp sugar:

extern "C" SEXP foo( SEXP xs, SEXP ys) 
    Rcpp::NumericVector x(xs), y(xs);
    return ifelse( x < y, x*x, -(y*y));
 

Same great taste, but much less filling! More details are in the Rcpp-sugar vignette. Doug Bates is already a fan of this and is employing it in the lme4a development version of the well-known lme4 package. The full NEWS entry for this release follows below:

0.8.3   2010-06-27
    o	This release adds Rcpp sugar which brings (a subset of) the R syntax
        into C++. This supports : 
         - binary operators : <,>,<=,>=,==,!= between R vectors
         - arithmetic operators: +,-,*,/ between compatible R vectors
         - several functions that are similar to the R function of the same name:
        abs, all, any, ceiling, diff, exp, ifelse, is_na, lapply, pmin, pmax, 
        pow, sapply, seq_along, seq_len, sign
        
        Simple examples :
        
          // two numeric vector of the same size
          NumericVector x ;
          NumericVector y ;
          NumericVector res = ifelse( x < y, x*x, -(y*y) ) ;
        
          // sapply'ing a C++ function
          double square( double x )  return x*x ;  
          NumericVector res = sapply( x, square ) ;
        
        Rcpp sugar uses the technique of expression templates, pioneered by the 
        Blitz++ library and used in many libraries (Boost::uBlas, Armadillo). 
        Expression templates allow lazy evaluation of expressions, which 
        coupled with inlining generates very efficient code, very closely 
        approaching the performance of hand written loop code, and often
        much more efficient than the equivalent (vectorized) R code.
        
        Rcpp sugar is curently limited to vectors, future releases will 
        include support for matrices with sugar functions such as outer, etc ...
        
        Rcpp sugar is documented in the Rcpp-sugar vignette, which contains
        implementation details.
    o   New helper function so that "Rcpp?something" brings up Rcpp help
    o   Rcpp Modules can now expose public data members
    o   New classes Date, Datetime, DateVector and DatetimeVector with proper
        'new' API integration such as as(), wrap(), iterators, ...
    o   The so-called classic API headers have been moved to a subdirectory
        classic/ This should not affect client-code as only Rcpp.h was ever
        included.
    o   RcppDate now has a constructor from SEXP as well
    o   RcppDateVector and RcppDatetimeVector get constructors from int
        and both const / non-const operator(int i) functions
        
    o   New API class Rcpp::InternalFunction that can expose C++ functions
    	to R without modules. The function is exposed as an S4 object of 
    	class C++Function

As always, even fuller details are in Rcpp Changelog page and the Rcpp page which also leads to the downloads, the browseable doxygen docs and zip files of doxygen output for the standard formats. A local directory has source and documentation too. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page

From time to time I see people arguing that the wide deployment of IPv6 will force DNSBLs to significantly change their operations and semantics. Usually they have little or no actual experience with IPv6 and the main argument used to justify this view appears to be more or less "OMG so many bits!1!!!". I have no reason to believe that this will really happen. People are already sending mail over IPv6 and I expect that the same reputation mechanisms used for IPv4 will be deployed as soon as they will be needed, with small policy changes to cope with the fact that end users typically get a whole network instead of a single IP address. So I expect that DNSBLs will continue to operate as usual like they currently do for IPv4 addresses, optionally by promoting "single IP" listings to "whole /64" listings when appropriate.

Let's briefly categorize the common types of DNSBL listings:

• spam sources or spam support (e.g. SBL): a human investigator decides that a specific server or network needs to be listed. Nothing will change.
• dynamic or dynamic-like (residential) networks (e.g. PBL): a human investigator decides that a specific network needs to be listed, or the ISP itself provides their relevant ranges. Nothing will change, except that dynamically-assigned networks will be much less common.
• fully automatic behaviour-driven listings (e.g. CBL): the IP addresses of spam sources are automatically listed as soon as they are detected. Are adjustments needed for this case? The only issue I can think about are the randomly-generated "privacy extensions" addresses used by some clients, but since they can be automatically detected the BL operator may also choose to just list the corresponding /64 from start.

If anything, with IPv6 much less guessing is needed to know how large each customer assignment is, so some things will actually be easier for operators. (I have been using IPv6 for at least ten years and I have received email over IPv6 networks for at least five.)

Debian Packages

Every now and again I look over server logs and see people downloading random .deb packages from my mirrors, or from my servers, via wget or Firefox (rather than with apt-get/aptitude). Personally I don't often download random binaries even from people I believe I can trust. Instead I'll download source and rebuild. But it bugs me that somebody might download a work-in-progress, decide it isn't complete or otherwise good enough, and miss out on an update of awesome-sauce a day or two later. I suspect there is no real solution to this "problem", and that including /etc/apt/sources.list.d/ entries inside a binary package to "force" an upgrade behind the scenes is a little too evil to tolerate. And yet .. something something dark-side .. something something seductive something?

Blog Update

This is my last film-subject entry. In the future I will have more accurate subjects, albeit more dull ones. I still amuse myself with quotations, and before that the song lyrics, but I guess that now is a good time to call it a day with that.

ObFilm: Cruel Intentions

I've grown tired of the firefox/iceweasel memory hog for a couple of reasons:

• it is such a memory hog and this isn't a machine particularly short of RAM but I still notice it when compiling with a few FF windows open and FF is just generally slow.
• the upgrade behaviour of iceweasel is getting tedious, begging for a restart every time I do apt-get upgrade recently.

Chromium is just faster, everywhere. A nice touch is the new tab behaviour too - previews of the most recently viewed pages is far more useful than the usual homepage. Chromium is also faster than epiphany/webkit.

I'll still be using two different browsers, because neither iceweasel nor chromium can do the clever smart bookmarks thing of epiphany. I have come to utterly rely on bookmark input boxes for direct access to specific Debian bug numbers, individual PTS pages, Google searches, dwww searches, specific manpages, buildd reports and various other pages where a bookmark really needs to be an input box, not a label, and the bookmark itself sits on a toolbar and can then use http://url/%s which makes life so much easier.
Update
I wondered if epiphany smart bookmarks would be understood but apparently not. So:

The smart bookmarks are BTS, Google, PTS, buildd and man. At work, I've got a wider screen and I add several more for internal bug tracker numbers and similar.
Chromium and iceweasel/FF have nothing like this. Entering a bug number is utterly trivial - it even works with middle mouse button paste which, cleverly, opens the page in a new tab too. Iceweasel/FF can do this ONCE but only once, via an extension. The key points with epiphany are:

• It is trivial to setup - just make a bookmark with a %s and put that bookmark onto a toolbar - any toolbar.


• it is trivial to use and never needs any use of the address bar or editing anything


• It works for any query which can have a single argument at any point in the URL.


• You can have as many as you want, subject to how many toolbars you can use

So, yes, Leo, the way epiphany does this is massively more effective for a "work-type" browser situation where there is such a need to access a specific page immediately, with no editing or keyboard usage.

Hey there gentle readers, I am completing paperwork on behalf of Collier Technologies LLC which is required (as described in chapter 19.34 RCW [pdf]) to file with the state of Washington to become a licensed Certification Authority. CPS (signed) Look! I m also running a(n unrecognized) repository! Neat, huh? C.J.

The GNOME Foundation released their conference speaker guidelines today. This is an important step not just in helping speakers know what's acceptable, but also in helping audience members understand in advance what the community is likely to find objectionable and ensure that they can feel comfortable in raising concerns.

Of course, guidelines mean little without enforcement. My original draft of these suggested that event runners be able to stop presentations if they felt they were gratuitously in breach of the guidelines. Opinions on this were fairly strongly split, with several people concerned that this effectively allowed individuals to immediately shut down presentations with little oversight. That's a genuine concern, but it does seem to assume bad faith on the part of conference organisers in a way we've rarely (never?) seen. On the other hand, conferences in our field have endured presentations that have contained offensive material from start to finish. If an offended individual is in a minority then it's not easy for them to potentially challenge the audience by vocally expressing their unhappiness, and even standing up and leaving may be a difficult and obvious act.

But I don't set the behavioural standards of the community, and attempting to enforce standards that people don't agree with isn't going to fly. Some people are likely to feel that even the level of enforcement suggested is an unwelcome intrusion into free discussion of some topics, so I think this is a good compromise that is a great signal for our unwillingness to accept inappropriate presentations. With luck we'll see other communities enact similar guidelines and we can come to a broad consensus that covers the majority of our conferences.