We may not have did up my house like this OSJ one but you did a fine job > thanks (and thanks for laughing with me over this odd dog and his portrait!)

When we first moved to the US, I wanted to try and do the telecoms and Internet stuff as cheaply as possible, while still trying to be technologically "fun" as well. I elected to get the cheapest home phone line (local calls only) I could get with AT&T (then SBC) (which is still remarkably expensive) so that I could get DSL (which, while three times faster than what I'd had previously in Australia, was still not that fast). (I went for DSL over cable because I'd heard horror stories about Comcast's reliability, and also they weren't cool with running servers or inbound services, Sonic is, and is a great ISP). I chose Vonage for US long-distance, I think because I'd heard of them previously, and was interested in trying out VoIP. We initially used Vonage for calling Australia as well, until I played around with Engin, and then deployed Asterisk. Anyway, I've been a loyal Vonage customer for exactly three years. The truth is, we hardly make any US long distance calls. We've got friends in Phoenix (hi Craig and Sarah!) that we call infrequently. I guess there might be the odd cell phone that isn't considered a local call, but we tend to use our cell phones more than the home phone anyway... So we were a very cheap $14.99 a month for Vonage. So when I called T-Mobile to purchase a SIM card for the Android Dev Phone 1 I'd been recently given, the opportunity to have their T-Mobile @Home in place of Vonage, for an extra $9.99 on top of what I was paying for the cell phone line seemed like worthwhile cost saving. I have no attachment to the incoming number for the Vonage line, we give out the AT&T land line number to people as our "primary" home number. Later that day, Vonage announced they were increasing their monthly fee from $14.99 to $17.99, which made the decision seem all the more prodigious. I did a spot of research while I was waiting for the @Home box to ship to me, and it turns out that they have a wireless router and non-wireless router option. They hadn't specified what I was getting at order time, and I was hoping I'd get the non-wireless router, since I already had an access point I was perfectly happy with it. As luck would have it, I received the wireless router model (a Linksys WRTU54G-TM). I'd mentally prepared myself to be receiving just another Linksys RTP300, like what Vonage used, and so I was quite surprised by the differences in technology used. It's been a while since I tried to do any reverse-engineering of the Vonage ATA, but I think from memory, it was essentially that: an ATA. It talked SIP back to Vonage. My DSL provider gave me a few static IP addresses, so when I got it, I just allocated one for Vonage, and did a static NAT through my firewall to for it, and everything just worked. It was fairly apparent that the WRTU54G-TM was nothing at all like this (given it was also an access point). It takes a GSM SIM card (up to two in fact), which I presume has all of the provisioning information on it. I was initially worried that I was going to need to completely rejig my network to accommodate it. I guess the SIM card approach means there's no customer specific provisioning that presumably otherwise needs to be done to the device itself, and they can just ship vanilla devices. I presume the Vonage RTP300 had some sort of customer-specific configuration, because I never had to do anything. I briefly toyed with the idea of running two access points, and having a "guest" wireless network, but I am interested in reducing heat and power consumption in my linen cupboard, so I decided to try and just swap over to using the WRTU54G-TM as my access point, retiring my Linksys WAP54G. So that's exactly what I sat down to do last night. Sometimes in their efforts to make these things "simple", they end up making things more complicated. What I wanted to do was just plug the "Internet" side of the device into my internal network, have it get an IP address with DHCP, and then let me get on with configuring it. The "advanced" setup instructions talked about the box having a default IP address of 192.168.0.1 ("or try 192.168.24.1 if that doesn't work"). I figured the DHCP IP address would override that, so I was trying to hit the IP address I could see the unit had picked up from my DHCP server. No response from the HTTP server. I could see it replying to ARP requests, and I could see what appeared to be an IPsec NAT-T connection. After some fiddling around, I twigged that I was trying to manage it over the Internet port, and any device worth its salt was going to be very hardened out of the box on what it expected to be the raw Internet-facing side, so I plugged my laptop directly into one of the four Ethernet ports that it also had, and then I could hit the 192.168.0.1 IP address and access the management interface. It never occurred to me prior to fighting with getting the WRTU54G-TM configured, that the RTP300 might have been more than a black box, as I'd never tried plugging anything into the "inside" ports on it, mainly because I'd never intended to use it as a router or a switch. Out of the box, the WRTU54G-TM was configured to be a router, which I didn't need, and it was configured to be a DHCP server for 192.168.0.0/24, which I definitely didn't want. An interesting side-effect of putting it into bridging mode was the MAC address changed (to the one that was printed on the unit, which was not how out of the box it presented itself). I have no need or intention of plugging anything into the "inside" Ethernet ports on the WRTU54G-TM, as I have a separate switch for that, and indeed I run my wireless network on a different interface of my firewall to my wired network. Fortunately I was able to enable the management interface via the "Internet" interface. The added benefit of the device doing NAT-T is I don't need to allocate an IP address to it for the VoIP stuff it work, it all just works fine without it, so I've freed up an IP address, removed two devices from my linen cupboard, added one new one, and saved some money. Woot! Next, just because I can, I'm going to try and break into the RTP300 and see if I can reconfigure it as a general purpose ATA to talk to Asterisk. There's evidence it's possible. It's otherwise of no further use to me. Even if I get it working with Asterisk, it's not of a lot of use to me. Maybe I can send it to a family member, and they can call us over the Internet for free instead of for the cost of a call to Brisbane...

While searching for different options to share files among different computers in a local LAN I found via Planet Ubuntu giver a little util which uses avahi/zeroconf to publish shared folders or files. It is a easy tool, that works and that serve well its purpose. But its development has stopped some time ago, and that means that nobody has cared to port it to Windows, where it would be an interesting tool to avoid the need of using windows network to share folders, which in an ad-hoc network tends to work problematically. These little gems should receive more care from freedesktop.org, GNOME Foundation, Canonical, Novell, as are the utils that make a difference in desktop expecience. Having a nice visual effects is not enough to attract people to free destops, but having tools that make everyday life is.

For a while I have been pondering over a problem that arises when your functionally written program has some state with cross references for example a list of users, each of which uses a number of computers, and a list of computers, each having an owner. Implicit referencing For doing queries on such data, it would be convenient if every reference is just the referenced object itself. Although we would visualize this as a graph, semantically, it is more like an infinite tree. This is possible in Haskell, due to laziness, and if you create the data structure cleverly, it even uses constant memory, no matter how deep you enter this infinite tree (a recent post of mine talks about this). A possible data definition would be:

data User = User  
	userName :: String,
	uses :: [Computer]
	 
data Computer = Computer  
	computerName :: String,
	owner :: User -- references the Users
	 
data State = State [User] [Computer]

testState = let user = User "Conrad" [computer]
                computer = Computer "Z3" user 
            in State [user] [computer]

Explicit referencing But such a representation is very unsuitable for updates (at least I can t think if a nice way of updating such a graph without breaking the internal cross-references) and serialization, which is a must for a HAppS based application. So what one would probably start with is this data structure:

data User = User  
	userId :: Int,
	userName :: String,
	uses :: [Int] -- references the Computers
	 
data Computer = Computer  
	computerId :: Int,
	computerName :: String,
	owner :: Int -- references the Users
	 
data State = State [User] [Computer]

testState = State
		[User 0 "Conrad" [1]]
		[Computer 1 "Z3" 0]

I think the semantics of this are clear. Note that the referencing is currently not type-safe, but this can be provided by phantom types. Maybe I ll write more about that later. Now imaging you want to display the information about the first computer with your web application. You extract the information with let State _ cs = testState in head cs and pass that to your templating engine. But what if your template wants to display the name of the owner? It only has access to his userId. You would either need to know what information the template will ever need, extract that from the state beforehand and pass it along, or give the template access to the whole state. In that case, though, there has to be lookup-logic in your output code, which is also not nice. Woudln t it be nice if you could, in your application logic, work with the explicit references, which are easy to modify and store, but somehow turn that into the implicit referencing? Duplicated representation One way would be to have two unrelated sets of data structures, ExplicitState, ExplicitUser, ExplicitComputer, which use explicit identifiers to reference each other, and ImplicitState,... which are defined as the first representation of our state. It is then mostly trivial to write a function that converts ExplicitState to ImplicitState. The big disadvantage of this is that you have to maintain these two different hierarchies. It also means that every function on the state has to be defined twice, which often almost identical code. Clearly, this is not desirable. Annotated representation It would be more elegant if the state is stored in one data type that, controlled by a type parameter, comes in the one or the other representation. To do that, we need two types: One that contains a value, and one that contains just a reference:

newtype Id v = Id v deriving (Show, Typeable, Data)
newtype Ref v = Ref Int deriving (Show, Typeable, Data)

Then we need to adjust our data definitions, to make use of these. (I ll leave out the names, to keep the code smaller)

data User ref = User  
	userId :: Int,
	uses :: [ref (Computer ref)]
	 
data Computer ref = Computer  
	computerId :: Int,
	owner :: ref (User ref)
	 
data State ref = State [User ref] [Computer ref]

Here we introduce a type parameter ref , which will later be either Id or Ref. Note that now a reference also states the object it is a reference for, which greatly increases type safety. Functions on these data types that don t work with the references will be polymorphic in the ref type parameter, so only need to be written once. A User Id is a complete user with all related data, while User Ref is a user with only references. And a Ref (User Ref) is reference to a user, which contains references... Not so kind kinds Did you notice the lack of a deriving clause? Our data structures have the relatively peculiar kind ((* -> *) -> *), which makes it hard for the compiler to derive instances for things like Show. But we already know that we will only use Id or Ref for the type variable, so we can use ghc s StandaloneDeriving language extension and have these instances created:

deriving instance Show (User Id)
deriving instance Show (User Ref)
deriving instance Show (Computer Id)
deriving instance Show (Computer Ref)
deriving instance Show (State Id)
deriving instance Show (State Ref)

Toggling a type parameter The next step is to write the conversion function. It will have type

unrefState :: State Ref -> State Id

For that, and for later, we need lookup functions:

unrefUserRef :: State Id -> Ref (User Ref) -> Id (User Id)
unrefUserRef (State l _) (Ref i) = Id $ fromJust $
	find (\u@(User i' _) -> i == i') l 
unrefComputerRef :: State Id -> Ref (Computer Ref) -> Id (Computer Id)
unrefComputerRef (State _ l) (Ref i) = Id $ fromJust $
	find (\u@(Computer i' _) -> i == i') l 

These expect a State (with implicit referencing) and a reference, and look up this reference. The function unrefState then looks like this:

unrefState :: State Ref -> State Id
unrefState (State us cs) =
	let unrefState = State (map (unrefUser unrefState) us)
	                       (map (unrefComp unrefState) cs)
	in unrefState
  where unrefUser :: State Id -> User Ref -> User Id
        unrefUser s (User i refs) = User i (map (unrefComputerRef s) refs)

	unrefComp :: State Id -> Computer Ref -> Computer Id
	unrefComp s (Computer i ref) = Computer i (unrefUserRef s ref)

Note how we tie the knot in the let expression. This is the trick that ensures constant memory consumption, because every reference points back to the same place in memory. Satisfied already? So what do we have? We have no duplication of data types, we can write general functions, and we can resolve the explicit referencing. We can also easily write functions like unrefUser :: State Ref -> User Ref -> User Id, which transform just a part of the state. But writing unrefState is very tedious when the State becomes more complex. Each of the other unrefSomething functions are very similar, but need to be written anyways. This is unsatisfactory. What we want, is a generic function, something like

gunref :: State Ref -> a Ref -> a Id

which, given a state with explicit references, replaces all explicit references in the first argument (which could be State Ref or User Ref or anything like that) with implicit ones. This function can not exist, because we would not know anything about a and b. But maybe we can do this:

gunref :: (Data (a Id), Data (a Ref)) =>  State Ref -> a Ref -> a Id

Typeable and Data Before being able to do so, we need to derive Data for our types. We can start with

deriving instance Data (User Id)
deriving instance Data (User Ref)
deriving instance Data (Computer Id)
deriving instance Data (Computer Ref)
deriving instance Data (State Id)
deriving instance Data (State Ref

but that will complain about missing Typeable instances. Unfortunately, ghc s deriver for Typeable (even the stand-alone-one), does not handle our peculiar kind, so we need to do it by hand. With some help from quicksilver on #haskell, I got it to work:

instance Typeable1 ref => Typeable (User ref) where
	typeOf _ = mkTyConApp (mkTyCon "User")     [typeOf1 (undefined :: ref ())]
instance Typeable1 ref => Typeable (Computer ref) where 
	typeOf _ = mkTyConApp (mkTyCon "Computer") [typeOf1 (undefined :: ref ())]
instance Typeable1 ref => Typeable (State ref) where 
	typeOf _ = mkTyConApp (mkTyCon "State")    [typeOf1 (undefined :: ref ())]

everywhere is not enough Turning to the documentation of Data.Generics, I notice with some disappointment that there is no function that is able to change a type they all seem to replace a value by another value of the same type. But the functions gfoldl and gunfold sounded like they could be used for this. Warning: What comes now is a very non-haskellish hack that subverts the type system, just to get the job done. Please read it with a healthy portion of distrust. If you know of a cleaner way of doing that, please tell me! Wrapped Data I want to do some heavy type hackery, so I need to disable haskell s type system. There is Data.Dynamic, but not even that is enough, as we need to carry a type s Data instance around as well. So let s wrap that up:

data AData where AData :: Data a => a -> AData
instance Show AData where show (AData a) = "<" ++ show (typeOf a) ++ ">"

fromADataE :: forall b. Data b => AData -> b
fromADataE (AData d) = case cast d of
	Just v -> v
	Nothing -> error $ "Type error, trying to convert " ++
			   show (typeOf d) ++ " to  " ++
			   show (typeOf (undefined :: b)) ++ "."

There is also a function that converts an AData back to a normal type, if possible. If it s not possible, then there is a bug in our code, so we give an informative error message. AData transformers Similar to everywhere, we want to have transformers that combinable. They need to have the change to convert an arbitrary value, but also signal that they could not convert something (and this something has to be recursed into). I came up with this:

type ADataT = AData -> Maybe AData

extADT :: forall a b. (Data a, Data b) => ADataT -> (a -> b) -> ADataT
extADT at t a@(AData v) = case cast v of 
		Just x ->  Just (AData (t x))
		Nothing -> at a

doNothing :: ADataT
doNothing = const Nothing

ADataT is the type for such a transformer. doNothing will not transform anything, and extADT can be used to add any function to the list of tried transformers, in the spirit of extT. The ugly part To apply such a transformer, I want to use this function, which I ll describe in the code comments:

everywhereADT :: forall a b. (Data a, Data b) => ADataT -> a -> b

-- first check if we can transform this value already
everywhereADT f v = case f (AData v) of
        -- if so, coerce it to the users  requested type, which hopefully goes well
	Just r ->  fromADataE r 
	-- if not, we need to recurse into the data structure
	Nothing -> recursed

	-- for that, we first need to figure out the arguments to the 
        -- constructor. We store them in the untyped list
  where args :: [AData]
        (Const args) = gfoldl k z v
	-- gfoldl lets us have a look at each argument. We wrap it in AData
        -- and append it to the list
	k (Const args) arg = Const (AData arg : args)
	z start = Const []

	-- We need the data constructor of the input data type. If the user did not want
        -- it to be transformed, it will be the same
	c = toConstr v

	-- To give better error messages, we make sure that the outmost type constructor
        -- of the requested type actually has the data constructor we were given. Otherwise
        -- gunfold will complain in a non-helpful way
	input_type = dataTypeRep (constrType c)
	output_type = dataTypeRep (dataTypeOf (undefined :: b))

	recursed = if input_type /= output_type
                   then error $ "Can not convert <" ++ show input_type ++ ">"++
                                            " to <" ++ show output_type ++ ">."
	-- the types match, so we assemble the output type, using gunfold
                   else snd (gunfold k' z' c)
	k' :: forall b r . Data b => ([AData], b -> r) -> ([AData],r)

	-- we start by reversing the input list
	z' start = (reverse args,start)
	
	-- then we call us recursively on the argument and feed the result
        -- to the output constructor
	k' ((AData a) : args, append) = (args, append (everywhereADT f a))

-- Used for folding (we don t need to retain the original constructor)
data Const a b = Const a 

What a beast! But surprisingly, it works. Here are some examples. Note that I always have to specify the requested output type:

bool2Int :: Bool -> Int
bool2Int False = 0
bool2Int True = 1

*Main> everywhereADT (doNothing  extADT  bool2Int) True :: Int
1
*Main> everywhereADT (doNothing  extADT  bool2Int) True :: ()
*** Exception: Type error, trying to convert Int to  ().
*Main> everywhereADT (doNothing  extADT  bool2Int) (True,False) :: (Int,Int)
(1,0)
*Main> everywhereADT (doNothing  extADT  bool2Int) ([True,False],True,()) :: ([Int],Int,())
([1,0],1,())
*Main> everywhereADT (doNothing  extADT  bool2Int) ([True,False],True,()) :: [()]
*** Exception: Can not convert  to .
*Main> everywhereADT (doNothing  extADT  bool2Int) [True] :: [Bool]
[*** Exception: Type error, trying to convert Int to  Bool.

I hope this code does not inflict too much pain on any Haskell-loving reader. I know I violated the language, but I didn t know how to do it better (at least not without using Template Haskell). I also know that this is not very good performance wide: Every single value in the input will be deconstructed, type-compared several times and re-assembled. If that is an issue, then this function should only be used for prototyping. Almost done To apply this to our state, we only need to glue it to our lookup functions from above:

gunref :: (Data (a Id), Data (a Ref)) =>  State Ref -> a Ref -> a Id
gunref s w = let unrefState = gunref' unrefState s
             in gunref' unrefState w

gunref' :: (Data (a Id), Data (a Ref)) =>  State Id -> a Ref -> a Id
gunref' unrefState = everywhereADT unref'
  where unref' = doNothing  extADT 
		 unrefUserRef unrefState  extADT 
		 unrefComputerRef unrefState

Now we have a generic unreferencer. I set the type a bit more specific than necessary, to make it safe to use (under the assumption that the list of lookup functions is complete and will not leave any Ref in the output).

*Main> testState 
State [User  userId = 0, uses = [Ref 1] ] [Computer  computerId = 1, owner = Ref 0 ]
*Main> gunref testState testState 
State [User  userId = 0, uses = [Id (Computer  computerId = 1, owner = Id (User  userId = 0,
uses = [Id (Computer  computerId = 1, owner = Id (User  userId = 0, uses = ..

Oh, and by the way, if you want to test this code, you ll need at least:

 -# LANGUAGE DeriveDataTypeable, FlexibleInstances, GADTs,
             FlexibleContexts, StandaloneDeriving, ScopedTypeVariables #- 

A few hours and this year will be over. Instead of making a summary of 2008, I feel the need to list a few of my goals for 2009:

• Finally complete the building of our house;
• Buy a piano and start again playing it, I really miss it;
• Buy a carom billiard, I really miss it;
  
• Run the Turin Marathon 2009 in spring (my first marathon ever);
  
• Run the Budapest Marathon 2009 in autumn (plus: in less than 3.5 hours);
• Launch ZeNEM.hu and OXA (two business projects of mine);
• Further develop our database marketing company in Hungary, and make it grow;
  
• Start practicing a martial art for self-defense;
• Start the training for my PPL (plus: complete it in 2009);
  
• Travel across Spain (and attend to the DebConf in Extremadura);
• Teach Greta and Sophia to swim;
• Deploy SATISFLY to the launch customer;
• Buy a new car for my wife;
  

This list is not exhaustive, but writing down these items helps me. After a so-so 2008, I'm looking forward for a new exciting year: I feel very motivated and I'm sure that 2009 will be positive and rich of satisfactions.

Happy New Year to everybody, may the Force be with you.

A teacher in Arizona steals Linux CDs from a student and then accuses a Linux distributor of being a criminal [1]. Even though she had used Linux in the past she didn t believe that software was free. Of course that implies that in the past she had performed actions that she believed were criminal. Neat Little Mac Apps interviews Marshall Kirk McKusick - he describes how the BSD Daemon logo was designed and one of his most significant bugs [2]. OurDelta.org offers MySQL builds with some extra features and support [3]. I was recommended to use their builds by Arjen Lentz of Open Query [4], as one of my clients is going to use the services of Open Query it seems best to use the Our Delta builds if only to get better support from Open Query. The extra features in the Our Delta builds seem interesting, but I m not sure that my client needs any of them at this time. The Global Guerilla blog reports on a man who single-handedly invaded the most heavily guarded power station in Britain and shut it down to protest against new coal power stations [5]. The entire blog is worth reading, the author has a lot of interesting ideas. PhpMyVisites is a free web site analytics system that competes with Google Analytics [6]. I haven t implemented it yet, but it looks promising. It seems that PhpMyVisites is not being updated any more (not even security updates) and the replacement is Piwik [11]. Andrew Lahde was a fund manager who made significant amounts of money by betting on the inability of US mortgagees to repay their debts, he wrote an interesting goodbye letter (Telegraph.co.uk) [7]. He now has a Wikipedia page which gives some interesting background to his career [8]. An Employee of the Financial Times is famous for flaming Andrew [9], I have submitted a comment pointing out that being famous for flaming someone who is more successful than yourself is nothing to be proud of and suggesting that he advocate his own political views when criticising those of others - I doubt that it will get through moderation. It s a pity that Andrew doesn t have a blog, I would like to read more from him. At CCC a paper by Alexander Sotirov, Marc Stevens, Jacob Appelbaum, Arjen Lenstra, David Molnar, Dag Arne Osvik, and Benne de Weger on how to crack the PKI infrastructure used for SSL signing was presented [10]. The root cause is some CAs still using MD5 even though it was broken a long time ago. Updated to note that Piwik is the replacement for PhpMyVisites.

• [1] http://linuxlock.blogspot.com/2008/12/linux-stop-holding-our-kids-back.html


• [2] http://www.neatlittlemacapps.com/show/nlma-episode-048-interview-marshall-kirk-mckusick


• [3] http://ourdelta.org/


• [4] http://openquery.com.au/


• [5] http://globalguerrillas.typepad.com/globalguerrillas/2008/12/journal-climate.html


• [6] http://www.phpmyvisites.us/


• [7] http://www.telegraph.co.uk/finance/financetopics/financialcrisis/3219870/The-astonishing-sign-off-of-a-wealthy-fund-manager-in-full.html


• [8] http://en.wikipedia.org/wiki/Andrew_Lahde


• [9] http://blogs.ft.com/gapperblog/2008/10/yours-truly-angry-mob/


• [10] http://www.phreedom.org/research/rogue-ca/


• [11] http://piwik.org/

Last week, I spent some time building a good (that I liked) feed auto-discovery tool to use in Ruby for other project I m building, rFeed. I liked CPAN s Feed::Find, and at some point I made a wrapper class to run a Perl script using such module, however, I wasn t happy by mixing it all. So, Feedbag was born:

>> require "rubygems"
=> true
>> require "feedbag"
=> true
>> Feedbag.find "log.damog.net"
=> ["http://feeds.feedburner.com/TeoremaDelCerdoInfinito",
 "http://log.damog.net/comments/feed/"]
>> planet_feeds = Feedbag.find("planet.debian.org")
[ ... ]
>> planet_feeds.first(3)
=> ["http://planet.debian.org/rss10.xml",
 "http://planet.debian.org/rss20.xml",
 "http://planet.debian.org/atom.xml"]
>> planet_feeds.size
=> 104
>>

It makes smart use of relative and absolute bases, hrefs, links, content types, etc. It is also a single Ruby file, so you can grab it and use it on your application. Plus, it only requires Hpricot as dependency. It can find all feeds linked on a web page, but it will return the most important at the beginning of the resulting array, so you will have the important one on the first results (see example above with Planet Debian). Synopsis, README and a brief tutorial have been placed at axiombox.com/feedbag. You can also take a look at the git repo, hosted in GitHub.

Recently the Debian Administration website has been besieged by spammers submitting bogus comments. I also run a couple of blogs which are constantly having random gibberish submitted to them - although it is less of a problem there because I approve them manually due to the offline nature of my blog, and the blog compiler I use. Regardless I figured it was time to do something about it, in part because Chris Searle suggested that I should be able to query arbitrary text for spammyness due to my experience with fighting email spam. (As it turns out that is not so useful experience. Comments and emails are not the same. The same header-trickery you see in bogus email, false EHLOs, etc, are not useful concepts to carry accross.) Anyway we were saying, I was going to do something? So I did. My solution? An XML-RPC server which you submit your comments to. This will return either:

OK	The comment is clean.
SPAM:[reason]	The comment is spam, and the optional reason explains further

The server itself is very simple, and it comes with a collection of simple plugins which do the job of testing different things. More plugins will almost certainly be added over time - so far I've seen the lotsaurls plugin capturing most of the SPAM, although the DNSRBL lookup is also working nicely having captured live spam without my touching it. Huzzah. & etc.. If there is any interest feel free to let me know, although as the server is open, and the client side usage is documented I imagine other people could run with it easily enough too - avoiding me becoming a single point of failure. For reference my initial change to hook this up live on the site was only a few lines of code - although later I did need to make some additional changes to make it cope with failures, & etc. ObFilm: Ghostbusters

So the failings of the GR result in just what everyone didn't need - a restarted discussion about the firmware issue without first releasing Lenny. Come on people, leave it alone and get Lenny done. Anthony, Thomas and Theodore (also in his blog) all seem to be missing the point(s):

• We had the GR and despite all the manifold failings of the GR itself, Further Discussion FAILED as did the option to delay Lenny ad infinitum (Option 1 on the vote).


• Everyone seems hung up on firmware as something that requires source code, when the real point is that firmware isn't necessarily like other content in Debian and in some (many?) situations, there is no, nor can be any, human-readable form - it really can be nothing more than a series of initialisation flags, not instructions that are specific to that chip or would not make any sense to any CPU.

To me, it's more like data for the chips - it's saying "this is who you are, this is what you do and this is how to understand all subsequent instructions". Software is what follows - the instructions themselves.

We have a very good understanding of the separation between software and data in userspace - we all recognise that unless the output of a program consists to a large extent of modified versions of already licensed code (e.g. code generators), then the data created by your free software GPL-3+ text editor (or blogging client) cannot be deemed to be under the GPL-3+ or any other licence until such a statement is made by the copyright holder and that the copyright holder is the user of the software, not the author of the software. I could quite easily put this blog under a Microsoft-style EULA or even NDA if I wanted to be obtuse, despite it being created using a free software stack. The point remains - I am the copyright holder of the content created using the free software on my machines and I am free to publish that data as I see fit. The authors of the software itself have no rights to dictate what I do with the content I create using the software, just as I - as a user - cannot dictate whether the authors of my editor choose to migrate from GPL-2+ to GPL-3+ or even BSD.

Where firmware can have source code, let it have source code - we have such packages in Debian already. Even then, much of the code may be simply calculations or conditionals to select the right combination of binary flags for specific situations. This isn't about Nvidia kernel drivers that do real runtime calculations during the entire runtime of the machine, this is about tiny bits of initialisation data that are set once and not touched again. Can we not accept that some firmware is derived from a process that does not involve source code? How about experimentation and calculation? We know the voltage on pins 5 and 19, we know the state of register A and that for this specific case, it needs to be toggled to match the state of register E because otherwise another chip on the same board gets confused, so setup the chip to do that by setting a firmware datum point to initialise A? Where's the source code in that process? There is none - it is a piece of data, an item in the design notes (or more likely the errata). Yes, maybe the chip should know how to set this datum itself but for whatever reason (probably financial), it doesn't.

Source code could be used to calculate the datum point for a range of chips and chipsets according to a range of data inputs but the creative effort is then in the calculation not the data. The chip doesn't want the calculation, it cannot perform a calculation in this uninitialised state, it needs the datum point. Once it has the registers set, it can then understand subsequent instructions - the software itself. The datum point itself is just a raw piece of initialisation junk for the chip, it has no source code, it is determined by a physical process. It is a reference point - a fixed value that is determined during the design of the chip and/or chipset.

It's like saying you want the source code that determines whether there are one hundred cents in a Euro so that you could modify it to create a thousand or just ten or maybe turn the Euro into a hexadecimal currency. Weird. The fact that the Euro has one hundred sub-divisions is not up for negotiation, it isn't a modifiable value - changing the value unilaterally is not going to magically make everyone use the new value. It is a fixed reference point - a single datum point that initialises all subsequent behaviour of all software utilising such a currency. Even if there was source code (gee, what could that be, do you think? "%.2i" ?) it would be meaningless to insist on being able to modify it.

Few people can make the judgement about whether particular firmware can or cannot have source code - in most cases it comes down to the hardware design of either just the chip or how the chip is utilised for a particular board.

At some point, Debian has to accept that we cannot know if source code was used to create a particular blob. We simply have to ensure that maintainers ask upstream, elucidate as much information as possible about how the firmware was created and document that in debian/copyright. If that information is proved to be incorrect, then we have to deal with it as best we can - on a package-by-package basis. The assumption has to be that firmware does not necessarily have any source code.

In the end, I think the GR came up with the right result, despite all the failings - Option 5, assume that firmware is compliant with the GPL unless there is evidence to the contrary.

Sometimes, there simply is no source code - sometimes all you have is data.

Debian cannot insist on source code for data that is not generated from source code and it is the difficulty in deciding whether one set of figures is initialisation data or a set of compiled instructions that will defeat all debate on a general solution for the firmware debate. It is comparing apples and oranges - we can insist on source code where source code was used but we also have to accept that some firmware is identified, not programmed. In the end, Debian will have to trust the maintainer and the maintainer will have to trust upstream. Documenting that in debian/copyright seems to be the only sane option.

Now can we all just shut up and release Lenny? Please?

This is the final part of my review of popular articles on my news websites during 2008. It follows part 1 and part 2, but it can be enjoyed on its own, like an nice ripe cheese. Unlike one friend, I do not like cheese for breakfast.

September

• Are Your Free Software T-Shirts Ethical? This question got a lot of views, but no answers yet. Again, those statistics surprised me.
• Away From Keyboard was a listing of events I attended in early September, with some links to summaries in the article and the comments. (I can t find my Bristol Knowledge Unconference report just now. Did I post it to another site?)

October

• The Trouble With Big Webmail built on my past comments about Yahoo and broadens them to Hotmail/Live and GoogleMail. I m still convinced those sites remain popular because they are good at marketing, rather than being good at email. In fact, they re so bad at email, almost no-one should use them.
• Pulling One File From a Plesk Backup - It seems I wasn t the only person not to know about that.
• Nestle Boycott 20th Anniversary seems to be the only time I wrote about boycotts this year, surprisingly (I know I m pretty militant and direct-action-y), but what more deserving target?

November

• Which Social Bookmarking? was by far the most popular article in November and it also got lots of helpful comments. I posted my review and choices (such as they are) a few days ago.

December

• Bristol Alternative Christmas Quiz. Everyone loves a good seasonal quiz, it seems. I didn t get to it (arrrrrrrrrgh busy!) but I hope it went well for BWCC.
• Smart Tricks with ssh was only the second meme-like post to make the most popular list this year.
• and it s still a bit early to evaluate articles from late December! I may add a few to this list later.

Happy New Year 2009, in case I don t see you before then!

Given the recent brouhaha in Debian, and General Resolution regarding Lenny s Release policy as it relates to Firmware and Debian s Social Contract, which has led to the resignation of Manoj Srivastava from the position of Secretary for the Debian Project, I m reminded of the following passage from Gordon Dickson s Tactics of Mistakes (part of Dickson s Childe Cycle, in which he tells the story of the rise of the Dorsai):

No, said Cletus. I m trying to explain to you why I d never make an Exotic. In your calmness in the face of possible torture and the need to kill yourself, you were showing a particular form of ruthlessness. It was ruthlessness toward yourself but that s only the back side of the coin. You Exotics are essentially ruthless toward all men, because you re philosophers, and by and large, philosophers are ruthless people. Cletus! Mondar shook his head. Do you realize what you re saying? Of course, said Cletus, quietly. And you realize it as well as I do. The immediate teaching of philosophers may be gentle, but the theory behind their teaching is without compunction and that s why so much bloodshed and misery has always attended the paths of their followers, who claim to live by those teachings. More blood s been spilled by the militant adherents of prophets of change than by any other group of people down through the history of man.

The conflict between idealism and pragmatism is a very old one in the Free and Open Source Software Movement. At one end of the spectrum stands Richard Stallman, who has never compromised on issues regarding his vision of Software Freedom. Standing at various distances from this idealistic pole are various members of the Open Source Community. For example, in the mid-1990 s, I used to give presentations about Linux using Microsoft Powerpoint. There were those in the audience that would give me grief about using a non-free program such as MS Powerpoint, but my response was that I saw no difference between driving a car which had non-free firmware and using a non-free slide presentation program. I would prefer to use free office suite, but at the time, nothing approached the usability of Powerpoint, and while dual-booting into Windows was a pain, I could do a better job using Powerpoint than other tools, and I refused to handcap myself just to salve the sensibilities of those who felt very strongly about Free Software and who viewed the use of all non-Free Software as an ultimate evil that must be stamped out at all costs. It is the notion of Free Software as a philosophy, with no compromises, which has been the source of many of the disputes inside Debian. Consider, if you will, the first clause of the Debian Social Contract:

Debian will remain 100% free We provide the guidelines that we use to determine if a work is free in the document entitled The Debian Free Software Guidelines. We promise that the Debian system and all its components will be free according to these guidelines. We will support people who create or use both free and non-free works on Debian. We will never make the system require the use of a non-free component.

This clause has in it no room for compromise. Note the use of words such as 100% free and never make the system require the use of a non-free component (emphasis mine). In addition, the Debian Social Contract tends to be interpreted by Computer Programmers, who view such imperatives as constraints that must never be violated, under any circumstances. Unfortunately, the real world is rarely so cut-and-dried. Even the most basic injunctions, such as Thou shalt not kill have exceptions. Few people might agree with claims made by the U.S. Republican Party that the war in Iraq qualified as a Just War as defined by Thomas Aquinas, but rather more people might agree that the July 20, 1944 plot to assassinate Hitler would be considered justifiable. And most people would probably agree most of the actions undertaken by the Allied Soldiers on World War II battlefields that involved killing other soldiers would be considered a valid exception to the moral (and for those in the Judeo-Christian tradition, biblical) injunction, Thou shalt not kill . As another example, consider the novel and musical Les Mis rables, by Victor Hugo. One of the key themes of this story is whether or not Thou shalt not steal is an absolute or not. Ultimately, the police inspector Javert, who lived his whole life asserting that law (untempered by mercy, or any other human considerations) was more important than all else, drowns himself in the Seine when he realizes that his life s fundamental organizing principle was at odds with what was ultimately the Right Thing To Do. So if even the sixth and eighth commandments admit to exceptions, why is it that some Debian developers approach the first clause of the Debian Social Contract with a take-no-prisoners, no-exceptions policy? Especially given the fourth clause of the Debian Social contract:

Our priorities are our users and free software We will be guided by the needs of our users and the free software community. We will place their interests first in our priorities. We will support the needs of our users for operation in many different kinds of computing environments. We will not object to non-free works that are intended to be used on Debian systems, or attempt to charge a fee to people who create or use such works. We will allow others to create distributions containing both the Debian system and other works, without any fee from us. In furtherance of these goals, we will provide an integrated system of high-quality materials with no legal restrictions that would prevent such uses of the system.

This clause does not have the same sort of absolutist words as the first clause, so many Debian Developers have held that the needs of the users is defined by 100% free software . Others have not agreed with this interpretation but regardless of how needs of the users should be interpreted, the fact of the matter is, injuctions such as Thou shalt not kill are just as absolute and yet in the real world, we recognize that there are exceptions to such absolutes, apparently unyielding claims on our behavior. I personally believe that 100% free software is a wonderful aspirational goal, but in particular with regards to standards documents and firmware, there are other considerations that should be taken into account. People of good will may disagree about what those exceptions should be, but I think one thing that we should consider as even higher priority and with a greater claim on how we behave is the needs of our users and fellow developers as people. For those who claim Christianity as their religious tradition, Jesus once stated,

Thou shalt love the Lord thy God with all thy heart, and with all thy soul, and with all thy mind. This is the first and great commandment. And the second is like unto it: Thou shalt love thy neighbour as thyself. On these two commandments hang all the law and the prophets.

Even for those who do not claim Christianity as their religious tradition, most moral and ethical frameworks have some variant on the Golden Rule: Do unto others as you would have them do unto you . I would consider, for example, that the Golden Rule is at least a high priority claim on my behavior as the notion of free speech, and in many cases, it would be a higher priority claim. The recent controversy surrounding Josselin Mouette was started precisely because Joss has taken a something which is a good thing, namely Free Speech, and relegated it to a principle more important than all else, and claiming that any restraint on such a notion was equivalent to censorship. I think the same thing is true for free software, although it is a subtler trap. Philosophical claims than 100% free software as most important consideration is dangerously close to treating Free Software as the Object of Ultimate Concern or in religious terms, idolotry. For those who are religious, it s clear why this is a bad thing; for those who aren t if you are unwilling to worship a supernatural being, you may want to very carefully consider whether you are willing to take a philosophical construct and raise it to a position of commanding your highest allegiance to all else, including how you treat other people. Ultimately, I consider people to be more important than computers, hardware or software. So over time, while I may have had some disagreements with how Mark Shuttleworth has run Canonical Software and Ubuntu (but hey, he s the multimillionaire, and I m not), I have to give him props for Ubuntu s Code of Conduct. If Debian Developer took the some kind of Code of Conduct at least as seriously as the Social Contract, I think interactions between Debian Developers would be far more efficient, and in the end the project would be far more successful. This may, however, require lessening the importance of philosophical constructs such as Free Speech and Free Software, and perhaps becoming more pragmatic and more considerate towards one another.

Suppose you have a free software package that includes an OpenID login form. Such forms are supposed to include a little OpenID logo . But as hard as you look, you can't find a license for the OpenID logo. Though there seem to be indications that it might get one in 2009, it seems like it will not be free enough to be included in a free software package. Due to this problem, all the ikiwiki sites out there have not used the OpenID logo, or indeed any logo, in their login form for the 2+ years that ikiwiki has proudly supported OpenID. That was so suboptimal that I spent some donations to commision an unofficial OpenID logo . It's freely licensed for use in whatever. Putting the little OpenID logo in the login form is a nice touch, it helps spread awareness about OpenID and users learn to look for it. The fact that free software packages can't include it weakens that. Having an alternative logo that evokes the "real" logo and the concept of a login is better than no logo at all. But it is also needlessly confusing. I encourage the new OpenID board to reconsider plans for logo licensing, and bear in mind that many free software packages can have native OpenID support, and should be able to include a copy of the logo, without worrying about it conflicting with their license, or not being free enough to be included in a free software distribution.

I was asked for my borscht recipe. This is loosely derived from a recipe that is really weird -- it says to throw away the beets! Both that recipe and mine are probably very unauthentic. But good. 6 cups water
3 medium size beets
2 medium size potatoes, quartered
1/2 cup chopped carrots
1 stalk celery, chopped -- optional
1/2 a bell pepper, chopped (red or green) -- optional
1/3 cup butter
1/2 to 1 cup chopped onion
1/2 cup tomatoes (fresh are best, canned or tomato paste + water is ok)
1/4 cup milk
2 cups finely chopped cabbage
1 tablespoon vinegar
1 teaspoon dried dill weed
sour cream Put water in a large pot on high heat. Add beets, potatoes, carrots, celery, and bell pepper. Cover and boil until potatoes are tender. Meanwhile, melt butter in a skillet. Saute onion in butter until tender, about 5 minutes. Stir in tomatoes, reduce heat to medium low, cover, and simmer for 15 minutes. Remove half of sauce from skillet into a medium size bowl. Add cabbage to remainder of sauce in skillet and cook coverted on medium low heat, stirring occasionally for ten minutes, or until tender. Reduce heat on pot to a simmer. Remove beets from pot and set aside to cool. Remove quartered potatoes and add to bowl with sauce. Mash potatoes, adding milk, until creamy. Stir mashed potato mixture into soup in pot. Grate beets, removing skin if desired, or grating it in. Combine grated beets and cabbage into pot. Add vinegar to taste (optional but recommended with sweet beets). Add salt and pepper to taste. Stir in a teaspoon of dill. Cover and cook for at least another 5 minutes on low heat, then let it cook in its residual heat for as long as desired. Serve hot, garnished with sour cream and dill.

Back a while I created an updated mutt package which builds upon the previous work with mutt-ng. The package which I had constructed had two new features, well if you're generous. I guess there was only technically one new feature:

• The ability to cause the sidebar to only display folders with new mail in them.
• + Updated navigation to make next/prev work correctly with only new mail-containing folders.

I still hate quilt with a passion, but I've synced my patches with the current unstable Mutt - and there are now source & binary packages (x86 + amd64) in my apt-get repository for Etch. I can rebuild for Lenny if there is any interest. But it might take me a while to setup lenny chroots. As a graphical example this is what you will end up with:

• The sidebar with all folders visible (the default).
• The sidebar listing only folders with unread messages.

OK I've tweaked the settings a little, but only a little:

# set up the sidebar, default to being visible
set sidebar_width=45
set sidebar_visible=yes
set sidebar_delim=' '
color sidebar_new red default
# ctrl-n, ctrl-p to select next, prev folder
# ctrl-o to open selected folder
bind index \CJ sidebar-scroll-down
bind index \CK sidebar-scroll-up
bind index \CP sidebar-prev
bind index \CN sidebar-next
bind pager \CO sidebar-open
bind index \CO sidebar-open
# Remap bounce-message function to "B"
bind index B bounce-message
#
#  Toggle visability of the sidebar.
#
macro index b '<enter-command>toggle sidebar_visible<enter><refresh>'
macro pager b '<enter-command>toggle sidebar_visible<enter><redraw-screen>'
#
#  Show folders with new mail only
#
macro index E '<enter-command>toggle sidebar_newmail_only<enter>'
macro pager E '<enter-command>toggle sidebar_newmail_only<enter>'

ObFilm: Joan of Arc

Being left in the IT medieval, thanks to my lovely telco, the only way I can (very far away) simulate a broadband connection, is stipulating a flat 56k contract (don't laugh, please). The name of this "offer" is TeleconomyInternet.

Do you think that firing up gnome-ppp on a Thinkpad T30 laptop would work at the first try? Of course not... The problem seems to be in the telephone number to call, that badly interact with SmartLink 56k modem. Here's the story, in case you'll find yourself in the same situation.

The modem on the T30 is:

$ lspci   grep Modem
00:1f.6 Modem: Intel Corporation 82801CA/CAM AC'97 Modem Controller (rev 02)

that is a SmartLink modem, supported well by sl-modem-deamon package and by alsa:

$ ps -eo command   grep slm
/usr/sbin/slmodemd --alsa -c ITALY modem:1

Remember to edit /etc/default/sl-modem-daemon file and set the right country for you.

Now, I installed gnome-ppp (just to try an easy tool), that uses wvdial under the hood.

wvdial auto-detection utility works quite well, and the found Init2 string:

ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0

works well if the number to call is a local one (I tried it before, and worked). Since Teleconomy provides a state-wide number (702...) with the above string the connection will fail with a sad:

NO CARRIER
No Carrier!  Trying again.

But there is a fix: first, the Init1 should be ATX3 (but it seems to be enbedded in the one below), and the last arguments must be changed (well, I think it has something to do with line speed (MS=34 will result in a 33.6k line) or protocol, but don't care too much as long as it works) resulting in this Init2:

ATX3Q0 V1 E1 S0=0 &C1 &D2 +MS=90

And remember to select the "stupid mode" in gnome-ppp, and you'll have a nice:

--> Initializing modem.
--> Sending: ATX3
ATX3
OK
--> Sending: ATX3Q0 V1 E1 S0=0 &C1 &D2 +MS=90
ATX3Q0 V1 E1 S0=0 &C1 &D2 +MS=90
OK
Modem initialized.
Sending: ATM1L3DT7020187187
Waiting for carrier.
ATM1L3DT7020187187
CONNECT 52000
--> Carrier detected.  Starting PPP immediately.
--> Starting pppd at Fri Dec 26 19:12:23 2008
--> Pid of pppd: 4870
--> Using interface ppp0
--> local  IP address ......

I'll move to ppp soon, since gnome-ppp seems to not understand "fsck, reconnect when disconnect, dude" option.

How nice is to be in the 21st century...

Can anybody point me to a beginners tutorial for running PyGTK on OS X?I've written this little BPM tap program in PyGTK, that gives you a speed plot and confidence interval, works well for me.A friend of mine would like to use it, but is running OS X, and doesn't know much about computers. I know next to nothing about OS X. I know it already has Python2.5 installed, and that's about it.So anyone got a tutorial that doesn't require previous knowlege about using fink, ports or whatever? Something that I can point a real novice user to?Via email please, my blog intentionally doesn't have comments. Erich AT Debian DOT org.[Update: so far, I've only had requests about my bpm toy and about the same kind of instructions, but no links to refer people to...]

When a cat is on your lap or very close to you, it means there is no better place for him in the whole world. And cats are very smart. And demanding. They are absolutely free souls and natural born gourmets.

But where is the tip? Just around the corner, read on.

Listen to the world around you, the more related to nature the message is, the more important and meaningful is the wisdom coming with it.