Search Results: "rul"

21 January 2007

Daniel Burrows: How to Breathe, Functionally (part 1)

David Welton writes that the problem with functional languages is that you don't get to mentally "catch your breath" between computations; piling function invocations on function invocations makes it hard to understand what's actually being calculated. He has an amusing summary of how functional code can become unreadable:


Do this with the result of this with what this produces when given this input that is derived from this function which takes the result of this other thing after calculating the result of something that is derived from the output of a function that ...


Indeed, I've seen functional code like this (much of it written by myself). Here's a particular gem:


f splitPoints = map fst $ tail $ scanl (flip ($) . snd) ([], toSplit) $
map splitAt splitPoints


I wrote this code a year ago, and I don't know what it does! It's hard to read and hard to understand. So, however, is imperative code that says


Add 5 to A, now if B is not zero we jump to where we subtract one from B and if C is less than 3 we add 4 to C otherwise we divide C by 2 and then add it to A, then we ...


The reason both pieces of pseudocode are hard to understand is that they don't use the abstraction features built into the language. Good imperative code uses meaningful variable names, uses the structured-programming primitives the language provides, and abstracts complex operations into functions.

My rule of thumb (which is, heh, not always religiously followed) is that a piece of program structure that does more than about 3-6 things is probably too complicated; it seems like that's about as much local context as I can hold in my head comfortably (and the number is more 3 than 6 these days, if you get my drift). As David puts it,


If you'll bear with a poor comparison, sometimes programming is like a song. You do the tricky section, then you go back to an easier section and catch your breath for a bit.


The problem seems to be that people trained on imperative languages, when they're tossed into the ocean of functional coding, can't find the guideposts and landmarks that would help them impose some structure on the water. Should I put this complicated expression inside a "let", or break it out into another function? When do I use a higher-order function to unify two dissimilar pieces of code? Should I write this code in terms of higher-order library functions or directly? It's easy to wander off down a chain of nested invocations of anonymous higher-order functions, or composed partially applied library functions, just "because you can" and end up forgetting what you were trying to do in the first place.

Luckily, some of the basic rules of keeping your code readable remain more or less the same: assign intermediate results to variables with meaningful names, and abstract complicated operations into separate functions. To tackle David's example:


You get sections with DoThisAndThisAndThatWithTheResultsOfSelectThisFromTableXYZ that you have to slow down to think through.

...

GoToTheNextTrickyBit where ManyThingsHappen.

All in all, you wind up with a sort of ebb and flow that works out pretty well if you're in tune with the code being written, going from intense calculations, to a line or two where you do no more than store things in a variable, take a breather, and prepare for the next important stanza.


The way to "take a breath" in a functional language is generally to assign an intermediate result to a variable, or to extract a complicated operation into a function and give it a meaningful name. One Haskell approximation to David's example is


do DoThis
AndThis
results - SelectThisFromTableXYZ
someVal - AndThat results
GoToTheNextTrickyBit someval


If you're writing pure functional code, you might write


let
results = SelectThisFromTableXYZ
someVal = AndThat results
in
GoToTheNextTrickyBit someVal


Of course, this will become unreadable if you write a huge complicated nested "let" expression in place of "AndThat". For a truly dreadful example of this kind of coding, check out the Gnucash Scheme code that generates the "Cash Flow" report (warning: read this in short bursts, or you WILL go blind). This is exactly analogous to writing too many nested loops and conditionals in imperative code, and the fix is the same: find logically distinct bits of code and extract them into separate functions.

I have a few more thoughts, but this post is getting too long, so I'll inflict them on the world in another post. The rule that short things are easier to understand applies to blog posts as well as code. :-)

17 January 2007

Martin F. Krafft: Stuff I forgot to pack

After an amazingly stressless journey [0], I've arrived yesterday evening in Limerick and met Daren and Mel, two alumni of UL who let me move into a spare room in the house they're renting. To be exact, Mel actually lives next door, but who cares about details. After a bit of unpacking, we headed for the pub and returned for an extensive session of Guitar Hero on the Playstation II, which is a truly fun and excellent game.
[0]

I brought my bike along, for transport and exploring. This involved packing a bit of extra stuff, borrowing a friend's Tranzbag, arriving early at the airport to get it boxed to ensure a safer journey, picking it up in Dublin, and somehow getting it to Limerick, where I'd have to reassemble it. I wasn't really looking forward to the trip. It turned out to be stressless, as I said. The boxing cost me 10 Franks at the airport and was completed in less than 5 minutes, nobody at the check-in counter said anything, as if it was the most normal thing for them to check in bikes, some dude retrieved the box in Dublin and placed it on my cart, and while I was waiting, I found out that I wouldn't have to hop on a coach (bus) to go to the Dublin train station, catch a train to Limerick, and then finish the last mile in a taxi: a bus operates fairly regularly directly from Dublin airport to Limerick, taking four hours for the journey. That's about how long it would have taken me via the train station, but I only had to pay 19 the train itself would have been 25 , plus the taxi charges.

The coach driver was curious when he saw the box, but I didn't have to pay extra, and after watching Airport on my laptop and listening to the new Katatonia album, I found myself with my bag and the bike next to the university's main entrance, with Mel and Daren on the way to pick me up. Five minutes later, I was in my home-to-be-for-five-weeks.

I dropped into bed at around 4am and promptly overslept the next morning, then decided to use the day for errands, hitched a ride from Laura (the stressed-out Skynet president and Skycon organiser) to an excellent nearby bike store where I reassembled the bike, got myself a lock (see below) and a mud guard for the front wheel, then headed off for the campus bank to open an account, rode back and forth between the Lero building and the bank to satisfy their bureaucratic thirsts and finally obtained the details for my very own Irish bank account. And I squeezed a bit of grocery shopping in between. Meeting Brian (my supervisor) in the hallway, we agreed on a meeting on Monday, so the remainder of this week is for finishing the various proposals and papers on my to-do list, preparing for the meeting by getting back to speed with my research, as well as getting to know the local party crowd exams are about to end in Limerick and I've been told to have picked quite an excellent time of year for my visit because the university will become a mad house this weekend. We'll see how compatible those tasks are, but boy am I glad that I am not living anywhere near the student villages, or as I put it last night: I don't mind going to parties, but if the party comes to where I am, I'd rather not have it be every night. So this would bring me to the title of this post: stuff that I forgot to pack:
  • A second sweater, it's pretty cold here and I don't want to be wearing the same sweater every day (even though it's merino wool and thus woudn't start to smell).
  • A pair of windcheater gloves, the thin wool ones I have aren't exactly made for cycling.
  • Rain trousers to wear over my regular pants. It's raining and wet and the light colours of the pants I am wearing are a high-contrast basis for the mud.
  • My bike's lock key. Fortunately I left it unlocked, but still had to go out and buy a new lock (which was more painless than having the key sent here).
  • Comfy sneakers for home use, as it's too cold to just be wearing socks.
Due to popular request, I shall be back here with pictures of my surroundings one of these days. NP: I Love You but I've Chosen Darkness / Fear Is on Our Side

16 January 2007

Joey Hess: dealing with dialup

Prompted by a friend who finds a dialup internet connection about as appetising as being food for mosquitoes, I thought I'd share some tips for using the current internet on dialup. It's easy to say that dialup is dead, but I've transitioned from dialup to broadband and back something like 4 times so far, and since I don't want to rule out spending time in the middle of nowhere in Costa Rica, or at McMurdo, or in an RV, or in a Bigelo orbital hotel, amoung other dream destinations, I don't let availability of bandwidth deter me. Here's some habits you can learn to lessen the pain: work local This is the most important thing. ssh is great, but latency sucks. If you need to edit a file on a remote system, edit it locally, and copy it across. Since I keep most of my files in subversion, I edit it locally, commit it, and update on the remote machine. If ssh isn't appropriate, use rsync or something. Editing files is just one example; I try to avoid doing anything on a remote machine if I can do it locally. For example, even though an irc client in screen on a remote machine is a nice setup, it's to be avoided on dialup. Another example is using offlineimap to download your mail, so you can read it locally. Yet another example: I have all my projects scripted so I can make a complete release with just single locally ran command, that goes off and automatically triggers everything that needs to happen on the server. Same for blogging, and for making changes to my wikis. cache everything This is an obvious one, but worth mentioning. The most important cache is probably actually a caching dns server. Dns lookups are an easily missed slowdown (true even for many cable/dsl connections). Other useful caches include a squid web cache, and a shared apt cache. Also get in the habit of squirelling files away, instead of deleting them, to avoid re-downloads. know your traffic Look at everything that causes internet traffic, and turn off the fluff. Even the small fluff will make a difference on dialup. This includes turning off VPNs, not keeping chat programs running in the background. If you're not currently using irc, close it. Use a remote irc proxy server to cache messages while you're away, to avoid missing anything. Avoid sitting on web pages that refresh or stream content. Keep iftop running all the time, and know what all the sources of traffic on it are. You can easily waste 25% of a dialup pipe with fluff if you don't keep it under control. downloads are batch jobs Learn to treat downloads (and uploads) as batch jobs that go on in the background or overnight. Don't let your work be blocked waiting on a download if at all possible; don't sit staring at the screen every time a five minute download pops up; go do something else instead. When possible, try to anticipate what you'll need and download it the night before. Set up a remote download queue, of things you want to download at some point, sorted by priority, and rsync from it as a batch job overnight or when you don't need bandwidth for anything else. If you upload a lot of stuff, set up an upload queue too. don't try to do everything on dialup Some things are just not appropriate for dialup. There's no point in trying to download CD images, or much music, or apt-get dist-upgrade, or watch the latest videos on YouTube, or whatever. Even things like Google Maps are just not designed for dialup; the one square you want to see is always the one that will fail to download while the other 8 squares slooowly download in parallell. And AJAX sucks. So avoid this stuff on dialup, don't frustrate yourself. Schedule pit stops where there's high bandwidth to do this kind of thing. Find alternatives you can use while on dialup -- use old fashioned mapping services, and instead of pursuing the latest and greatest videos, download an e-book -- a whole book downloaded in just 5 minutes -- who said this dialup was slow?

14 January 2007

Andree Leidenfrost: Debian Pre-Release of Mondo Rescue 2.2.1, Take 2

mindi-2.21~r1021-2 and mondo-2.21~r1021-2 are now on http://people.debian.org/~andree/packages/ with the following changes:
  • petris works again during restore (self-inflicted, oh well).
  • Restore of ISO images and friends should now work when gzip (i.e. '-G') is used.
  • The network interfaces should be fine now when booting into a restored system for the first time.
Other than that:
  • The crash in mondorestore when nuking from tape has disappeared. I have no idea what caused it or why it went away again, though...
  • Kernel 2.6.18-3-k7 hangs when 'acpi=off' is specified (which is the default as per mindi's ADDITIONAL_BOOT_PARAMS, so restore fails with this kernel). I have filed bug #406809 which may or may not be related to #389931.
  • The issue with booting a (NTFS) Windows partition failing after a restore appears to be normal as per the ntfsclone manpage:
    Usually, Windows will not be able to boot, unless you copy, move or restore NTFS to the same partition which starts at the same sector on the same type of disk having the same BIOS legacy cylinder setting as the original partition and disk had.
  • I have tried a few things playing with parted and ntfsresize, but so far the only thing that works reliably in order to get Windows to boot is resizing the partition using gparted. I still have to figure out what it is that gparted does differently. (If you know, please tell me!)
Off to bed now so that I'm fine and dandy when I pick up Bruno from the airport in the morning for lca2007. :-)

Manoj Srivastava: Mail Filtering with CRM114: Part 3

Uphold, maintain, sustain: life in the trenches Now that I have a baseline filter, how do I continue to train it, without putting too much of an effort? There are two separate activities here, firstly selecting the mails to be used in training, and secondly, automating the training and saving to the mail corpus. On going training is essential; Spam mutates, and even ham changes over time, and well trained filters drift. However, if training disrupts normal work-flow, it won't happen; so a minimally intrusive set of tools is critical Selecting mail to train filters There are three broad categories of mails that fit the criteria: 1. Misclassified mail This is where human judgement comes in, to separate the wheat from the chaff. 2. Partially misclassified mail This is mail correctly classified overall, but misclassified by either crm114, or spamassassin, but not both. This is an early warning sign, and is more common than mail that is misclassified, since usually the filter that is wrong is wrong weakly. But this is the point where training should occur, so that the filter does not drift to the point that mails are misclassified. Again, the mlg script catches this. 3. Mail that crm114 is unsure about This is Mail correctly classified, but something that mailreaver is unsure about -- and this category is why mailreaver learns faster than mailtrainer. Spam handling and disposition Spam handling schema. At this point I should say something about how I generally handle mails scored as Spam by the filters. As you can see, the mail handling is simple; depending on the combined score given to the mail by the filters. The handling rules are: So, any mail with score less than 15 is accepted, potentially after grey-listing. The disposition is done according to the following set of rules: In the last 18+months, I have not seen a Ham mail in my realspam folder; chances of Ham being rejected are pretty low. My Spam folder gets a ham message every few months, but these are usually spamassassin misclassifying things; and mlg detects those. I have not seen one of these in the last 6 months. So my realspam canary has done wonders for my peace of mind. With ideally trained filters, spam and realspam folders would be empty. mail list grey I have created a script called mlg ("Mail List Grey") that I run periodically over my mail folder, that picks out mails that either (a) are classified differently by spamassassin and crm114, or (b) are marked as unsure by mailreaver. The script takes these mails and saves them into a grey.mbox folder. I tend to run them over Spam and non-Spam folders in different runs, so that the grey.mbox folder can be renamed to either ham or junk, in the vast majority of the cases. Only for misclassified mails do I have to individually pick the misplaced email and classify it separately from the rest of the emails in that batch. At this point, I should have mail mbox folders called ham and/or junk, which are now ready to be used for training either crm114 or spamassassin or both. Processing these folders is the subject of the next article in this series.

Steve McIntyre: EPIC - we're done!

A group of us have been doing the Every Pub In Cambridge challenge for the last year and a bit. There are various lists of the pubs in the City of Cambridge around the Net in various places, and we based ours on the one at www.cambridgepubs.co.uk. Most Thursdays, Phil, Pod, Jon and I picked 3 or 4 pubs from their handy tube-style map and went and visited. The rules are simple: Today, we visited the last 4 pubs and finished after 14 months. We would have finished in under a year, but the football World Cup and Pod's travel-heavy new job slightly dented our schedule. In all, we visited 108 pubs and couldn't make it to 11 due to them being closed. The point of the exercise? We already knew of several good pubs in Cambridge, but expected there to be plenty of others. Plus, it was something fun to do on Thursday nights and a good excuse to drink more beer... :-) In the end, we each found some good new places to go, and some that we'd never go back to by choice. Our choices for best pub vary, but our respective best 5s basically agree. My own are the Live and Let Live, the Kingston Arms, the County Arms, the Castle Inn and the Free Press. Alongside those, there are plenty more I'd be happy to visit again. After more discussion, our other opinions were unanimous: the worst was the Duke of Argyle (nasty!), and without a doubt the most entertaining was the Cross Keys (with an amazingly funny barman). At some point we may get around to writing up reviews and posting them somewhere, but not right now... *grin*

12 January 2007

Gustavo Franco: X0 won't meet the goals in Brazil

First, don't get me wrong. The project is really interesting, i haven't played with a beta yet but so far it looks real good. I've some doubts about the UI, but it isn't what this post is about. I think that X0 will fail here in Brazil.

The Brazilian government doesn't want to buy the laptop to some children in public schools (the poor pay for it through taxes, the not so poor pay for private schools and the public ones through the same taxes). The gov't is going to give it to some teachers to motivate them and will try to add X0 in the daily routine of a school. The cons of this approach:
We've a huge cellphone market here, initiatives like OpenMoko would work way better than X0. The cell phone companies would sell the phones (eg.: Neo1973) to the people that can buy (just a few teachers in really poor areas haven't cell phones yet) and the gov't could give one for each children in the public schools. There's a gov't program that try to keep the children in the public schools giving the family (when it's really poor) some money. The same rules could be applied for the cell phone usage, if you leave the school, you loose the gadget. Let us give the kids something that's more than useful and they need. We can give them something they want. I want one X0, but the teachers and children here doesn't. I'm sorry, somebody should tell you about what's really going on here, sometimes it's much more about social aspects than tech specs.

Erich Schubert: RDF representation of packages?

I wonder if we should do some RDF representation of packages - especially including dependency data. That way we can maybe use some RDF reasoners to query our data, and maybe extract some interesting information. On the other hand, people with interest in RDF to use our real-world data for their experiments, and maybe we get something back from them. There is a couple of package metadata we currently are not tracking inside the actual archive, but in different places. Including licensing information (debian/copyright), homepage location (on packages.qa.debian.org), download location (debian/watch) - it would be nice to aggregate these into some RDF store, and export them somehow. For most of the package information (especially dependency information), we'll have to write our own ontology (I wonder if we can map version numbers to some standard rule language, or if applications will need an external reasoner to process them?); for some things we can reuse the FOAF (Friend-of-a-friend) or DOAP (Description of a project) ontologies. The first is rather common for describing people, people-people and people-thing relationships; the latter was designed for describing opensource projects (but won't be directly applicable to packages of a project). I've blogged about my RDF export of Debtags data before; the canonical first step would actually have been to export the package data, and enrich it with the Debtags collected data... Note that RDF is designed in a way that you can have one site provide metadata for another site. For example, the Debtags RDF export contains "category" information for Debian packages, but does not contain e.g. the description of the packages it talks about via an URI. So there is nothing wrong from a RDF point of view of keeping e.g. the licensing, watch or homepage data separate. For the Google Summer of Code, there was a proposal including "collaborative repository of meta-informations about source packages (CRMI)"; but the first part of the proposal, the "distribution wide tracker tool (DWTT)" showed to be a bigger task than expected. But maybe we'll still see CRMI at some point, and maybe we can have it provide an RDF export of data (using a semantic wiki might be a good starting point for CRMI maybe?). [P.S. this blog posting maybe belongs more into the en/linux/debian category. But only the xml category is also syndicated on planet.XMLhack, and I want this post to go there to reach more RDF users. I really need to switch my blog to some software which supports tagging...]

11 January 2007

Eddy Petrișor: how to disable the pcspkr in Etch?

The new laptop I just bought had the bad habit of using the pc speaker at a loud level. It was deranging the coleagues at work when autocompletion yelded more than one result or none (it also scared me a few times during the last nights when I was configuring and testing the laptop).

So I unloaded the module (pcspkr) by hand and wanted to make the system not load it anymore.

So I thought that blacklisting it would do the trick. Probably some people know already that adding al line like:

blacklist pcspkr

in /etc/modprobe.d/blacklist doesn't work. The rest of you find out now. I found out at the next reboot, when I heard again the beep.

Ok, I googled a bit and found some info about restricting ipv6 modules being loaded so I though I could do the same for pcspkr. So I tried:

to change the line from /etc/modprobe.d/pnp-hotplug:

alias pnp:dPNP0800 pcspkr

into

alias pnp:dPNP0800 off

That didn't work either.

So, at this point I googled more and found some outdated information about update-modules.

That brought me to modprobe.conf(5) which pointed to the "right" solution.Write the following in some file in /etc/modprobe.d/ (I added it in blacklist, since it seemed the right place)

install pcspkr true

Which seems to do what I want.



Still something tells me that either:

1) there is a cleaner solution I am not thinking about

or

2) I could have told udev not to load the module... And I feel this solution should have to do something with these lines and "ACTION":

eddy@bounty /etc/udev $ grep -r pcspkr *
persistent-input.rules:DRIVERS=="pcspkr", ENV ID_CLASS ="spkr"
rules.d/z20_persistent-input.rules:DRIVERS=="pcspkr", ENV ID_CLASS ="spkr"



A clean solution would be appreciated.

Martin F. Krafft: Destinationen

Just now on the Swiss hotline, a lady announced in the typical Swiss way of speaking "high German" that the airline now offers bla bla bla to over 200 "Destinationen". I had to snicker. English readers may wonder what the deal is, and even Germans might just yawn: "Destinationen" is not a German word, it's "destinations" germanified, and it's no news that the German language is seriously deteriorating as English words are creeping in. At the risk of being repetitive: it's not the English words per se, it's the fact that they are being conjugated or declinated according to German rules, which irritates me (and many others). The reason for this blog post was simply the humour: a Swiss lady speaking "high German" with her subtle Swiss accent, using words that don't exist as if there was nothing to it. I actually had to laugh out loud. Note that I have great respect for Swiss people speaking German, it being somewhat of a foreign language to them. I am perfectly aware that many are uncomfortable doing so, and this post is not trying to make fun of them, but rather expose the irony of the use of a non-German word. NP: Dream Theater / When Dream and Day Reunite Update: Christof Roduner informs me that "Destinationen" is actually a German word, or at least recognised by the Duden:
De s ti na ti on, die; -, -en <lat.> (Reiseziel; veraltet f r
Bestimmung, Endzweck)
Aus: Duden - Die deutsche Rechtschreibung, 24. Aufl. Mannheim 2006.
That must be the new orthography, which I won't comment on at this point.

Miriam Ruiz: Connecting my MIDI keyboard to the laptop

Laptops don’t usually have as much ports as desktop computers. I used to connect my MIDI keyboard to the computer via a game port adapter. Now the game port has been to some extent replaced by USB connectors, especially in laptop computers, so I decided to get myself a MIDI-USB adapter. After searching the web investigating which device would be the best for my Debian system, I decided for the MidiSport Uno, which is elegant, compact, and compatible with Linux. Yesterday I was finally able to test it, and it works perfectly. The only thing I don’t like is that its firmware doesn’t seem to have a free license. After some initial trials with fxload, which didn’t work due to some pipe error, I realized that it wouldn’t work under the current udev system (I’m using a 2.6.18 kernel right now), but the rule for udev I added seems to work. Ubuntu’s package for it has also helped me a lot. After installing it, I was able to test that it worked both with a direct cat from the MIDI device and using rosegarden. There seems to be also an OSS driver supporting the MidiSport 1×1/2×2 with a GPL’d firmware, but I haven’t tried it yet. It would be nice to be able to have a GPL’d firmware in it.

Manoj Srivastava: Mail Filtering with CRM114: Part 2

Or, Cleanliness is next to godliness The last time when I blogged about Spam fighting Mail Filtering With CRM114 Part 1, I left y'all with visions of non-converging learning, various ingenious ways of working around a unclean corpus, and perhaps a sinking feeling that this whole thing was more fragile than it ought to be. During this eriod of flailing around, trying to get mailtrainer to learn the full corpus correctly, I upgraded to an unpackaged version of crm114. Thanks to the excellent packaging effort by the maintianers, this was dead easy: get the debian sources using apt-get source crm114, download tghe new tarball from crm114 upstream, cp the debian dir over, and just edit the changelog file to reflect the new version. I am currently running my own, statically linked 20061103-Blame Dalkey.src-1. Cleaning the corpus made a major difference to the quality of discrimination. As mentioned earlier, I examined every mail that was initially incorrectly classified during learning. Now, there are two ways this can happen: That the piece of mail was correctly placed in the corpus, but had a feature that was different from those learned before; or that it was wrongly classified by me. When I started the chances were almost equally likely; I have now hopefully eliminated most of the misclassifications. When mailtrainer goes into cycles, retraining on a couple of emails round after round, you almost certainly are trying to train in conflicting ways. Cyclic retraining is almost always a human's error in classification. Some of the errors discovered were not just misclassifications: some where things that were inappropriate mail, but not Spam; for instance there was the whole conversation where someone one subscribed debian-devel to another mailing list, there was the challenge, the subscription notice, the un-subscription, challenge, and notice -- all of which were inappropriate, and interrupted the flow, and contributed to the noise -- but were not really Spam. I had, in a fit of pique, labelled them as Spam; but they were really like any other mailing list subscription conversations, which I certainly want to see for my subscriptions. crm114 did register the conversations as Spam and non-Spam, as requested, but that increased the similarity between Spam and non-Spam features -- and probably decreased the accuracy. I've since decided to train only on Spam, not on inappropriate mails; and let Gnus keep inappropriate mails from my eyes. I've also waffled over time about whether or not to treat newsletters from Dr. Dobbs Journal or USENIX as Spam or not -- now my rule of thumb is that since I signed up for them at some point, they are not Spam -- though I don't feel guilty about letting mailagent squirrel them away mostly out of sight. A few tips about using mailtrainer:

10 January 2007

Erich Schubert: Linux distributions to be illegal in Germany?

The biggest party in Bavaria and part of the current german government, CSU, has announced that they want to make the "production and distribution of killer-games" illegal in germany. They claim it's because of a recent highschool shooting (no fatalities except the shooter; there was one other highschool shooting in germany a few years ago with 16 fatalities. Yes, that is 17 fatalities in over 40 years, the 42 years old amok runner back then didn't play killer games, I guess, although he used a flame thrower...). This would make the production and distribution in Germany unlawful, since we include games such as overkill, tremulous and openarena. Other distributions such as SuSE probably include some games that classify as "killer games" as well. Also note that "distribution" is a rather wide term. Does it apply to web sites? ISPs? If there is an online killer game, is the ISP "distributing" it? Polticians (especially those from CSU/CDU/SPD) are such a bunch of idiots. Please do NOT ever again vote for CSU. Software patents, killer games, "Problemfrauen & Schadbären", surveillance cameras and other surveillance, military inside germany (note that in desaster cases such as the flooding some years ago we could already deploy soldats without extra laws), shooting down of civilian airplanes, transrapid, reducing math and natural sciences in highschools, "google earth threatens worldcup security", police installing trojans on home PCs (because criminals don't install virus scanners?), kicking "irritating" students out of school, study fees, ... Often they make George W. Bush look like a genius, don't they? They don't do any important stuff, but just say whatever a majority people might want to hear. They waste our money (Transrapid). They destroy our biggest resource (changes in school system and university system, which will reduce average knowledge in germany a lot). They do lots of stupid things, which heavily invade privacy or public safety (military, surveillance) with no benefits in safety and many of which even where only stopped because they violated our constitution. Do not vote for them! [Updates: Daniel, sorry for posting a non-english post to planet. I again put it into the wrong category; I intended to do a longer post (this one) in english later on. For clarification: CSU is the sister party of CDU, these parties have an arrangement not to compete by region and are closely affiliated. On the nation-wide level they have a joint list, which often is called CDUCSU. So they're not just an "affiliate", but a strong part of the ruling party. Holger Levsen pointed out we are likely going to violate some laws much earlier by distributing ("hacker-") tools such as nmap, ettercap or whireshark.]

Daniel Burrows: The US isn't the only country with crazy politicians

[UPDATE 2007-10-01 22:43: It appears that Erich wrote an expanded and better-translated post while I was at work]

Erich Schubert writes that
Nach aktuellen Pl nen der CSU w re der Vertrieb von Debian GNU/Linux illegal.

Auch zahlreiche andere Distributionen w hren wohl davon betroffen: Alle, die sogenannte "Killerspiele" vertreiben...

For non-German speakers, I provide the following approximate translation:
CSU wants to make Debian illegal

The current plans of the CSU [Christian Social Union] will make the distribution of Debian GNU/Linux illegal.

In addition, many other distributions will be affected: specifically, those that distribute so-called "killer games"...

For instance, Overkill (Debian also includes, e.g., Tremulous and OpenArena, which are based on the Quake3 engine and have 3d graphics...)

(hopefully I got the gist of Erich's post without mangling it too badly!)

What I don't know is how serious this is. The CDU (Christian Democratic Union) is IIRC the current ruling party in Germany; however, according to Wikipedia, the CSU is some sort of local affiliate in Bavaria. Does it have the clout to get this proposal made into law?

9 January 2007

Michael Janssen: Learned Bad Ideas

There are some things that I have learned over the years as bad practice. There are some that I have learned through reading and contemplation, and a little experimentation. One of the categories of bad things that I have learned is visual aids for statistics. There is basically one cardinal rule for data presented in a graphical form:
Let the user see the actual data.
It seems kindof obvious, doesn't it? I'm presenting data, you should be able to see it. If you want to show me that there are 450 widgets, and 230 whazits, and only 210 whositz, then you should show those. The simplest way of doing this is by just listing them out in a table:
TypeAmount
Widgets450
Whazits230
Whositz210
That's simple, isn't it? However, it doesn't help the viewer any. They're pretty good if you want to know exactly how many of something there is, but not great at other things that you want to do with data. How do you find the minimum (or maximum) of all of the values? You scan, remembering the highest one you saw so far, and then go back to that one and find out. If the goal is to somehow compare, the viewer will be helped out by using a graphical display of some sort. One thing that humans are good at estimating is length. Quick, tell me what we have the least of: Bar graph.  If you're reading this with a text-only browser, this post won't be as interesting. Sorry, lesser-sighted folks.
That was easy, wasn't it. Bar charts are good - they can still be abused by people in evil ways, but most of the time they're pretty useful.

All of this is working up to pie charts. What's smaller, Whazits or Whositz here:
Pie graph.  Text-only people can breathe easy, they can't see the evil.
Is Widgets half or more than half of the pie? Pie charts are the bad seed of the graph world. They aren't very useful, hang out a lot, and don't help you much. The worst thing about pie charts is that they aren't even good at the thing they're supposed to be the best at: comparing relative sizes. Consider thanksgiving, or christmas, or whatever large gathering you prefer where there is pie. Lots of pie slices are laid out on the table. Which one is the biggest? It's hard to tell. It's even hard to cut even slices from a normal pie. Sure, you can cut in 4ths, 6ths, or 8ths, but I challenge you to cut a pie in 5 pieces equally - it's really hard! It's because humans aren't great at judging the differences between angles. They're only really good for seeing one thing: the largest piece in the pie. Even then, it's sometimes questionable - if you looked at a evenly sliced pie, could you tell the difference between a 23% piece from a 20% piece? The bad thing about this is that they are ingrained into our culture. There wasn't a finance report in the last 10 years without a pie chart.

All of this ranting was basically brought to you by a recent post on BoingBoing. This post features the most horrible monstrosity I have seen in a long time: the hierarchial pie chart. Feast your eyes on this monstrosity. I challenge any reader to tell me: what is the smallest third-level category? What is the largest?

8 January 2007

Evan Prodromou: 18 Niv se CCXV

Our brunch yesterday went super-well. We had about 15 people over, which was far too many for our supposedly mellow evening. But it was great -- we got to catch up with a lot of folks I've been missing over the holidays and haven't gotten to see. I hadn't seen Lucinda Catchlove much for a few months, and Heather Kelley and Damian DiFede made an appearance, too. We got to see John Usher, who's off to do weird things to people's ears at the Consumer Electronics Show in Las Vegas. Niko came with friend Johanie, who made some great black-bean brownies; DJ Ghostbeard was still feeling ill from NYE; and Meg showed off her deep purple hockey bruises. It was also nice to have some new friends come, too. I've been wanting to hang out with MC Turgeon, one of the most experienced bloggers in Quebec, founder of me dia and toiles-du-Web, since we met last year at South by Southwest. She and her partner Chris came over with a nice bottle of wine and got to know everyone there. Actually, she knew a lot of the people already -- she and Anne-Marie had met at Mutek, for example. When Patti came in, she said, "Oh, yeah, I know M-C. We do a lot of panels together." Ah, the panel circuit. Altogether a really good time. Probably the only downside is that our oven was really dirty, and while I was making quiches and roasted potatoes I smoked up the entire house. But it cleared pretty quickly, and with the nice weather we're having, I could even have the kitchen door open to clear the smoke. Unheard of! tags:

Making a point Speaking of unseasonably warm weather, the Green Party of Canada held a picnic and ice cream social on Mount Royal last Saturday, which would have a couple of feet of snow on it in a regular year. The subject of discussion? Climate change, of course. I dunno -- clever move, but do they have to rub it in? tags:

Shameless plugs My friend Allegra has started a great new blog, Sucking-stones. She's got a really good voice and it's now in circulation as one of the blogs I read each morning. Also, my friend Anne-Marie Bergeron has her first travel TV show on 33mag tomorrow. She's got a four-part series on her trip to Iceland which they're going to show. It's in French, but her husband Jeff only cusses in English, so that's probably what you'll hear when they're climbing glaciers and lava floes. Did that get everyone? I hope so. You know what they say: You can plug your friends, and you can plug your ears, but you can't plug your friends' ears. Or something. tags:

Chix I was just reading Angela Beesley's end-of-year blog post and realized that I'd missed the establishment of WikiChix. It's a new organization for female wiki participants, in the great tradition *Chix tradition of Linux Chix or Debian Women. All the more galling is that two other blogs I regularly read also covered the new group. Joseph Reagle made a note about gendered spaces and the role they play in any movement (like Open Content and wikis). Biella blogged about his bloggage and put some social-science twist that was a little hard for me to follow. In any event, good luck to WikiChix. I find clubs extremely fun and useful and by nature exclusionary, and nothing to get worked up about. Of course, as one of the two founding members of the Maldekstrulaj Vegeteranaj Esperantistaj Linuxamantoj (Left-handed Vegetarian Esperantist Linux-users Club), I'd probably be pretty hypocritical calling anyone else exclusionary. tags:

Pierre Habouzit: POSIX DNS APIs considered harmful.

I'm quite upset with the POSIX APIs. I'm trying to write some kind of simple piece of software, to be included in a mail chain, doing a lot of DNS queries, mostly to RBLs. I've met two kind of problems:
  1. the POSIX API is synchronous, and absolutely no usual API for non blocking requests exists, there is of course:
    • adns, but it's very bloated, does not work with ipv6,
    • c-ares, but it's quite bloated, does not work with ipv6, and need select (cannot be used with epoll e.g.),
    • libdnsresolv (hahahaha, that's an horrible patch over the BSD resolving API, it's horrible, yuck),
    • udns, that's the best of the four, but still, the code do not feels very well, solid (do not check recvfrom/sendto errors e.g.).
  2. there is no way to escape the system resolver, which is good for usual applications, but very bad when it come to RBLs.
The second point is in fact the worse. On mail queues I administrate, there is also very often a bind that is used as a caching DNS resolver. Very well, except that it complectely sucks with RBLs. RBL generate a lot of queries that resolve and that you will almost never ask again before its TTL expires, and the other kind of queries you do get NOT FOUND answers, that are usually not cached. Too bad, the most useful answers are the NOT FOUND ones, and the cached answers are just here to make our local cache use huge amounts of memory for nothing. So RBLs just end up poisonning your system efficiency. That's quite ridiculous. I'd really love to have a decent async resolver API, and a way to tell my DNS cache (here BIND) that the domains serves a RBL, and that I (for at the same time the sake of the RBL and from my system) want specific caching features for it. I fear that it wont be possible, and that I will end up writing some API specifically designed to craft DNS queries to the RBL servers (once again there is absolutely no reason to use forwarders for that, as it will end up by screwing your forwarder cache for almost no gain, and won't save you from a lot of useless queries with no answers at all). The good point is that the kind of queries you need to do to RBL servers is indeed completely trivial: Then implementing on top of that some efficient caching (RBL-aware) will be rocking fast, and way more efficient than bind caching, without the poisonning effects. A big gain I hope. What suprises me the most, is that I've not found anybody speaking about those side effects of RBLs on nameservers, almost no discussion at all. Very surprising for a protocol that is so vital for the internet as a general rule.

6 January 2007

Martin F. Krafft: Credit card verification codes

There once was a credit card number tied to a name and when it was abused, the credit card company decided to tie it to an expiration date. It didn't take long until gifted hackers found out that twelve months and an average life cycle length of two years required them to check each credit card number against the 24 possible expiration date month-year pairs, so the credit card companies also required the billing address to match (at least fuzzily). Unfortunately, many of the companies accepting and storing credit card data are run by clueless people and the data sets of names, numbers, expiration dates, and billing addresses leaked, causing even more organised abuse. The credit card companies did not like that and proceeded to invest tiny amounts of additional energy and manpower into improving the status quo. So then, one day, in the dungeons of the credit card company offices, a bright young graduate from one of the American elite colleges yelled "heureka" [*] and outlined how to secure credit cards with the "credit card verification code" (CVC), printed on the back of each card printed, not stamped like the number. Maybe his/her theory was that the stamped number could be read from both sides, so a printed CVC would only be readable from one side of the card, bringing instantaneous 200% security increase. Maybe it was some other theory. In any case, the companies readily adopted the CVC and by today, every web form will ask you to also enter the CVC. They'll probably store and leak it too. What's next? Credit card colour?
[*]"Heureka" is Greek for "this bath is too hot" [Dr. Who].
NP: OSI / Free Update: Andrew Pollock points me to the Wikipedia article on CVC as good reading. Indeed, the article explains security benefits in face-to-face transactions and in the socalled "card not present" situations, like ordering over the Internet. Apparently the CVC must not be stored, so in some ways they have learnt from the past. I know of at least one merchant in the US who stores the CVC in my profile though. Update: Mark Brown adds: "The CVC is being used because it defeats attacks like taking a photo of a card with a mobile phone: it makes it usefully harder (not impossible) for people to transparently clone cards." This squares with the Wikipedia article.

5 January 2007

Russell Coker: core files

The issue of core file management has come up for discussion again in the SE Linux list.

I believe that there are two essential security requirements for managing core files, one is that the complete security context of the crashing process is stored (to the greatest possible extent), and the other is that processes with different security contexts be prevented from discovering that a process dumped core (when attacking a daemon it would be helpful to know when you made one of it's processes dump core).

The core file will have the same UID and GID as the process that crashed. It's impossible to maintain the complete security context of the crashing process in this manner as Unix permissions support multiple supplementary groups and Unix filesystems only support one GID. So the supplementary groups are lost.

There is also a sysctl kernel.core_pattern which specifies the name of the core file. This supports a number of modifiers, EG the value "core.%p.%u.%g" would give a file named "core.PID.UID.GID". It would be good to have a modification to the kernel code in question to allow the SE Linux context to be included in this (maybe %z).

To preserve the SE Linux context of the crashing process with current kernel code we need to have a unique type for each process that dumps core, this merely requires that each domain have an automatic transition rule for creating files in the directory chosen for core dumps. In the default configuration we have core files dumped in the current directory of the process. This may be /tmp or some other common location which allows an attacker to discover which process is dumping core (due to the directory being world readable) and in the case of SE Linux there may be multiple domains that are permitted to create files in /tmp with the same context which gets in the way of using such a common directory for core files.

The traditional Unix functionality is to have core files dumped in the current directory. Obviously we can't break this by default. But for systems where security is desired I believe that the correct thing to do is to use a directory such as /var/core for core files, this can be easily achieved by creating the directory as mode 1733 (so that any user can create core files but no-one but the sys-admin can read them) and then setting the core_pattern sysctl to specify that all core files go in that directory. The next improvement is to have a poly-instantiated directory for /var/core such that each login user has their own version. That way the user in question could see the core files created by their own processes while system core files and core files for other users would be in different directories. Poly-instantation is easier to implement for core files than it is for /tmp (and the other directories for which it is desirable) because there is much less access to such a directory. When things operate correctly core files are not generated, and users never need to access each other's core files directly (they are mode 0600 so this isn't possible anyway).

This area will require a moderate amount of coding before it works in the ideal manner. I aim to briefly describe the issues only in this post.

Benjamin Mako Hill: Who Owns <em>Free Culture</em>?

The previous year saw far too much fighting over who gets to define and control the term free culture. The biggest problem, in my opinion, is that these fights conflate a very important discussion about the goals of a set of nascent social movements -- or the lack thereof -- with much less important issues of semantics, definitions, and control over terms. The term is being used in a way that describes a whole lot of projects I support and participate in fully -- and a few I don't. And I think that's OK. When Erik M ller and I launched the Free Cultural Works Definition (at the time, the Free Content and Expression Definition), we struggled to find a good term for the works that we wanted to liberate. We thought about using the terms content, expression, knowledge, information, art, data and communication but each word seemed to exclude an important body of works or producers. Few musicians we knew thought of their productions as "content" while few encyclopedia writers did not. The term we liked most was culture: it defines a very broad set of practice and has very positive connotations. Of course, others had already been using the term free culture so we spent some reading up on the term and talking to the people most closely associated with it. Originally, the term seems to have its roots in the book Free Culture by Lawrence Lessig. I reread the book to get an idea for exactly what Lessig meant when he used the term but, upon reaching the end, I found myself without a good answer. The book's index included a promising entry for "Free culture, defined" which pointed to a short section in the preface:
A free culture is not a culture without property; it is not a culture in which artists don't get paid. A culture without property, or in which creators can't get paid is anarchy, not freedom. Anarchy is not what I advance here. Instead, the free culture that I defend in this book is a balance between anarchy and control. (emphasis mine)
Framed by a negative definition of what free culture is not, Lessig's definition describes the broad space between two unattainable extremes. This resulting ambiguity is fully intended: Lessig has not only spoken out against my particular suggestion but against any definition and the process of offering ex cathedra definitions or goals altogether [1]. In personal conversations about our definition, Lessig was initially very supportive. In fact, it was Lessig who introduced Erik and I to each other and suggested that we work together. What Lessig did disagree with us on however, was calling the definition the Free Culture Definition. I think that Lessig felt some sense of ownership of the term and felt that he and others had defined it and been using it in a way that was broader and incompatible with the definition we were proposing and with any definition of the type of we were suggesting. Early on, Lessig blessed a group of students to create a Free Culture student movement. Most active now in Harvard Free Culture and Free Culture NYU and but in a handful of other places as well, these groups have been involved in everything from the promotion of transgressive approaches to IP, to speech bubbles, to anti-DRM work, to protecting the right of cereal restaurants to operate. When Erik and I suggested to this group that they might benefit from adopting the Free Cultural Works Definition as a set of explicit ideals or goals for their movement, the larger part of the coalition soundly rejected the idea. Like Lessig, they wanted free culture to refer to wide variety of projects and did not feel good about describing any work by sympathetic parties as "non-ideal." Erik and I were faced with two choices: we could call our definition the Free Culture Definition and in effect engage in a power struggle with Lessig and with some portion of the free culture student movement or we could pick another term. While we don't like the alternatives as much as free culture, we didn't have a lot of trouble deciding that going with a term like free cultural work or free content and expression was the better choice. This is why I am a little worried about the recently announced UK-based Free Culture Foundation. I have nothing but respect for the founders (Matt Lee, Tom Chance, and Rob Myers) and trust them to create the type of free culture organization that I would like to see. I am very much looking forward to working closely with them on this project in the future. They seem likely to choose a set of goals and adopt a set of strategies in line with the ones I've argued for. But in that my goals and strategies have run into opposition among many of the most visible people using the term free culture in the past, that's also why I'm a little worried. SJ Klein and I were recently saying that its time to start naming organizations and projects in this area using only words in dead languages. That way, we can side-step the (unimportant) semantic arguments over who gets to control existing terms and focus on the real goal of building stronger social movements, setting goals that sound as unthreatening to each other as they actually are, and building better tools. Without semantic arguments in our way, we'll be able stronger to build coalitions and work together in all the ways we should be.
[1]

This is an important distinction because it is also possible to disagree with the first fully articulated definition but also feel that offering another set of goals -- for example, a set that allowed for commercial use or anything under current CC licenses -- was productive.

After all, I have been arguing for much longer in favor of any set of goals much longer and more strenuously than I have argued for any particular set of goals and I still feel that a set goals is much more important than any paritcular one.

Next.

Previous.