Indymedia Centre Tech recently reviewed available CAPTCHAs and alternatives. Please, if your blog uses these images:
I spent the weekend finding discovering several new ways to hate SCO Unixware. Almost everything on these systems is a symlink. Even such files as /etc/passwd:Debian$ ls -l /etc/passwd lrwxrwxrwx 1 root root 38 Jan 27 13:44 /etc/passwd -> /var/opt/K/SCO/Unix/5.0.7Hw/etc/passwdNeedless to say if you manage to screw up these symbolic links then fixing it is painful. (At one point I resorted to using tar to archive a working system’s directories, then unpacking it over a broken installation.) So a busy weekend full of being in the office.
In happier news I played around with webcam stuff, after somebody posted a nice guide to using Motion. I now have a webcam upon my desk which records video when something moves in front of it, and otherwise does nothing. A few test movies seem to work nicely (no audio) and I shared one with a friend - although no feedback yet. (I need to make sure the codec is playable upon Windows ..) I also updated the polls upon the Debian Admin site, so they contain date information about when they were posted. Feel free to suggest a new poll topic! The final thing I did over the weekend was put the finishing touches towards a new release of xen-tools. This contains a minor fix for networking under Xen 3.0.1.
 |
 |
| Old | New |
Jan 31 10:29:22 localhost kernel: hda: drive_cmd: status=0x51 DriveReady SeekComplete Error
Jan 31 10:29:22 localhost kernel: hda: drive_cmd: error=0x04 DriveStatusError
Beagle 0.2.1 is out. Beagle is a very useful desktop search engine for GNOME (but you can use it with KDE, too, or even without any desktop environment like I do). Despite the low version number it is already quite stable and usable, and has lots of features.
It can index all kinds of files and information on your computer (txt, pdf, doc, emails, IM logs, IRC logs, source code, images, music, and whatnot) and provides a very nice (new) search interface (see screenshot).
Yes, Google Desktop Search and Spotlight do the same thing, but neither is available for Linux, and neither of them is Free Software. They can "phone home" without telling you and do other funny things. With Beagle you can easily check what it does (use the source, Luke!).
Oh, and Beagle can now also parse Ruby files, a simple filter I had written and submitted has been included in the latest release (well, actually it was there in 0.2.0 already, but I didn't tell anybody ;-)).
$ time wc -l testfile 7798137 testfile real 0m12.127s user 0m0.765s sys 0m0.399sThen the most idiomatic program. Honestly, I expect it to suck performance-wise:
main = getContents >>= print . length . linesAnd it does:
$ time ./wcl1.hs < testfile 7798137 real 1m14.993s user 0m54.182s sys 0m0.573sBut only by a factor of six. My first hypothesis is that garbage collection is the culprit. And, indeed:
$ time ./wcl1.hs +RTS -sstderr -RTS < testfile ./wcl1.hs +RTS -sstderr 7798137 25,236,174,340 bytes allocated in the heap 5,070,193,924 bytes copied during GC 117,484 bytes maximum residency (3750 sample(s)) 96274 collections in generation 0 ( 26.36s) 3750 collections in generation 1 ( 0.74s) 2 Mb total memory in use INIT time 0.00s ( 0.00s elapsed) MUT time 27.17s ( 42.72s elapsed) GC time 27.10s ( 33.72s elapsed) EXIT time 0.00s ( 0.00s elapsed) Total time 54.27s ( 76.44s elapsed) %GC time 49.9% (44.1% elapsed) Alloc rate 928,824,966 bytes per MUT second Productivity 50.1% of total user, 35.5% of total elapsed real 1m16.616s user 0m54.278s sys 0m0.700sAs you can see, GC takes half of the processing time. Unfortunately, minimizing GC time does not have as big an effect as I hoped:
$ time ./wcl1.hs +RTS -H100M -RTS < testfile 7798137 real 1m1.354s user 0m44.371s sys 0m1.171sHere, I’m effectively telling it to use a 100-megabyte heap. A separate GC profiling run tells me that GC time has dropped from 50 % to 1 %. The memory problem seems to be a red herring. What we’re doing here is creating four separately allocated memory cells for each character in the input. Of course, the cells are generated lazily, that is, a cell pair is allocated when the lines function demands another character; it then creates another cell pair to denote the same character in a list of lines. As soon as each line has been counted, the four cells become garbage. This sounds like a lot of work! But apparently it isn’t; I tried a few variations that did away with the stream of input altogether, not much unlike what Daniel did in his tests. Surprisingly, the performance worsened a lot. The lesson: functional programming language implementations, unlike typical imperative ones, are built on the assumption that you’ll be doing a lot of allocation. What I suspect is taking a lot of the time is arithmetic. Haskell’s Int is boxed, and lazy by default, adding a lot of overhead. I hand-fused length and line, yielding a tail-recursive countLines function, and I added the command-line switch -funbox-strict-fields, which enables a few significant optimizations of the laziness of Int. The results were nice:
ajk@kukkamaljakko:~/scratch$ cat wcl4.hs
module Main (main) where
main :: IO ()
main = getContents >>= print . countLines 0
countLines :: Int -> String -> Int
countLines n ('n':r) = countLines (n + 1) r
countLines n (_:r) = countLines n r
countLines n [] = n
ajk@kukkamaljakko:~/scratch$ ghc -O -Wall -funbox-strict-fields --make wcl4.hs -o wcl4
Chasing modules from: wcl4.hs
Compiling Main ( wcl4.hs, wcl4.o )
Linking ...
ajk@kukkamaljakko:~/scratch$ time ./wcl4 < testfile
7798137
real 0m37.961s
user 0m29.853s
sys 0m0.495s
ajk@kukkamaljakko:~/scratch$ time ./wcl4 +RTS -H100M < testfile
7798137
real 0m25.153s
user 0m13.497s
sys 0m0.803s
ajk@kukkamaljakko:~/scratch$
With a large heap, the program is only twice as slow as the baseline. This is pretty good, considering that the code is still idiomatically functional.
The following rather ugly, imperative program does away with the lazy stream and carefully avoids any laziness without doing any too dirty tricks:
ajk@kukkamaljakko:~/scratch$ cat wcl7.hs
module Main (main) where
import Data.Word
import Data.Array.IO
import IO (stdin)
count :: Int -> (Int, Int) -> IOUArray Int Word8 -> IO Int
count 0 _ _ False = undefined -- force ac evaluation
count ac (l,u) _ l > u = return ac
count ac (l,u) ar
= do a < - readArray ar l
let ac' = case a of 10 -> ac + 1
_ -> ac
count ac’ (l+1, u) ar
maxn :: Int
maxn = 4096
main :: IO ()
main = do
arr < - newArray (0,maxn) 0
let loop :: Int -> IO Int
loop acc = do n < - hGetArray stdin arr maxn
if n == 0
then return acc
else do acc' <- count acc (0,n) arr
loop acc'
res <- loop 0
print res
ajk@kukkamaljakko:~/scratch$ ghc -O -Wall -funbox-strict-fields --make wcl7.hs -o wcl7
Chasing modules from: wcl7.hs
Compiling Main ( wcl7.hs, wcl7.o )
Linking ...
ajk@kukkamaljakko:~/scratch$ time ./wcl7 < testfile
7798137
real 0m13.431s
user 0m10.342s
sys 0m0.467s
Of course, this program is not Haskell 98, but the nonstandard things it uses are fairly common. As can be seen, this version’s performance is, considering the margin of error, equal to the baseline.
The compiler used in this test is The Glorious Glasgow Haskell Compilation System, version 6.4.1, as packaged in Debian sid.
"I think it's insane to require people to make their private signing keys available, for example. I wouldn't do it. So I don't think the GPL v3 conversion is going to happen for the kernel, since I personally don't want to convert any of my code."- Linus Torvalds, GPL V3 and Linux, 25 Jan 2006 <p class="silly">I wonder if he'll consider GPLv4 instead?</p>
tretkowski@kyllikki:~/repos/pkg-ion3% darcs pull
Password:
Password:
Password:
Password:
Password:
Pulling from "rollcage.inittab.de:repos/pkg-ion3"...
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Fri Jan 27 13:50:07 CET 2006 Norbert Tretkowski <norbert@tretkowski.de>
* Copy ion3-dev lintian override file.
Shall I pull this patch? (1/1) [ynWvpxqadjk], or ? for help: y
Finished pulling and applying.
tretkowski@kyllikki:~/repos/pkg-ion3% darcs pull
Password:
Password:
Password:
Password:
Password:
Pulling from "rollcage.inittab.de:repos/pkg-ion3"...
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Password:
Fri Jan 27 13:50:07 CET 2006 Norbert Tretkowski <norbert@tretkowski.de>
* Copy ion3-dev lintian override file.
Shall I pull this patch? (1/1) [ynWvpxqadjk], or ? for help: y
Finished pulling and applying.
early bird rebatefor flights. At least from germany it was the cheapest flight I could find, but it's only a couple of more days (I think till end of January). So if you didn't booked yet your flights, this might be a good time to start searching for flights ;) BTW: We'll fly with BA0903 from FRA to LHR and with BA0243 from LHR to MEX at the 10th of May. We'll fly back on the 21st of May with BA0242 and BA0910.
Next.