Search Results: "ed"

22 September 2011

Biella Coleman: Congressional Quarterly on Hackers

When I was in graduate school, there were two publications that when I found out about them, they changed my life, for the better. One is the Annual Review of Anthropology (and there are many other Annual Reviews) and Congressional Quarterly Researcher. Both, albeit in very different ways, provide clear and crisp road maps and resources related to the topic under discussion. I had the pleasure of writing an ARA on digital media and I had always hoped that there was a Congressional Quarterly on hackers and hacking. Well now there is one and I have to say, they have done a fantastic job. I am not surprised. They tend to be pretty matter of factual and present many sides to an issue. The author, Marcia Clemmitt, did just that, covering open source, transgressive hacking, Anonymous, and cybersecurity among many other issues. Out of all the recent publicly oriented publications on hackers and hacking, this is the one that is the most in-depth but accessible. The only hitch is that the article is behind a pay wall but most universities subscribe to them and you can order an individual copy. update: I forgot to mention that there is an excellent segment in the 24 page report by Daniel Flower, which covers the history and significance of the protests. It is compact but the best account I have seen.

Lucas Nussbaum: Windows dynamic disks and debian installer ?

Windows has a feature called Dynamic Disks, which makes it ignore the DOS partition table and manage partitions its own ways. gparted is supposed to detect that, and warn the user that it can t deal with that. The solution is to reboot in Windows, switch the partition back to Basic disks , and then proceed with installing Debian (or Ubuntu). However, a student of mine had an interesting experience because gparted did not detect the dynamic disk stuff and did not warn about it, so the student went ahead with installing Debian squeeze. After reboot in Windows, Windows sees the partition as invalid . The same happens with Ubuntu 11.04 (not really surprising). I m surprised I did not find much information about that issue. Does someone has more info ? The student was using two disks in his laptop, and only the second one (not the one used to boot windows) was using dynamic disks .

Christian Perrier: 3rd update about Debian Installer localization

Yet another progress report about D-I l10n. Full stats are here Status for D-I level 1 (core D-I files):

15 languages 100% for all sublevels: bg cs de eo es fr it ja ko pt ru si ta th zh_CN
3 languages 96%: eu nl tr
5 languages 95%: be bn gu sk uk
6 langauges 94%: ast bs fa ga ro sv
4 languages 93%: hi hu lo nb
2 languages 92%: id sl
5 languages 91%: et mr pa vi zh_TW
others are 90% or below

New complete languages since last update: Esperanto, Spanish, Tamil, Simplified Chinese. Status for D-I level 2 (packages that have localized material that may appear during default installs, such as iso-codes, tasksel, etc.):

3 languages 100%: de fr pt
16 languages 99%: ca cs da eo fa id it ja kk nl ru sk sl th uk zh_CN (mostly missing ONE string in espeakup)
7 languages 98%: be bg es eu pt_BR ro sv
3 langauges 95%: el he nb
8 languages 94%: ar ast dz fi gl hr vi zh_TW
1 language 93%: is
9 language 92%: bn bs ga hu ko ne pl si tr
9 languages 91%: gu ka km lt mk mr sr ta te
others are 90% or below

Status for D-I level 3 (packages that have localized material that may appear during non-default installs, such as win32-loader)

22 languages 100%: be bg bs ca cs de el eo fi fr ga gl he is ja pt ru sk sv th tr zh_CN
2 languages 98%: hu uk
others are 90% or below

Christian Perrier: R.I.P. Troy Davis

Just like Saudi Arabia, China and a few other democratic countries, the state of Georgia which is part of United States of America still murders its citizens. Troy Davis has just been murdered by lethal injection 27 minutes ago. I'm not always proud to live in the country I'm living in, but today I am. Even if we were the last country in EU to abandon death penalty, on october 9th 1981, we can't legally murder our citizens for nearly 30 years now. I hope that M. Troy Davis now rests in peace. My only hope is that the noise made around this will help all those people who daily fight against deatch penalty in USA. Don't abandon hope. Full abolition will happen.

Axel Beckert: Emacs Macros: Repeat on Steroids

vi users have their . (dot) redo command for repeating the last command. The article Repeating Commands in Emacs in Mickey Petersen s blog Mastering Emacs explained Emacs equivalent for that, namely the command repeat, by default bound to C-x z. I though seldomly use it as I mostly have to repeat a chain of commands. What I use are so called Keyboard Macros. For example for the CVE-2011-3192 vulnerability in Apache I added a line like Include /etc/apache2/sites-common/CVE-2011-3192.conf to all VirtualHosts. So I started Emacs with all the relevant files:

grep
CVE-2011-3192 -l /etc/apache2/sites-available/*[^~]   xargs emacs
&

To remove those Include lines again M-x flush-lines is probably the easiest way in Emacs. So for every file I had to call flush-lines with always the same parameter, save the buffer and then close the file or in Emacsish kill the buffer. So while working on the first file I recorded my doing as a keyboard macro:

C-x (: Start recording
M-x flush-lines<Enter>CVE-2011-3192<Enter>: flush all lines which contain the string CVE-2011-3192
C-x C-s: save the current buffer
C-x C-k<Enter>: kill the current buffer, i.e. close the file
C-x ): Stop recording

Then I just had to call the saved macro with C-x e. It flushed all lines, saved the changes and switched to the next remaining file by closing the current file with three key-strokes. And to make it even easier, from the second occasion on I only had to press e to call the macro directly again. So I just pressed e for a bunch of time and had all files edited. (In this case I used git diff afterwards to check that I didn t wreck anything by half-automating my editing. :-) Of course there are other ways to do this, too, e.g. use sed or so, but I still think it s a neat example for showing the power of keyboard macros in Emacs. More things you can do with Emacs Keyboard Macros are described in the EmacsWiki entry Keyboard Macros. And if you still miss vi s . command in Emacs, you can use the dot-mode, an Emacs mode currently maintained by Robert Wyrick which more or less automatically defines keyboard macros and lets you call them with C-..

21 September 2011

Dirk Eddelbuettel: digest 0.5.1

A new version of the digest package (which generates hash function summaries for arbitrary (and possibly nested) R objects using any of the standard md5, sha-1, sha-256 or crc32 algorithms) is now on CRAN. Bryan Lewis, who is doing very stuff with websockets and Html5 as he demoed last week the Chicago R User Group meeting, needed raw support and was kind enough to send a patch. After some more testing, this change set is now in the new version which went onto CRAN earlier today. CRANberries provides the usual summary of changes to version 0.5.0. As usual, our package is available via the R-Forge page leading to svn and tarball access, my digest page and the local directory here.

Sune Vuorela: Missing tools for people looking for jobs

I have mentioned it before in some blog posts that I m out looking for a job. In my job searching process, there is a couple of nice tools that I am missing. Maybe if I describe one of them, someone will say that it already exists over there -> or something. Tool 1. Tracking of data related to a job application.
For each such application I send I have a set of data, some pieces only for some applications:

Contact data for company
The open position I m applying for on the web
The web form I submitted a application thru
The email I used to submit my application
Further emailing with the company
Further chat logs (jabber or irc) with the company
My jotting down during or after a (phone) interview
Other data I ve gathered about the company

I would like one simple way of browsing all this information for a given position. Does this exist? preferably something that can interact with Konqueror, KMail, KAddressbook, KJots, Kopete and irssi on a remote machine. Tool 2: Tracking of tasks with states There is at least the following steps in a job searching process:

Interesting company found
Initial contact
More information sent
Phone interview
First interview
Second interview
First draft contract
Signed contract

A job application might go thru some or all of these steps or even more steps, and from each step it can end prematurely with a rejection . Here, I m missing a tool to track my applications and in which state they are currently in. My first thought here is a bug tracking system, but I m wondering if there is something nicer. Currently I m tracking it manually in a KJots notebook. Comments are open. And no, I haven t yet found a job.

Ian Wienand: Exploring $ORIGIN

Anyone who works with building something with a few libraries will quickly become familiar with rpath's; that is the ability to store a path inside a binary for finding dependencies at runtime. Slightly less well known is that the dynamic linker provides some options for this path specification; one in particular is a path with the special variable $ORIGIN. ld.so man page has the following to say about the an $ORIGIN appearing in the rpath:

  ld.so understands the string $ORIGIN (or equivalently $ ORIGIN )
  in  an  rpath specification to mean the directory containing the
  application  executable.  Thus,  an   application   located   in
  somedir/app   could   be  compiled  with  gcc  -Wl,-rpath,'$ORI 
  GIN/../lib' so that it finds an  associated  shared  library  in
  somedir/lib  no matter where somedir is located in the directory
  hierarchy.

As a way of a small example, consider the following:

$ cat Makefile
main: main.c lib/libfoo.so
	gcc -g -Wall -o main -Wl,-rpath='$$ORIGIN/lib' -L ./lib -lfoo main.c
lib/libfoo.so: lib/foo.c
	gcc -g -Wall -fPIC -shared -o lib/libfoo.so lib/foo.c
$ cat main.c
void function(void);
int main(void)
 
   function();
   return 0;
 
$ cat lib/foo.c
#include <stdio.h>
void function(void)
 
   printf("called!\n");
 
$ make
gcc -g -Wall -fPIC -shared -o lib/libfoo.so lib/foo.c
gcc -g -Wall -o main -Wl,-rpath='$ORIGIN/lib' -L ./lib -lfoo main.c
$ ./main
called!

Now, wherever you should choose to locate main, as long as the library is in the right relative location it will be found correctly. If you are wondering how this works, examining the back-trace from the linker function that expands it is helpful:

$ gdb ./main
(gdb) break _dl_get_origin
Function "_dl_get_origin" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (_dl_get_origin) pending.
(gdb) r
Starting program: /tmp/test-origin/main 
Breakpoint 1, _dl_get_origin () at ../sysdeps/unix/sysv/linux/dl-origin.c:37
37	../sysdeps/unix/sysv/linux/dl-origin.c: No such file or directory.
	in ../sysdeps/unix/sysv/linux/dl-origin.c
(gdb) bt
#0  _dl_get_origin () at ../sysdeps/unix/sysv/linux/dl-origin.c:37
#1  0x00007ffff7de61f4 in expand_dynamic_string_token (l=0x7ffff7ffe128, 
    s=<value optimized out>) at dl-load.c:331
#2  0x00007ffff7de62ea in decompose_rpath (sps=0x7ffff7ffe440, 
    rpath=0xffffffc0 <Address 0xffffffc0 out of bounds>, l=0x0, 
    what=0x7ffff7df780a "RPATH") at dl-load.c:554
#3  0x00007ffff7de7051 in _dl_init_paths (llp=0x0) at dl-load.c:722
#4  0x00007ffff7de2124 in dl_main (phdr=0x7ffff7ffe6b8, 
    phnum=<value optimized out>, user_entry=0x7ffff7ffeb28) at rtld.c:1391
#5  0x00007ffff7df3647 in _dl_sysdep_start (
    start_argptr=<value optimized out>, dl_main=0x7ffff7de14e0 <dl_main>)
    at ../elf/dl-sysdep.c:243
#6  0x00007ffff7de0423 in _dl_start_final (arg=0x7fffffffe7c0) at rtld.c:338
#7  _dl_start (arg=0x7fffffffe7c0) at rtld.c:564
#8  0x00007ffff7ddfaf8 in _start () from /lib64/ld-linux-x86-64.so.2

Just from a first look we can divine that this has found the RPATH header in the dynamic section and has decided to expand the string $ORIGIN.

$ readelf --dynamic ./main
Dynamic section at offset 0x7d8 contains 23 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libfoo.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/lib]
 0x000000000000000c (INIT)               0x4004d8
 0x000000000000000d (FINI)               0x4006f8

At this point it proceeds to do a readlink on /proc/self/exe to get the path to the currently running binary, effectively does a basename on it and replaces the value of $ORIGIN. Interestingly, if this should fail, it will fall back on the environment variable LD_ORIGIN_PATH for unknown reasons. This might be useful if you were in a bad situation where /proc was not mounted and you still had to run your binary, although you could probably achieve the same thing via the use of LD_LIBRARY_PATH just as well. This feature should be used with some caution to avoid turning your application in a mess that can't be packaged in any standard manner. One common example of use is probably your java virtual machine to find its implementation libraries.

$ readelf --dynamic /usr/lib/jvm/java-6-sun/bin/java
Dynamic section at offset 0x9a28 contains 25 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libjli.so]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [lib.so]
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/../lib/amd64/jli:$ORIGIN/../jre/lib/amd64/jli]
 ...

There is a small but interesting security issue. If an attacker can hardlink to your binary that is using $ORIGIN then any path expansion is now relative to the attackers directory. Hence it is possible to make the linker read in arbitrary libraries and hence run arbitrary code. Clearly if your program is running setuid as someone else, such as root, you've just given up the keys to the house! Luckily, the dynamic linker will not let you shoot yourself in the foot like this, and prevents expansion of the $ORIGIN field if the program is running setuid.

$ sudo chown root ./main
$ sudo chmod +s ./main
$ ./main
./main: error while loading shared libraries: libfoo.so: cannot open shared object file: No such file or directory

However, this is a good example of why you might want to keep users home and temp directories on separate file systems away from where binaries live. There is also a similar variable $PLATFORM, which describes the current running system. This is gained via the auxiliary vector, which I have written about here and here. Setting LD_SHOW_AUXV=1 will allow you to examine this value.

$ LD_SHOW_AUXV=1 ls
...
AT_PLATFORM:     x86_64
...

20 September 2011

Axel Beckert: Creative Toilet Paper Usage in Webcomics

Funnily two of my daily web comics recently featured interesting things you could do with toilet paper: Zits on 19th of September 2011 involving a fan and Calvin and Hobbes on 13th of September 2011 involving flushing the toilet. Although both experiments are obviously resource wasting, they look like quite some fun and I m tempted to actually try them both at least once. (I though don t plan to try this, too. :-)

Matthew Garrett: UEFI secure booting

Since there are probably going to be some questions about this in the near future:

The UEFI secure boot protocol is part of recent UEFI specification releases. It permits one or more signing keys to be installed into a system firmware. Once enabled, secure boot prevents executables or drivers from being loaded unless they're signed by one of these keys. Another set of keys (Pkek) permits communication between an OS and the firmware. An OS with a Pkek matching that installed in the firmware may add additional keys to the whitelist. Alternatively, it may add keys to a blacklist. Binaries signed with a blacklisted key will not load.

There is no centralised signing authority for these UEFI keys. If a vendor key is installed on a machine, the only way to get code signed with that key is to get the vendor to perform the signing. A machine may have several keys installed, but if you are unable to get any of them to sign your binary then it won't be installable.

This impacts both software and hardware vendors. An OS vendor cannot boot their software on a system unless it's signed with a key that's included in the system firmware. A hardware vendor cannot run their hardware inside the EFI environment unless their drivers are signed with a key that's included in the system firmware. If you install a new graphics card that either has unsigned drivers, or drivers that are signed with a key that's not in your system firmware, you'll get no graphics support in the firmware.

Microsoft requires that machines conforming to the Windows 8 logo program and running a client version of Windows 8 ship with secure boot enabled. The two alternatives here are for Windows to be signed with a Microsoft key and for the public part of that key to be included with all systems, or alternatively for each OEM to include their own key and sign the pre-installed versions of Windows. The second approach would make it impossible to run boxed copies of Windows on Windows logo hardware, and also impossible to install new versions of Windows unless your OEM provided a new signed copy. The former seems more likely.

A system that ships with only OEM and Microsoft keys will not boot a generic copy of Linux.

Now, obviously, we could provide signed versions of Linux. This poses several problems. Firstly, we'd need a non-GPL bootloader. Grub 2 is released under the GPLv3, which explicitly requires that we provide the signing keys. Grub is under GPLv2 which lacks the explicit requirement for keys, but it could be argued that the requirement for the scripts used to control compilation includes that. It's a grey area, and exploiting it would be a pretty good show of bad faith. Secondly, in the near future the design of the kernel will mean that the kernel itself is part of the bootloader. This means that kernels will also have to be signed. Making it impossible for users or developers to build their own kernels is not practical. Finally, if we self-sign, it's still necessary to get our keys included by ever OEM.

There's no indication that Microsoft will prevent vendors from providing firmware support for disabling this feature and running unsigned code. However, experience indicates that many firmware vendors and OEMs are interested in providing only the minimum of firmware functionality required for their market. It's almost certainly the case that some systems will ship with the option of disabling this. Equally, it's almost certainly the case that some systems won't.

It's probably not worth panicking yet. But it is worth being concerned.

comments

John Goerzen: Mexico Part 6: Conclusion

The sixth in a series; see also parts 1, 2, 3, 4, and 5. It s been about two weeks now since we got back home. Every few days, our boys still talk about Mexico. Jacob talks about what he will want to do when we go back and how he d like to see Jonathan over there again. Jacob, Oliver, and I look at photos from our trip a few times a week. This is all a sure sign that our boys loved the trip. And I keep trying to find Mexican food that tastes as good as the real thing (and, so far, failing). Another sign of a good trip! I think it is possible to have fun, relax, and enjoy new experiences all at the same time. We did that in Mexico. We had a lazy afternoon or two with the boys taking naps or playing with other children at the house we stayed at. And we stayed in a beautiful hotel without air conditioning, explored old downtown areas and ancient ruins, and bought things from crowded markets and people selling things from a table along a road. To anyone thinking about visiting: Go. Enjoy it, bring back memories, and live a little more serendipitously than usual. That s what I hope to do when we visit Greece before too long. Finally, here s a photo of the painting I bought from a roadside vendor for $17. Make sure to view it full screen. I think it says more about Mexico than 6 blog posts do.

Maximilian Attems: Recurring Maxima failures

Every once in a while I try out latest Debian/sid Maxima as it sees continuous development: Maxima git repo. Todays failure is an integral given at the Bloc course "Aspects of QCD at Finite Density". It is an exercise to calculate following simple integral that should just give a Bessel function:

(%i3) integrate(exp(z*cos(t))*cos(a*t), t, 0, %pi);
                           %pi
                          /
                          [               cos(t) z
(%o3)                     I    cos(a t) %e         dt
                          ]
                          /
                           0

As usual the result it returns is the integral itself.. :/ So yes indeed Maxima is nice for simple undergrads calculation: Maximum Calculus with Maxima, but unfortunately don't expect much for more complex problems. The result is the partition function of the chiral perturbation theory in the simple setup of equal quark masses and one quark flavour family. Sadly Integrals returning Bessel functions seem to regularly fail.

Kai Wasserb ch: TNEF or how to lock-up your e-mails

As the administrator for several MTAs I've received my fair share of support requests for "unreadable" e-mails. They generally have an attachment called winmail.dat. This is what all (for exceptions see the rest of this blog post) non-Outlook MUAs can display, when they receive an e-mail with the application/ms-tnef (or application/vnd.ms-tnef) MIME type. TNEF is a propietary format cooked up in Redmond. When chosen, the e-mail in question isn't sent as text/plain or at least text/html, but the entire e-mail is put into an TNEF-encoded attachment, most often called winmail.dat, in which you (most commonly) find the content of the e-mail in RTF, possible attachments are also put into the winmail.dat. The easieast solution for all involved parties is to convince the sender, that her Outlook and/or Exchange setup needs fixing (disabling TNEF in Outlook and Exchange 2003). Sadly this won't work all the time. The other solutions are to either filter the e-mails on your e-mail server and run them through something like ytnef_smtpd.py (please note, that this might be legally problematic for you as a service provider, opening an e-mail, even automatically, is a violation of communication privacy rights in many jurisdictions). Or, if the legal problem is too big or can't be solved through an local server installation (e.g. because there is no server running 24/7), you're left with either using Outlook (probably what Microsoft intended) or you can use a plug-in/add-on for your client. I've tested the LookOut add-on for icedove and it seems to work well. There are of course plug-ins for many other clients, but I can't honestly tell whether they work or not. [UPDATE 2011-09-20]: Fellow Debian Developer Josselin Mouette wrote me, that Evolution is capable of reading TNEF-encoded e-mails out of the box, but he also said that [TNEF e-mails should] be banned by all means.[/UPDATE 2011-09-20] Now, before I close, please always try to convince the sender/postmaster to fix their setup. It has only advantages: text/plain e-mails are smaller, they're less likely to contain malicious code (i.e. no parts hidden away from the virus scanner on the mail server (amavis can read TNEF e-mails, if libconvert-tnef-perl is installed)) and all recipients can open the e-mails with the client of their choice.

MJ Ray: Six of the Best Podcasts?

I m listening to quite a few podcasts recently. Here are my current favourites:

Cyberunions which covers the tech/work crossover space and is appearing as Ogg near the start of most weeks. I reviewed an earlier episode and it s kept on improving. It s under CC-BY-NC-SA, so you can share it, but it isn t free software (FOSS) itself. Thanks to John Atherton for the tip.
The Command Line is more tech, but with a leaning towards activism too. This podcast is actually FOSS sadly the only one of this round-up! It s recently dropped to once a week on Sundays, which is a shame in one way, but suits me better. I don t remember how I got started on this.
More or Less: Behind the Stats is about numbers in the news. Maybe my love of statistics influences me, but I think this is brilliant, puncturing the pompous politicians who try to mislead (figures don t lie, but liars do figure). Imagine the FactCheck Blog with its own show and a sense of humour. It s just gone on a break, but there s plenty of recent episodes to catch up on.
Real Peloton is about pro cycling, by reporter-presenter Ned Boulting and journalist Matt Rendell. They probably should not be left alone to make podcasts, but it s great that they do. They appear sporadically, depending on other work schedules of the presenters. The Banjo Cycles team including Matt Rendell won the IG Markets Hot Lap during the final stage of the Tour of Britain in London and the world championships are this week so maybe they ll have something to say soon.
Answer Me This appears on Thursdays (although it has just gone on a month s break) and gives right-but-sideways answers to listener questions.

And now for number six, what are you listening to? Anything you d recommend?

19 September 2011

Joey Hess: happy haskell hacker

There are certian things haskell is very good at, and I had the pleasure of using it for one such thing yesterday. I wanted to support expressions like find(1) does, in git-annex. Something like:

git-annex drop --not --exclude '*.mp3' --and \
    -\( --in usbdrive --or --in archive -\) --and \
    --not --copies 3

So, parens and booleans and some kind of domain-specific operations. It's easy to build a data structure in haskell that can contain this sort of expression.

 - A Token can either be a single word, or an Operation of an arbitrary type. - 
data Token op = Token String   Operation op
        deriving (Show, Eq)
data Matcher op = Any
          And (Matcher op) (Matcher op)
          Or (Matcher op) (Matcher op)
          Not (Matcher op)
          Op op
        deriving (Show, Eq)

(The op could just be a String, but is parameterized for reasons we'll see later.) As command-line options, the expression is already tokenised, so all I needed to do to parse it is consume a list of the Tokens. The only mildly tricky thing is handling the parens right -- I chose to not make it worry if there were too many, or too few closing parens.

generate :: [Token op] -> Matcher op
generate ts = generate' Any ts
generate' :: Matcher op -> [Token op] -> Matcher op
generate' m [] = m
generate' m ts = uncurry generate' $ consume m ts
consume :: Matcher op -> [Token op] -> (Matcher op, [Token op])
consume m [] = (m, [])
consume m ((Operation o):ts) = (m  And  Op o, ts)
consume m ((Token t):ts)
          t == "and" = cont $ m  And  next
          t == "or" = cont $ m  Or  next
          t == "not" = cont $ m  And  (Not next)
          t == "(" = let (n, r) = consume next rest in (m  And  n, r)
          t == ")" = (m, ts)
          otherwise = error $ "unknown token " ++ t
        where
                (next, rest) = consume Any ts
                cont v = (v, rest)

Once a Matcher is built, it can be used to check if things match the expression the user supplied. This next bit of code almost writes itself.

 - Checks if a Matcher matches, using a supplied function to check
 - the value of Operations. - 
match :: (op -> v -> Bool) -> Matcher op -> v -> Bool
match a m v = go m
        where
                go Any = True
                go (And m1 m2) = go m1 && go m2
                go (Or m1 m2) = go m1   go m2
                go (Not m1) = not (go m1)
                go (Op o) = a o v

And that's it! This is all nearly completly generic and could be used for a great many things that need support for this sort of expression, as long as they can be checked in pure code. A trivial example:

*Utility.Matcher> let m = generate [Operation True, Token "and", Token "(", Operation False, Token "or", Token, "not", Operation False, Token ")"]
*Utility.Matcher> match (const . id) m undefined 
True

For my case though, I needed to run some IO actions to check if expressions about files were true. This is where I was very pleased to see a monadic version of match could easily be built.

 - Runs a monadic Matcher, where Operations are actions in the monad. - 
matchM :: Monad m => Matcher (v -> m Bool) -> v -> m Bool
matchM m v = go m
        where
                go Any = return True
                go (And m1 m2) = liftM2 (&&) (go m1) (go m2)
                go (Or m1 m2) =  liftM2 ( ) (go m1) (go m2)
                go (Not m1) = liftM not (go m1)
                go (Op o) = o v

With this and about 100 lines of code to implement specific tests like --copies and --in, git-annex now supports the example at the top. Just for comparison, find(1) has thousands of lines of C code to build a similar parse tree from the command line parameters and run it. Although I was surprised to see that it optimises expressions by eg, reordering cheaper tests first. The last time I wrote this kind of thing was in perl, and there the natural way was to carefully translate the expression into perl code, which was then evaled. Meaning the code was susceptible to security holes. Anyway, this is nothing that has not been done a hundred times in haskell before, but it's very nice that it makes it so clean, easy, and generic.

Sylvain Beucler: Building Android NDK samples

I searched forever for a way to just build the examples in the Android NDK, preferably from the command line. Here's how. First step: build the C++ code:

cd /usr/src/android-ndk-r6/samples/san-angeles/
/usr/src/android-ndk-r6/ndk-build

Simple enough. Second step:

You will then need to generate your final .apk through the usual means. -- Android NDK Dev Guide

Thanks a ****ing lot! In addition the documentation is very much Eclipse-centered - and I can't be bothered to install and configure all the Eclipse stack for a mere Java wrapper for the above-generated .so file XD We first need to use the 'android' wrapper and download the "SDK Platform Android 2.2, API 8, revision 3" - aka "android-8", the first environment to support OpenGL ES 2.

/usr/src/android-sdk-linux_x86/tools/android

It appears we first need to "manage the project", more precisely import from an existing project: http://developer.android.com/guide/developing/projects/projects-cmdline.html So we create a project by importing the sources from the current directory:

/usr/src/android-sdk-linux_x86/tools/android update project \
  --name san-angeles --path . --target "android-8"

Then we use ant to build the thing: http://developer.android.com/guide/developing/building/building-cmdline.html

apt-get install ant openjdk-6-jdk
ant debug  # avoid messing with signing

We now have a bin/san-angeles-debug.apk and we can run it in the emulator. Create a new "Virtual device" using the "android-8" platform we downloaded above, and launch it. Given the time needed to boot the OS, I suggest enabling snapshots. Scaling to display resulted in a very tiny, dark and slow display, in a first step I suggest using a non-scaled one.

/usr/src/android-sdk-linux_x86/tools/android

At last we can run it! Install it in the emulator:

/usr/src/android-sdk-linux_x86/platform-tools/adb install -r bin/san-angeles-debug.apk

You'll find a DemoActivity in your apps directory

Note: I tried to compile the "hello-gl2", showcasing OpenGL ES 2.0, but sadly it crashes:

Sorry! The application GL2JNI (process com.android.gl2jni) has stopped unexpectedly. Please try again.

The emulator apparently doesn't advertise EGL_OPENGL_ES2_BIT (checked in in GL2JNIView.java). If you try to by-pass it, you'll get loads of:

E/libEGL ( 333): called unimplemented OpenGL ES API

It's worth noting the OpenGLES20 sample from the "API Demos" application uses a fall-back OpenGL ES 1 renderer if v2 is not available - it made me mistakenly believe that OpenGL ES 2 was supported only from Java >( OpenGL ES 2 is not supported in the emulator as of 2011-09-17 - period.

Sylvain Beucler: Advanced 2D OpenGL plotting

Here's a third tutorial from Guus focused on GPU-assisted plotting. It also provides excellent exercices on manipulating the OpenGL windows using the Viewport and the Scissors

http://en.wikibooks.org/wiki/OpenGL_Programming/Scientific_OpenGL_Tutorial_03

The OpenGL wikibook is free documentation, and you are welcome to redistribute it under certain conditions; see the CC-BY-SA or GFDL licenses for details

Christoph Berg: Tab completion in vim

Vim's habit of completing the full filename of the first match on :e has always annoyed me. Rhonda pointed me to wildmode - thanks!

set wildmode=longest,list:longest,list:full

Stefano Zacchiroli: why there are so many debian derivatives

A while ago I've been interviewed for a forthcoming article in the LinuxUser magazine on the topic of why there are so many Debian derivatives and on the potential impact, on them, of the (potential) advent of the so called "Debian Rolling" experiment. Below you can find my take on those topics. Questions are by Ferdinand Thommes, who has been involved in various Debian derivatives, including the forthcoming Siduction.

1.) Why do you think people base distributions on Debian? Looking at how many there is, since Corel and Libranet set sail in 1999, there must be a reason why, for one, people fork Debian in the first place, and secondly, why they choose Debian to do so and not some other distribution.

There are always advantages in creating a so called "derivative" distribution with respect to creating one from scratch. The main one is the possibility of reusing the packaging work which is already done in the "upstream" distro, focusing derivatives' efforts on customization, usually with clear target public in mind. Debian pushes most of those advantages (for derivatives) at their extremes.

The Debian archive is huge (about 30'000 packages with Squeeze), meaning that the amount of packaging work that can be reused is similarly huge.
Debian has a long history and a well established community, which reduces the risk of "betting" the efforts of derivatives on a distro that might disappear anytime soon.
Debian is well known for its obsession with package quality, meaning that the risk of trusting pre-existing packages is greatly reduced, and that package behavior is on average very dependable as well as documented in standard documents such as the Debian Policy
Debian is also well known for its obsession with licensing, at the point of being considered by many one of the authoritative sources for deciding what constitutes Free Software and what does not. For derivatives that implies that they do not need to worry---as long as they trust Debian, of course---neither about legal redistribution risks nor about software freedom claims on their distro.

But there are also a couple of "political" reasons for basing derivatives on Debian. One is quite subtle and applies mostly to commercial distributions. If you are designing one such commercial distro, you have to be based on an independent distro with no commercial interests, lest risking that petty (technical or otherwise) choices might be made just to undermine your business. Among "popular" GNU/Linux distros, Debian is essentially the only one which is both volunteer-based and not ascribable to any specific company. The last reason why I think many derivatives are based on Debian is our vocation to universality. Since we are not targeting any specific public, but rather trying to provide an operating system suitable for several different use cases (e.g. the tasks supported by the Debian installer or the choice of Debian Pure Blends), we offer an ideal starting point for those who are going to have customization as their main business.

2.) What could be the benefits for Debian and for its Derivates, if the discussions about a second Debian Release based on the Testing branch really come to be reality?

We are considering the possibility of adding a new Debian suite which is "rolling" like Testing, in the sense that new software releases flow in regularly rather than only at periodic bumps corresponding to major distribution releases. There seem to be user demand for that and I personally believe that such a scheme could address the needs of both advanced desktop users and of developers, which are now strand to mix and match Debian suites ending up using package combinations which benefit, as a whole, of very little quality check. The main issues to face to get there is how to make sure that adding such a suite would not get in the way of preparing high quality Stable releases, in terms of quality control, developer focus, etc. Note that such a hypothetical "rolling" suite needs does not necessarily need to be based on Debian Testing, although that is a good starting point. The reason why it's not a good endpoint is that Testing has been created (circa 2000), and is still used, as an internal tool to prepare the forthcoming Debian Stable release. As such, it is not entirely suitable for final user consumption. For instance, during the Debian "freeze" period the flow of new software releases that reach Testing is greatly reduced up to stopping completely just before the release of a new Debian Stable. I've been told that users of testing-based (derivative) rolling releases regularly complain about this aspect during Debian freezes, which is not surprising. The main advantage for derivatives which are already offering a Debian-based "rolling" release will then be the possibility of basing their work on a suite less bound to the life cycle of Debian Stable than Testing is at present. On the converse side, that would also mean that derivatives will need to differentiate more from Debian than by just saying they are "rolling", which is a good thing: differences among distros are healthy and drive innovation. With those derivatives, on the other hand, which will recognize their goals as being aligned with Debian's goals---Debian Stable today, maybe Debian "rolling" tomorrow---we will be happy to join efforts.

Keith Packard: MacBook-Air-2

Fixing the Sandbrige MacBook Air display initialization We left our hero with Debian installed on the MacBook Air, but with the display getting scrambled as soon as the i915 driver loaded. As was reported to Matthew, the problem is as simple as a lack of the right mode for the eDP panel in the machine. This mode is supposed to come from the panel EDID data, but for some reason the driver wasn t able to query the EDID data, and so it decided to try some random panel timings it dug out of the VBT tables, which are generally supposed to be used by LVDS panels. Apple helpfully stuck valid data there, but for some other panel one that is 1280x800 pixels instead of the 1366x768 pixel panel in the MacBook Air. I heard rumors that some machines would get a black screen when the i915 driver loaded. I was fortunate my machine simply displayed a 1366x768 subset of the programmed 1280x800 mode. A bit of garbage on the right side, and a few scanlines missing at the bottom. Quite workable, especially after I ran fbset -yres 768 to keep the console in the visible portion of the screen. DDC failure Looking through the kernel logs, the Intel driver tries to access the EDID data and times out, as if DDC is just completely broken. This is rather unexpected; the eDP spec says that the panel is required to support DDC and provide EDID. Now, we ve seen a lot of panels which don t quite live up to the rigorous eDP specifications, but it s a bit surprising from Apple, who generally do VESA stuff pretty well. We ve heard reports about panels reporting invalid EDID data, or EDID data which didn t actually match the panel (causing us to prefer the VBT data on LVDS machines). But I ve not heard of an eDP panel which didn t have anything hanging off of the DDC channel. But X works fine? During early debugging, I happened to start X up. Much to my surprise, X came up at the native 1366x768 mode. Digging through the kernel logs after that, I discovered that EDID was successfully fetched from the eDP panel while X started up. At this point, I knew it was all downhill the EDID data was present, it just wasn t getting picked up during the early part of the driver initialization when the console mode is initialized. eDP power management The CPU is given complete control over the power management of the eDP panel; sequencing through various power states and waiting appropriate amounts of time when things change. Given the goal of keeping power usage as low as possible, this makes a huge amount of sense. The eDP spec is quite clear though, without power, the panel will not respond to anything over the aux channel, and that includes EDID data. The eDP panel power hardware in the Sandybridge chip has a special mode for dealing with this requirement. If the panel is not displaying data, you can supply power for the aux channel stuff by setting a magic bit in the panel power registers. When initializing the frame buffer, the kernel driver turns off the panel completely so that it has all of the hardware in a known state (yeah, this is not optimal, but that s another bug). When X started, the panel was already running with the console mode. Given the difference between these two states EDID querying with the panel off failed, while EDID querying with the panel on worked, it seemed pretty clear that the panel power wasn t getting managed correctly. So, it seemed pretty clear that the magic power the panel bit wasn t getting turned on at the right times. Getting the power turned on. I stuck a check inside all of the aux channel communication functions to see where things were broken. This pointed out several places missing the panel power calls. This wasn t quite sufficient to get EDID data flowing. The remaining problem was that the code wasn t waiting long enough after turning the panel power on before starting the aux channel communication. A few msleep calls and huzzah! EDID at boot time and the console had the right mode. Making it faster However, it turns out that the driver does this a lot, and the msleeps required were fairly long the eDP panel wants a 500ms delay from turning the panel power off before you can turn it back on. I fix this by simply delaying the panel power off until things were idle for a long time. Now mode setting goes zipping through, and a few seconds later, the bit to force panel power on gets turned off. Getting these bits for yourself I ve pushed out the code to my (temporary) kernel repository git://people.freedesktop.org/~keithp/linux in the fix-edp-vdd-power branch. I d love to hear if you ve tried this on either a MacBook Air or any other eDP machine from Ironlake onwards.

Next.

Previous.