Search Results: "js"

30 September 2011

Thorsten Glaser: Improvements welcome

No I don t really know any SQL. In fact, even at vocational school, where we focussed on database normalisation anyway, I tried hard to avoid the topic. Feel free to access here my entire knowledge about SQL (I did use Amaya, Arena and Arachne though. Liked only Arachne out of these three, and then, only under DOS, not its Unix version. Maybe the WWW could be named AAA instead? But then, lynx(1) is the one true browser ) Ah, well. While at it the entirety of my Perl knowledge is here: perltoc(1) with quick links to perlfunc(1). The entirety of my (X)HTML and ECMAscript knowledge, DE: SELFHTML; although, the spec and DTD helped; and to write my notes on JSON, I took a peek at the formal ECMAscript spec as well propos, does anyone know a (good enough) indent(1) equivalent for ECMAscript, as I am trying to strip down some, inherited (GPL, yes) code for a hobby project, but Geo-people seem to produce illegible code?

26 September 2011

John Goerzen: Five

Dad, will you and mom stay up all night decorating the house for me?
That was Jacob s question to me at bedtime the evening before his fifth birthday. Jacob had already had his birthday party a few weeks ago. When scheduling means that parties happen that far away from the boys real birthdays, they get a smaller celebration with just Terah and me where we open their presents from us. A low-key thing, so we weren t planning to decorate the house. I said, Well, I don t think we ll be doing all that since you already had your birthday party. And the look of eager anticipation on his face turned to a very sad and disappointed look, and made me feel really bad. Uhoh. So while Jacob was sleeping, I mentioned it to Terah. We decided we should improvise something simple, so she found some old streamers and we taped them up, running them through several rooms in the house and across his door. It took a few minutes using supplies we already had, but the joy the next morning was priceless. Oh dad, you said you wouldn t stay up all night, BUT YOU DID! Oh I am SO HAPPY! YAY YAY YAY! And he ran through the house to discover what else was set up. Then he ran to find Oliver and gave him a tour of everything. Then we sat down to open his presents. Here he is, holding a present from Terah and me: That s a copy of The Lorax. My friend Jonathan had brought a copy along during our road trip in Mexico, and Jacob was really excited about it. And didn t really want to give it up, because You can only get The Lorax in Mexico. He (and Oliver) really enjoyed all his presents he also got a train book from us (which he said, Oh, yay, it s the book they have at preschool! ), and a game and some other presents from the distant relatives. But the highlight was something of an impulse buy. I was at the RadioShack in Derby a place I ve written about before, It is what a RadioShack should (and used to) be. It has a large amateur radio section, sells all sorts of coax by the foot, and provides astonishingly good post-sale service. Well, I was there with a radio question, and Mark (the owner) who is an excellent salesman in a positive way pointed me to a display of snap kits. I noticed their Electronics 101 Snap-Kit (a rebranded Snap Circuits Jr. SC-100). A ham in Pennsylvania had suggested them to me once, and as Jacob s birthday was coming up, I gave it some thought. The kit said ages 8 and up. I asked Mark what he thought about a boy just turning 5. He said, Well, probably not normally. But knowing you, if you re there to work on them with him, I think he d enjoy it. But I wouldn t have him work on it by himself. I agreed and we bought it. I pondered how to explain the concept of this thing to Jacob. Eventually I decided I would call it a toy-building kit. He understood that. Jacob and I spent hours together working with it. He would flip through the book, either picking circuits that looked interesting or telling me what kind of circuit he wanted to build. Then I would tell him what to put where, and he d snap them together and play with them. He only played with each finished product a few minutes before he was ready to try another. Once I got him very excited with my offer to show him how to hook up two switches in parallel for the fan he built (and later introduced the parallel vs. series concept by hooking them up in series instead.) Here we are working on it together. Jacob repeatedly called Terah over to look at the things he built. He was very excited that he assembled it himself. Eventually, Oliver (age 2) came over wanting to help. So he sat on my lap, and handed parts to Jacob, then Jacob put them on the grid. Oliver really enjoyed being involved in this way, even though I had to keep him from doing things like ripping the capacitor off its mount. We tend to be modest in terms of the number of things we give the boys and their cost, reasoning that we, like many, already have too many toys in our house, and that greater cost doesn t necessarily equate with a better experience for the boys. I particularly look for things with lasting value and unique experiences for them, and I think we succeeded this year. But I realized quickly that the greatest value of this kit wasn t electronics. It was having a great way for me to spend a lot of time doing things with the boys which all of us enjoyed. Those hours building things together were as much a present for me as for Jacob, I m quite sure. Jacob s real party was a few weeks ago at the Great Plains Transportation Museum in Wichita. They let people rent a historic caboose to use for a birthday party for children. So we did that for Jacob this year. That was a huge hit for the boys. Jacob got to help his grandpa make some pie (instead of cake) for the party. He enjoyed eating it, of course. He enjoyed opening his presents high up on the observation chair in the caboose. And the boys got to play on all the other equipment in the museum. Jacob enjoyed playing tour guide for family since most of them hadn t been there. He also enjoyed watching freight trains pass on the other side of the fence from the museum preferably while sitting in one of the museum s engines. Oliver certainly didn t get left out. Train-watching is serious business, after all. Jacob has long talked about going to the train museum and the airplane museum (Kansas Aviation Museum) on the same day, so one of his birthday surprises was that we went to the airplane museum after his party. His favorite item there is a retired FedEx 727. Here he is walking down the rear of the plane. And, of course, they played captain and co-pilot in several different planes.

23 September 2011

Matthew Garrett: UEFI secure booting (part 2)

Microsoft have responded to suggestions that Windows 8 may make it difficult to boot alternative operating systems. What's interesting is that at no point do they contradict anything I've said. As things stand, Windows 8 certified systems will make it either more difficult or impossible to install alternative operating systems. But let's have some more background.

We became aware of this issue in early August. Since then, we at Red Hat have been discussing the problem with other Linux vendors, hardware vendors and BIOS vendors. We've been making sure that we understood the ramifications of the policy in order to avoid saying anything that wasn't backed up by facts. These are the facts:
Microsoft have a dominant position in the desktop operating system market. Despite Apple's huge comeback over the past decade, their worldwide share of the desktop market is below 5%. Linux is far below that. Microsoft own well over 90% of the market. Competition in that market is tough, and vendors will take every break they can get. That includes the Windows logo program, in which Microsoft give incentives to vendors to sell hardware that meets their certification requirements. Vendors who choose not to follow the certification requirements will be at a disadvantage in the marketplace. So while it's up to vendors to choose whether or not to follow the certification requirements, Microsoft's dominant position means that they'd be losing sales by doing so.

Why is this a problem? Because there's no central certification authority for UEFI signing keys. Microsoft can require that hardware vendors include their keys. Their competition can't. A system that ships with Microsoft's signing keys and no others will be unable to perform secure boot of any operating system other than Microsoft's. No other vendor has the same position of power over the hardware vendors. Red Hat is unable to ensure that every OEM carries their signing key. Nor is Canonical. Nor is Nvidia, or AMD or any other PC component manufacturer. Microsoft's influence here is greater than even Intel's.

What does this mean for the end user? Microsoft claim that the customer is in control of their PC. That's true, if by "customer" they mean "hardware manufacturer". The end user is not guaranteed the ability to install extra signing keys in order to securely boot the operating system of their choice. The end user is not guaranteed the ability to disable this functionality. The end user is not guaranteed that their system will include the signing keys that would be required for them to swap their graphics card for one from another vendor, or replace their network card and still be able to netboot, or install a newer SATA controller and have it recognise their hard drive in the firmware. The end user is no longer in control of their PC.

If Microsoft were serious about giving the end user control, they'd be mandating that systems ship without any keys installed. The user would then have the ability to make an informed and conscious decision to limit the flexibility of their system and install the keys. The user would be told what they'd be gaining and what they'd be giving up.

The final irony? If the user has no control over the installed keys, the user has no way to indicate that they don't trust Microsoft products. They can prevent their system booting malware. They can prevent their system booting Red Hat, Ubuntu, FreeBSD, OS X or any other operating system. But they can't prevent their system from running Windows 8.

Microsoft's rebuttal is entirely factually accurate. But it's also misleading. The truth is that Microsoft's move removes control from the end user and places it in the hands of Microsoft and the hardware vendors. The truth is that it makes it more difficult to run anything other than Windows. The truth is that UEFI secure boot is a valuable and worthwhile feature that Microsoft are misusing to gain tighter control over the market. And the truth is that Microsoft haven't even attempted to argue otherwise.

comment count unavailable comments

18 September 2011

Paul Wise: Revisiting personal software freedom

Since it was Software Freedom Day again, I figured I should revisit my personal software freedom and see what has changed since my post two years ago. There hasn't been a vrms meme again this year so I don't have anyone else to compare with. A small survey on IRC indicates folks still need non-free nVidia drivers, embedded software, GNU documentation, Java and more. Since then I have switched my laptop from the Dell Inspiron 6400 to the Thinkpad X201 Tablet. Nothing really changed with my laptop, GNU documentation is still non-free and the new Intel WiFi chip on my new laptop still uses non-free embedded software. To remind myself of the non-free bits embedded in the hardware (CPU microcode, BIOS, EC etc), I have installed intel-microcode and microcode.ctl. Since then I didn't get any new phone, still the same OpenMoko FreeRunner. I'm now using QtMoko on a 4GB microSD card and Debian and SHR on other cards for testing. QtMoko is based on Debian and is the latest incarnation of Qtopia. It is probably free but I haven't done any audit of it. One interesting thing that changed with the FreeRunner in the past two years is that Harald Welte and friends started OsmocomBB, a project to create free software for the GSM modem built into the FreeRunner and many old feature-phones. I haven't tried it yet due to lack of time. While at the Chaos Communication Camp 2011 I learnt about the blackbox that SIM cards are (video), which hopefully OsmocomBB has a chance to protect against. AFAIK nothing else has changed to improve software freedom on the FreeRunner, the WiFi, GPS and other parts still contain non-free software with no chance of a replacement in sight. Since then I am still using the same wireless router and ADSL modem and I wasn't brave enough to try replacing their software. I still use gmail out of intertia, but I at least have started using offlineimap to prepare for the day when I will move away from it. I still use Google Maps, mainly for searching for public transport routes. With my recent travels in Europe I encountered many places where OpenStreetMap has much better coverage than Google. One nice thing about Google Maps adding public transport information is that public transport organisations have been publishing public data feeds for Google to consume. Apparently there is also a realtime variant. I also found one website using a JSON API for Google street view. The combination of these gives me hope for better support for spatial data in free software. The data (aerial and streets) will obviously remain a much longer term freeness issue though. For flash video on the web get-flash-videos appeared and made it even easier to ignore flash, especially since I got a few patches added to it. In addition Swfdec died but Gnash improved enough to replace it. Lightspark was also created but has so far been pretty buggy and unstable. Free RAR v3 format support appeared in the form of TheUnarchiver. The Voxware audio codec and DigiTrakker MDL files are still not supported by Rhythmbox. Unicode has been updated and so now Debian doesn't support every script fully. I still manage to avoid Skype. The nouveau drivers have matured enough to provide 3D support, multi-monitor support and general stability for the one nVidia card I used in the past two years. Overall, I'm reasonably happy with my level of software freedom. My main strategy for preventing regressions in my software freedom remains to just avoid doing things that require non-free software. The most problematic FLOSS issues for me are embedded software, spatial data support and Flash support. Please feel free to contact me with any comments or questions. Happy Software Freedom Day everyone!

14 September 2011

Mike Hommey: Building a custom kernel for the Nexus S

There are several reasons why someone would want to build a custom kernel for their Android phone. In my case, this is because I wanted performance counters (those used by the perf tool that comes with the kernel source). In Julian Seward s case, he wanted swap support to overcome the limited memory amount on these devices in order to run valgrind. In both cases, the usual suspects (AOSP, CyanogenMod) don t provide the wanted features in prebuilt ROMs. There are also several reasons why someone would NOT want to build a complete ROM for their Android phone. In my case, the Nexus S is what I use to work on Firefox Mobile, but it is also my actual mobile phone. It s a quite painful and long process to create a custom ROM, and another long (but arguably less painful thanks to ROM manager) process to backup the phone data, install the ROM, restore the phone data. And if you happen to like or use the proprietary Google Apps that don t come with the AOSP sources, you need to add more steps. There are however tricks that allow to build a custom kernel for the Nexus S and use it with the system already on the phone. Please note that the following procedure has only been tested on two Nexus S with a 2.6.35.7-something kernel (one with a stock ROM, but unlocked, and another one with an AOSP build). Also please note that there are various ways to achieve many of the steps in this procedure, but I ll only mention one (or two in a few cases). Finally, please note some steps rely on your device being rooted. There may be ways to do without, but I m pretty sure it requires an unlocked device at the very least. This post doesn t cover neither rooting nor unlocking. Preparing a build environment To build an Android kernel, you need a cross-compiling toolchain. Theoretically, any will do, provided it targets ARM. I just used the one coming in the Android NDK:
$ wget http://dl.google.com/android/ndk/android-ndk-r6b-linux-x86.tar.bz2
$ tar -jxf android-ndk-r6b-linux-x86.tar.bz2
$ export ARCH=arm
$ export CROSS_COMPILE=$(pwd)/android-ndk-r6b/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86/bin/arm-linux-androideabi-
For the latter, you need to use a directory path containing prefixed versions (such as arm-eabi-gcc or arm-linux-androideabi-gcc), and include the prefix, but not gcc . You will also need the adb tool coming from the Android SDK. You can install it this way:
$ wget http://dl.google.com/android/android-sdk_r12-linux_x86.tgz
$ tar -zxf android-sdk_r12-linux_x86.tgz
$ android-sdk-linux_x86/tools/android update sdk -u -t platform-tool
$ export PATH=$PATH:$(pwd)/android-sdk-linux_x86/platform-tools
Building the kernel For the Nexus S, one needs to use the Samsung Android kernel tree, which happens to be unavailable at the moment of writing due to the kernel.org outage. Fortunately, there is a clone used for the B2G project, which also happens to contain the necessary cherry-picked patch to add support for the PMU registers on the Nexus S CPU that are needed for the performance counters.
$ git clone -b devrom-2.6.35 https://github.com/cgjones/samsung-android-kernel
$ cd samsung-android-kernel
You can then either start from the default kernel configuration:
$ make herring_defconfig
or use the one from the B2G project, which enables interesting features such as oprofile:
$ wget -O .config https://raw.github.com/cgjones/B2G/master/config/kernel-nexuss4g
From then, you can use the make menuconfig or similar commands to further configure your kernel. One of the problems you d first encounter when booting such a custom kernel image is that the bcm4329 driver module that is shipped in the system partition (and not in the boot image) won t match the kernel, and won t be loaded. The unfortunate consequence is the lack of WiFi support. One way to overcome this problem is to overwrite the kernel module in the system partition, but I didn t want to have to deal with switching modules when switching kernels. There is however a trick allowing the existing module to be loaded by the kernel: compile a kernel with the same version string as the one already on the phone. Please note this only really works if the kernel is really about the same. If there are differences in the binary interface between the kernel and the modules, it will fail in possibly dangerous ways. To use that trick, you first need to know what kernel version is running on your device. Settings > About phone > Kernel version will give you that information on the device itself. You can also retrieve that information with the following command:
$ adb shell cat /proc/version
With my stock ROM, this looks like the following:
Linux version 2.6.35.7-ge382d80 (android-build@apa28.mtv.corp.google.com) (gcc version 4.4.3 (GCC) ) #1 PREEMPT Thu Mar 31 21:11:55 PDT 2011
In the About phone information, it looks like:
2.6.35.7-ge382d80
android-build@apa28
The important part above is -ge382d80, and that is what we will be using in our kernel build. Make sure the part preceding -ge382d80 does match the output of the following command:
$ make kernelversion
The trick is to write that -ge382d80 in a .scmversion file in the kernel source tree (obviously, you need to replace -ge382d80 with whatever your device has):
$ echo -ge382d80 > .scmversion
The kernel can now be built:
$ make -j$(($(grep -c processor /proc/cpuinfo) * 3 / 2))
The -j part is the general rule I use when choosing the number of parallel processes make can use at the same time. You can pick whatever suits you better. Before going further, we need to get back to the main directory:
$ cd ..
Getting the current boot image The Android boot image living in the device doesn t contain only a kernel. It also contains a ramdisk containing a few scripts and binaries, that starts the system initialization. As we will be using the ramdisk coming with the existing kernel, we need to get that ramdisk from the device flash memory:
$ adb shell cat /proc/mtd awk -F'[:"]' '$3 == "boot" print $1 '
The above command will print the mtd device name corresponding to the boot partition. On the Nexus S, this should be mtd2.
$ adb shell
$ su
# dd if=/dev/mtd/mtd2 of=/sdcard/boot.img bs=4096
2048+0 records in
2048+0 records out
8388608 bytes transferred in x.xxx secs (xxxxxxxx bytes/sec)
# exit
$ exit
In the above command sequence, replace mtd2 with whatever the previous command did output for you. Now, you can retrieve the boot image:
$ adb pull /sdcard/boot.img
Creating the new boot image We first want to extract the ramdisk from that boot image. There are various tools to do so, but for convenience, I took unbootimg, on github, and modified it slightly to seemlessly support the page size on the Nexus S. For convenience as well, we ll use mkbootimg even if fastboot is able to create boot images. Building unbootimg, as well as the other tools rely on the Android build system, but since I didn t want to go through setting it up, I figured a minimalistic way to build the tools:
$ git clone https://github.com/glandium/unbootimg.git
$ git clone git://git.linaro.org/android/platform/system/core.git
The latter is a clone of git://android.git.kernel.org/platform/system/core.git, which is down at the moment.
$ gcc -o unbootimg/unbootimg unbootimg/unbootimg.c core/libmincrypt/sha.c -Icore/include -Icore/mkbootimg
$ gcc -o mkbootimg core/mkbootimg/mkbootimg.c core/libmincrypt/sha.c -Icore/include
$ gcc -o fastboot core/fastboot/ protocol,engine,bootimg,fastboot,usb_linux,util_linux .c core/libzipfile/ centraldir,zipfile .c -Icore/mkbootimg -Icore/include -lz
Once the tools are built, we can extract the various data from the boot image:
$ unbootimg/unbootimg boot.img
section sizes incorrect
kernel 1000 2b1b84
ramdisk 2b3000 22d55
second 2d6000 0
total 2d6000 800000
...but we can still continue
Don t worry about the error messages about incorrect section sizes if it tells you we can still continue . The unbootimg program creates three files: All that is left to do is to generate the new boot image:
$ eval ./mkbootimg $(sed s,boot.img-kernel,samsung-android-kernel/arch/arm/boot/zImage, boot.img-mk)
Booting the image There are two ways you can use the resulting boot image: one-time boot or flash. If you want to go for the latter, it is best to actually do both, starting with the one-time boot, to be sure you won t be leaving your phone useless (though recovery is there to the rescue, but is not covered here). First, you need to get your device in the fastboot mode, a.k.a. boot-loader:
$ adb reboot bootloader
Alternatively, you can power it off, and power it back on while pressing the volume up button. Once you see the boot-loader screen, you can test the boot image with a one-time boot:
$ ./fastboot boot boot.img
downloading 'boot.img'...
OKAY [ 0.xxxs]
booting...
OKAY [ 0.xxxs]
finished. total time: 0.xxxs
As a side note, if fastboot sits waiting for device , it either means your device is not in fastboot mode (or is not connected), or that you have permissions issues on the corresponding USB device in /dev. Your device should now be starting up, and eventually be usable under your brand new kernel (and WiFi should be working, too). Congratulations. If you want to use that kernel permanently, you can now flash it after going back in the bootloader:
$ adb reboot bootloader
$ ./fastboot flash boot boot.img
sending 'boot' (2904 KB)...
OKAY [ 0.xxxs]
writing 'boot'...
OKAY [ 0.xxxs]
finished. total time: 0.xxxs
$ ./fastboot reboot
Voil .

11 September 2011

Gregor Herrmann: RC bugs 2011/36

after some months of looking only at "my" packages, I'm back in the #RCBW flow. here's my overview of RC bug activities & NMUs in the last week: two short observations:

28 August 2011

Micha Lenk: Finally transitioning to a new GnuPG key

Finally I managed to write up a transition statement for my not so new, but stronger GnuPG key. See below:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1,SHA256
I am transitioning my GPG key from an old 1024-bit key to a new 4096-bit key.
The old key will continue to be valid for some time, but I prefer all new
correspondence to be encrypted for the new key, and will be making all
signatures going forward with the new key.
If you have signed my old key, I would appreciate signatures on my new key as
well, provided that your signing policy permits that without reauthenticating
me.
The old key, which I am transitioning away from, is:
pub   1024D/99E141B4 2004-02-10
      Key fingerprint = 25FE 4741 4770 0558 949D  1DB1 58DD 3FE2 99E1 41B4
The new key, to which I am transitioning, is:
pub   4096R/51B85139 2009-06-18
      Key fingerprint = A3EB B41F C5AB D675 CEE4  1C45 EA6C A6B9 51B8 5139
Thanks in advance.
Cheers,
Micha Lenk
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
iEYEARECAAYFAk5aOnkACgkQWN0/4pnhQbTxPgCgzRhREZlQiKJyI9UIdJLLs3Zq
bH4AnA1myFxgDWM7aUMHXgvvsujLTjiWiQIcBAEBCAAGBQJOWjp5AAoJEOpsprlR
uFE5gCsP/0dtCUPl9aQHV1MbQl7+bMofpsC2ikkpdZmrzi68jTG16We49BuzY+PV
S8FhXqg17/YxhKYDnDNTowfztUOyjAOJxy5vrqm3X5xiLwTqN3js9mra+vb4s35k
EVbKMzLLDhj3i0FjeargWEmJmm9cVhaZWKvOvQUhDJAilqvEQ0/50P7B8I+1YvtV
UHoQKbweTljVSlK5R1YPPy9i6r2/oZBYxK4nrknWwS+qPQ5luqelJd+mZdgQ6tow
7HIvtmPgCblJ+hYZWFpoZK6vxs8RaBbuCQcKwYArNhZT/v4TeD/LAaUmIkbQMyKV
J2TKuEHya4+5GMbtg6BGKeiZpleEHPnAq1AfvGpz6opkxjxCLG3RO3X8D3EuM3RW
mkq60mWM8+Zwu1yKbb62iHplp6jpyiQgdjJlB6eHjX7SdY7CvHgYZxGDx4kclP0/
HAdig2U1T+nG6Nn5XflmmKwvNLuKlQlIwJ5NeXyCONRnYvdomQn2hgvkjwMLCdFh
ulIhxa4UvDY7/aQNPZeOrvDHb2XYpiV3TwA9hLgQXXWd0FPmUMVVPpKpRjilaWth
Mtq/QiMGP5Mq/YxgLInRZHGyajDtE67RD4+RgHYOP50cP3UGPoB4ncc2EEtM2kfe
BzJrXPHmdtyGEA6Korl0YwUTRnYsqkqkY1VqDsO0UkOLlV7RAzb9
=9vw1
-----END PGP SIGNATURE-----

23 August 2011

Matthew Palmer: UPSes in Datacentres

(This was going to be a comment on this blog post, but it s a Turdpress site that wants JS and cookies to comment. Bugger that for a game of skittles. Rimuhosting s recent extended outage due to power problems was apparently caused by a transfer switch failure at their colo provider. This has led people to wonder if putting UPSes in individual racks is a wise move. The theory is that in the event of a small outage, the UPS can keep things humming, and in an extended outage you can gracefully shut things down rather than having a hard thump. I happen to think this theory is bunkum. Your UPS is a newly instituted single point of failure. I d be willing to bet that the cost of purchasing, installing, and maintaining the UPSes, as well as the cost of the outages that inevitably result from their occasional failure, would be far greater than the cost of the occasional power outage you get in a well-managed facility. Good facilities don t have small outages. They don t have squirrels in the roof cavities, and they don t have people dropping spanners across busbars. The only outages they have are the big ones, when some piece of overengineered equipment turns out to be not so overengineered the multi-hour (or multi-day) ones where your UPS isn t going to stop you from going down. Your SLA credit and customer goodwill is already toast, so all you re saving is the incremental cost of a little bit more downtime while you get fscks run. If you want the best possible power reliability, get yourself into a really well engineered facility, and run dual-power on everything. Definitely run the numbers before you go down the UPS road; I ll bet you find they re not worth it.

4 August 2011

John Goerzen: Please. Stop. Removing. Features.

apt-get dist-upgrade is getting a little less fun these days. It s not because of a problem with Debian or with apt-get. It s because of things upstream authors are doing. It seems that upstreams, for some reason, like to remove features from new versions of software. The two recent examples to bite me are the removal of the Gnome Terminal features to play the bell through sound card rather than the console speaker, and the recent upgrade to gtkpod the best iPod music manager out there which removed the ability to actually, well, play music. Wha ? Earlier examples where when Gnome removed the ability to type a filename in the Open dialog box (remember that? thankfully reverted eventually.) KDE 4 was infamous for this too, with maybe thousands of instances (and they broke Amarok so badly that it didn t actually play music from my devices either.) I don t understand this. The Gnome people somehow thought that they knew better than I what options I might want, I guess. But I am totally baffled about gtkpod. All it ever did was call xmms (or something like it such as qmmp) with appropriate arguments to play a file. Simple, configurable, and supremely useful. Probably just a few lines of code. And even that is gone. There is a post by Ingo Molnar complaining that perhaps Gnome and KDE are trying to mimic the Apple and Google result without internalizing the process. Perhaps. It all seems so baffling to me though. Updated: rewrote introduction.

31 July 2011

John Goerzen: A 4-year-old, Linux command line, and microphone

There are certain times when I m really glad that we have Linux on the house for our boys to play with. I ve already written how our 4-year-old Jacob has fun with bash and can chain together commands to draw ASCII animated steam locomotives. Today I thought it might be fun to install cw, a program that can take text on standard input and play it on the console speaker or sound card as Morse code. Just the sort of thing that I could see Jacob eventually getting a kick out of. But his PC was mute. We opened it up and discovered it didn t have a console speaker. So we traipsed downstairs, dug out an external speaker, and I figured out how to enable the on-board audio chipset in the BIOS. So now the cw command worked, but also there were a lot of other possibilities. We also brought up a microphone. While Jacob was busy with other things, I set to work getting things hooked up, volume levels adjusted, and wrote some shell scripts for him. I also printed out this reference sheet for Jacob: He is good at reading but not so good at spelling. I intentionally didn t write down what the commands do, hoping that this would provide some avenue for exploration for him. He already is generally familiar with the ones under the quiet category. I wrote a shell script called record . It simply records from the microphone and drops a timestamped WAV file in a holding directory. He can then type play to simply play back whatever he recorded most recently. Easy enough. But what he really wanted was sound for his ASCII steam locomotive. So with the help of a Google search for steam train mp3 , I wrote a script ssl (sound steam locomotive) that starts playing the sound in the background if it isn t already going, and then runs sl to show the animation. This was a big hit. I also set it up so he can type play train to hear that audio, or play song to play our favorite train song (Always a Train in My Dreams by Steve Gillette). Jacob typed that in and sat still for the entire 3 minutes listening to it. I had to hook up an Ethernet cable to his machine to do all this, and he was very interested that I was hooking his computer up to mine in some way. He thought all the stuff about cables in the walls was quite exciting. The last thing I did was install flite, a speech synthesis program. I wrote a small shell script called talk which reads a line at a time from stdin and invokes flite for each one (to give more instant feedback rather than not starting playback until after having read a large block from stdin). He had some fun hearing it say his name and other favorite words, but predictably the most fun was when he typed gibberish at it, and heard it try to pronounce or spell nonsense words. In all, he was so excited about this new world of computer sound opened up to him. I m sure there will be lots of happy experimentation and discovery going on.

28 July 2011

Tanguy Ortolo: Using a gamepad under GNU/Linux

Saitek PS2700 gamepad I have just bought a gamepad, the Saitek V.3 Cyborg Rumble Pad (also marketed as PS2700), mainly to play at SuperTuxKart under Debian GNU/Linux. It works perfectly, except for one minor feature, and it can be used for playing and as an X11 input device.System support This device uses USB HID so it is directly supported by the Linux kernel, with its module joydev, I guess: it appears as /dev/input/eventX and /dev/input/jsY. It can be tested with jstest-gtk from the package jstest-gtk: it has 7 axes (two sticks, one directional pad and two triggers that count as one single axis) and 13 buttons (4 regular, 2 rear, the 2 triggers that also send button events, 3 specials and the 2 sticks that can be clicked). They are presented in an order that is rather logical, except the trigger axis is inserted between the two axis of the second stick. It also features two strange buttons: Game support I only tested SuperTuxKart, but I think any game able to use a gamepad or a joystick should support it: after all, they all use the same protocol. The default mapping is rather logical but I find it more convenient to use the triggers rather than the main stick to accelerate and brake. I do not know if the vibration feature can be used at all, because I do not know how it is exposed and I do not think I have any game that would be able to take advantage of it. X.Org support This is an interesting feature: X.Org has a joystick(4) driver, available in the package xserver-xorg-input-joystick that allows you to use a joystick or a gamepad as an input device, both for pointing and typing. So, just for fun, I installed the joystick driver. It installs a default configuration file as /usr/share/X11/xorg.conf.d/50-joystick.conf, that is enough to provide features equivalent to a regular mouse. But given the number of axes and buttons of my pad, it deserved a special configuration in /etc/X11/xorg.conf.d/rumblepad.conf:
Section "InputClass"
    Identifier  "Rumble Pad"
    Option      "MatchIsJoystick"   "on"
    Option      "MatchProduct"      "Cyborg V.3 Rumble Pad"
    # Main stick axes: point
    Option      "MapAxis1"          "mode=relative axis=+1x deadzone=5000"
    Option      "MapAxis2"          "mode=relative axis=+1y deadzone=5000"
    # Second stick axes: scroll
    Option      "MapAxis3"          "mode=relative axis=+1zx deadzone=5000"
    Option      "MapAxis5"          "mode=relative axis=+1zy deadzone=5000"
    # Triggers: scroll vertically
    Option      "MapAxis4"          "mode=relative axis=+1zy deadzone=5000"
    # Directional pad: point
    Option      "MapAxis6"          "mode=accelerated axis=+1x deadzone=5000"
    Option      "MapAxis7"          "mode=accelerated axis=+1y deadzone=5000"
    # Buttons 1, 2 and 3 : click (no use for button 4)
    Option      "MapButton1"        "button=1"
    Option      "MapButton2"        "button=2"
    Option      "MapButton3"        "button=3"
    Option      "MapButton4"        "none"
    # Rear buttons: back and forward
    Option      "MapButton5"        "key=166"   # XF86Back
    Option      "MapButton6"        "key=167"   # XF86Forward
    # Triggers: no use
    Option      "MapButton7"        "none"
    Option      "MapButton8"        "none"
    # Special buttons
    Option      "MapButton9"        "key=135"   # select : Menu
    Option      "MapButton10"       "key=172"   # start : XF86AudioPlay
    Option      "MapButton13"       "key=180"   # home : XF86HomePage
    # Sticks click: click
    Option      "MapButton11"       "button=1"  # clic left stick
    Option      "MapButton12"       "button=3"  # clic right stick
EndSection
It is commented, but here is what it does: It is worth noting that joysticks have one feature that mice do not have: they allow to scroll at a variable speed, proportional to the shift of an analogue stick. This is the meaning of these zx and zy axes I mapped to the second stick, and it provides a scrolling experience that is far superior from that of a mouse wheel.

20 July 2011

Lior Kaplan: Linux 20th anniversary T-shirts

I m organizing August Penguin, the yearly free software / open source conference in Israel for and by the community. We celebrate the 10th time we re holding this conference. I started going to these conferences just after high school, later I got more and more involved and now this is my 2nd year of being the main organizer (with a few others who provide great help). To celebrate the Linux 20th anniversary and August Penguins 10th anniversary, I thought to print copies of the Linux 20th anniversary T-shirts. I tired to contact the Linux Foundation to coordinate this, but got no response. I m writing this post to ask for help with finding the people who can approve this printing in behalf of the Linux Foundation. For those who can read Hebrew, the conference schedule is available at Hamakor s wiki. The rest are welcome to read the English version (provided by Google translate, so not totally accurate).
Filed under: Israeli Community

17 July 2011

Thorsten Glaser: mksh R40b; joe 3.1jupp16 and 2.8jupp2 released

mksh R40b (nowadays with filled in user s caveats (for R40, too!) and packager s upgrade hints) has just been released. This is a should-have upgrade, fixing a number of admittedly some obscure bugs, changing things begun in R40, improving upon others. Thanks to the PLD Linux guys for spotting all these errors; thanks to them and phpnet.org both for adopting mksh so well. I have also fixed a bug in nroff(1) which will lead to an even nicer looking HTML manpage mksh(1) (after the next rebuild and upload of a MirBSD snapshot scheduled RSN). jupp 3.1.16 took on the task of merging Debian joe changes (aiming at an upload). I also split the jupprc file into three versions (2.8 generic/DOS, 3.1+jupp and 3.7/Unix) because of the differences in the baseline executables making rc files partially mutually incompatible (think Insert key), annoyingly warning (think syntax, hmsg), or less usable (joe s new menu system). jupp 2.8.2 is a companion to jupp 3.1.16 mostly because of the new help window character map Binaries for jupp should be updated RSN too. Considering Banja Luka is arriving quickly, the r in RSN should be taken with a few grains of salt. I ve also scheduled working on the pcc Debian package for the next future; updating lynx and maybe others like OpenSSH in MirBSD is also due; cvs(1) will receive more of my time, but before the next Upload I d like to fix LP#12230 once verified. Builds for Debian/m68k are also still running. I note I did in fact not manage to make a new base image, yet (but 2.6.39 kernels miss a patch, anyway, so waiting for 3.0 is ok). It s still using gcc-4.4 because nobody tests gcc-4.6 and gcj-4.6 FTBFS due to SIGSEGV, but that s ok in my books. rsyslog is broken but sysklogd works. The #ksh Freenode page finally got a well-deserved link to Planet Commandline. Throw more my way! Acronyms and translations, too. (Got Norwegian and Rumanian covered in the meantime. No idea whether any RTL languages will work in that beast. But I m young and need the money) Since I m writing a wlog entry anyway let me thank Gunnar for a nice summary on the current Free Culture discussion; my comments on Nina s site seem to be eaten, but let me support it fully, although, of course, I normally use a copycenter style licence, which is specifically written for general works of authorship under copyright law, not limited to software. I did in fact have that in mind. Maybe some people will like it (it s less than one Kibibyte long) either generally or just for their everyday random musings (they can then keep CC-BY-SA for the big works if they so desire). Wouter, grass background makes green headlines illegible. I ve never liked, and never installed manually, cups either. (Benny tells me that Apple s new version refuses to talk with a non-Apple cups, kinda defeating the whole idea I think.) Port 9100 is JetDirect (probably with an HP in front and some subset of trailing) and just nice. (Being able to talk ESC/P with your printer like print '\033K\x07\0\x3E\x81\x99\xA5\xA5\x81\x3E' >/dev/lpa too rocks though, IMHO. Yes, mine can, and I still can. /dev/lpa is BSD.)

Kai, thanks for your vimrc lines:

	:highlight TrailWhitespace ctermbg=red guibg=red
	:match TrailWhitespace /\s\+$\  \+\ze\t/
 

Automatic removal is harmful, though I just fell into the trap since jupprc contains needed whitespace at EOL but manual removal (bound to ^K] in jupp) rocks. And I like that your solution uses such strong a colour vim users are the single most represented offender group for actually leaving the redundant whitespace at EOL there, and it should hurt their eyes. (Sadly there is some vehement disagreement preventing them from inclusion in grml-etc-core but that s why I re-post them here.) Ah, and jupp can of course display whitespace visibly (although it uses / , replacing the arrow with if no UTF-8, not ), accessible with ^Ov. Steve, want to put up a checklist for sites? We can crowdsource the testing to maybe get some interesting results Some other people would get more comments if they were idling in IRC (Freenode) or allow comments on their blog, specifically without too high an entrance barrier OpenID is ok, but many other things, and ECMAscript, are not; but I can t really say that loud because our wlog is static HTML compiled from a flat plaintext data source so it doesn t allow such either. I often forget what I wanted to add if I can t get it out quickly enough (especially at work). Sowwy Me like the cat picture postings (Amayita, Tiago, Gracias!).

12 July 2011

Gunnar Wolf: Free Software must migrate to become Free Culture

Cineast and Free Culture activist Nina Paley wrote some days ago a rantifesto on why the FSF has a double standard: Why are the Freedoms guaranteed for Free Software not guaranteed for Free Culture?, by not following its own very strict rules on software when it comes to culture as a whole. Her post was widely circulated, and got (at least) one reply by fellow Debian Developer Wouter Verhelst, largely agreeing with her, and an anti-rantifesto by Joe Brockmeier Which was promptly answered again by Wouter with a very fun and inspired post, written from the right angle: From the viewpoint of a person who is both a programmer and a musician, and understands the concepts at hand. I'd love to write a longer, better thought post But I'm tired and frankly stressed by many things, so I am just echoing their very interesting discussion to other people who might want to read it. I have been thinking and writing bits on that subject over the last couple of months. An example of that was the talk I gave at the Senate ~6 weeks ago. Following that talk, I wrote a short article for Revista Z calo (a widely circulated magazine mainly dealing with Mexican politics and social issues) called simply Software libre, cultura libre (full text available, but in Spanish only You can try reading an automated translation if it suits you). I wrote the article, mind you, with very limited time, and I'll be the first to recognize the prose was quite poor this time :( Anyway My point is that our nature is to share culture, to build it in a collaborative fashion, and having the Internet as a practically zero-cost, zero loss medium with which we can interchange our creativity with other like-minded people will naturally boost creativity. Free Software emerged before other Free Culture groups just because programmers had privileged access to Internet in the 80s and early 90s; as network access and digital creation tools have got to more people, it's just natural for all kinds of free culture to grow. Software is just a form of knowledge. Code is just a notation for a certain kind of ideas, just as the mathematical or musical notations. I believe (and hope) it's just unavoidable for us all to eventually switch to a mainly free cultural creation system.

1 July 2011

Wouter Verhelst: New toy

My previous laptop was due for a replacement. It's not that old yet only two years but it was getting on my nerves. The (second!) battery only gave me 45 minutes anymore, the wireless had issues, and I never really liked the processor note the lack of the VT-x extension; at the time when I bought this laptop, I wasn't aware of the fact that Intel was segmenting their products based in processors with and without hardware virtualisation. Originally I was going to buy a Toshiba R700-14J, but after I'd been waiting for far too long, it turned out that it had been EOL'd. So eventually, I succumbed to the dark side and bought me a Lenovo Thinkpad X220 instead. I've never been a fan of thinkpads before; I don't like what they look like, especially not the earlier models. But since I didn't want to wait a very long time again, I decided to go instead for a model that my suppliers' extranet site told me they had in stock, which kindof limits your options. The model I got is one with an SSD, which is new for me. I'd previously been holding off on getting an SSD until the technology had matured a bit, having heard some horror stories in the past about how some earlier models would run very slow after a while, but I guess it's safe to say that kind of problem should now be past us. It's crazy how fast this SSD is. The original 5 second delay which grub implies almost doubled the boot time. I notice that the 'networking' initscript no longer waits for a DHCP request to complete, which has the rather funny result that sometimes when I log in, I don't have networking yet, which means that libpam-krb5 can't do it's job properly, yet. There's one thing I'm not very fond of, and that's the display aspect ratio. 16:9 is just wrong. While I still prefer 4:3, I can live with 16:10. Very unhappy about that. There are still some laptops available that do have 16:10, but that would've required me to wait, which I didn't want to do anymore... Oh well.

29 June 2011

Jonas Smedegaard: The year of the FreedomBox

I am involved in developing something coined as the FreedomBox. Explaining it to my mum the other day, she wisely asks if it, albeit clearly an exciting challenge we've picked, really is doable? Surely the World has gone sour, but is such radical change even possible? Annoying question! And clever :-) For some years, tech media has tried predict when Linux have reached momentum for ordinary users. That current or next year was to become the Year of the Linux Desktop. Funny thing, seen in restrospect, is how "the year" kept being postponed, and when finally OLPC paved the way for the boom of Netbooks and arguably we got there, the World had moved on: Now Free Sofware is as common and as usable on desktops as commercially driven systems. It is taken for granted, not praised, and we look forward for the Next Big Challenge (as geeks) or Next Big Excitement (as users). Perhaps a similar fate is to be expected for FreedomBox: Initially when sparking our interest, and repeatedly since although we still today have nothing concrete to show - indeed even before we started hacking on it or knew the name of our dreams - our Prophet declared the Year of the FreedomBox. Not explicitly, but the cleverly phrased "right now." I am excited and proud to be working on FreedomBox, and foolishly hope it will be ready for worldwide consumption in a very recent "right now" - well aware that most likely it won't happen like that. Thing is, I don't really care how it happens, if only something does. This is due to the way we work: Hacking may appear from outside as larger projects, but really is juggling piles of small pieces for an eternal jigsaw puzzle, with each piece usable in multiple ways and across projects. I do not work only on FreedomBox, just as I did not work only on Sugar before that, or only with Debian as my platform: I work on Freedom-enabling technologies and ways to frame them for the Real World to use them with a vengeance. I sure hope you take the results for granted. That's true success!

19 June 2011

Leo 'costela' Antunes: Importing an Outlook PST into IMAP

Well, every once in a while we re forced to do something that isn t particularly interesting or pleasant. Last week it happened again: I had to import a few pretty big PSTs (most 2Gig, one 10Gig, with about 100.000 emails) into our dovecot IMAP. Doing this with Outlook itself was out of the question: it took way too long, even on the local network (many hours for a 1G file) and was prone to hanging and crashes, which were obviously a pain to debug and start over.
Thunderbird was unfortunately not much better, since at least in our tests it didn t import the read status of the emails (they were all marked as unread) and also wasn t particularly good at handling folders with strange names, containing . , / or some more obscure characters. We had used it before for smaller files, where manually dealing with the problems was acceptable, but this time it required something a bit more elaborate, if we were to keep our sanity. Enter libpst. It includes the handy readpst utility which dumps all emails in usable formats in a directory tree, one directory per folder. Unfortunately the Debian version is somewhat outdated and doesn t support the newer Outlook formats, so I did some packaging and even a little bit of patching. It seems Thunderbird also uses this library, which would explain why it didn t handle the Read-Status (haven t confirmed this though; just read it somewhere). The last step was this not-so-little script, which uses the dumped directories from readpst and imports them in IMAP. It would have probably been a bit more elegant to use libpst directly, but I unfortunately didn t have the time to mess around with that. I did have to mess around a lot with encodings though, ergo the unholy chaos with unicode()s and str.encode()s thrown around like rice at a wedding (I could never really wrap my head around charset problems; the subject boggles my mind to this very day).
#!/usr/bin/env python2.7
#-*- encoding: utf-8 -*-
 
import os, sys, time
import re
 
from argparse import ArgumentParser
from getpass import getpass
 
import imaplib
from email.header import decode_header
from mailbox import mbox
 
from twisted.mail import imap4 # for their imap4-utf7 implementation
 
parser = ArgumentParser(description="""Recursively import mbox files in a directory 
                                        structure to an IMAP server.\n
                                        The expected structure is that generated by 
                                        'readpst -r'.""")
parser.add_argument('-s', dest='imap_server', default='localhost', help='IMAP server to import emails to')
parser.add_argument('-u', dest='imap_user', required=True, help='user for logging in to IMAP')
parser.add_argument('-p', dest='imap_passwd', help="will be prompted for if not provided")
parser.add_argument('-c', dest='charset', default='utf8', help='charset in which the folders are stored (for versions older than 2003)')
parser.add_argument('-f', dest='force', action='store_true', help='import mail even if we think it might be a duplicate')
parser.add_argument('-m', dest='mappings', help='a JSON file with mappings between folder names and mailbox names (no slashes or dots)')
parser.add_argument('folder', nargs='+', help="the base folders to import")
 
args = parser.parse_args()
 
if not args.imap_passwd:
    args.imap_passwd = getpass()
 
 
if args.mappings:
    import json
    folderToMailbox = json.load(open(args.mappings,'r'))
else:
    folderToMailbox =   
 
 
 
def mailboxFromPath(path):
    paths = []
    for p in path.split(os.path.sep):
        p = folderToMailbox.get(p, p)
 
        # only other invalid char besides '/', which can't be created by readpst anyway
        p = p.replace('.','')
 
        paths.append(p)
 
    return '.'.join(paths)
 
def imapFlagsFromMbox(flags):
    # libpst only sets R and O
    f = []
    if 'R' in flags or 'O' in flags:
        f.append(r'\Seen')
    if 'D' in flags:
        f.append(r'\Deleted')
    if 'A' in flags:
        f.append(r'\Answered')
    if 'F' in flags:
        f.append(r'\Flagged')
    return '('+' '.join(f)+')'
 
def utf7encode(s):
    return imap4.encoder(s)[0]
 
def headerToUnicode(s):
    h = decode_header(s)[0]
    try:
        if h[1]: # charset != None
            try:
                return unicode(*h)
            except LookupError:
                return unicode(h[0],'utf8','replace')
        else:
            return unicode(h[0], 'utf8')
    except UnicodeDecodeError:
        try:
            return unicode(h[0], 'cp1252') # the usual culprits for malformed headers
        except UnicodeDecodeError:
            pass
 
        try:
            return unicode(h[0], 'latin1') # the usual culprits for malformed headers
        except UnicodeDecodeError:
            pass
 
        return unicode(h[0], 'ascii', 'ignore') # give up...
 
def main():
    imap = imaplib.IMAP4_SSL(args.imap_server)
    imap.login(args.imap_user, args.imap_passwd)
 
    imap.select()
    for base in args.folder:
        print "importing folder "+base
        for root, dirs, files in os.walk(base):
            if 'mbox' in files:
                folder = unicode(os.path.relpath(root, base), args.charset)
                mailbox = mailboxFromPath(folder)
                print u'importing mbox in  0  to  1 '.format(folder, mailbox)
                mailbox_encoded = utf7encode(mailbox)
                m = mbox(os.path.join(root, 'mbox'), create=False)
                for msg in m:
                    if imap.select(mailbox_encoded)[0] != 'OK':
                        print "creating mailbox "+mailbox
                        r = imap.create(mailbox_encoded)
                        if r[0] != 'OK':
                            sys.stderr.write("Could not create mailbox: "+str(r))
                            continue
                        imap.subscribe(mailbox_encoded)
                        imap.select(mailbox_encoded)
 
 
                    # skip possibly duplicated msgs
                    query = 'FROM " 0 " SUBJECT " 1 "'.format(
                                utf7encode(headerToUnicode(msg['from']).replace('"','')),
                                utf7encode(headerToUnicode(msg['subject']).replace('"',r'\"'))
                            )
                    if msg.has_key('date'):
                        query += ' HEADER DATE " 0 "'.format(utf7encode(msg['date']))
                    if msg.has_key('message-id') and msg['message-id']:
                        query += ' HEADER MESSAGE-ID " 0 "'.format(utf7encode(msg['message-id']))
 
                    r = imap.search(None, '( 0 )'.format(query))
                    if r[1][0] and not args.force:
                        print "skipping "+mailbox+": '"+headerToUnicode(msg['subject'])[:20]+"' (mid: "+str(msg['message-id'])+")"
                        continue
 
                    r = imap.append(mailbox_encoded, '', imaplib.Time2Internaldate(time.time()), str(msg))
                    if r[0] != 'OK':
                        sys.stderr.write("failed to import  0  ( 1 ):  2 ".format(msg['message-id'], msg['date'], r[1]))
                        continue
                    num = re.sub(r'.*APPENDUID \d+ (\d+).*', r'\1', r[1][0])
                    r = imap.uid('STORE', str(num), "FLAGS", imapFlagsFromMbox(msg.get_flags()))
 
                    if r[0] != 'OK':
                        sys.stderr.write("failed to set flags for msg  0  in  1 ".format(num, mailbox))
 
    imap.logout()
 
if __name__ == '__main__':
    main()
Another useful automation is using the mappings file, which looks something like
 
"Deleted items": "Trash",
"Sent items": "Sent"
 
to automatically import in the right destinations, in case you have a standardized structure.

16 June 2011

Raphaël Hertzog: Installing GNOME 3 on Debian 6.0 Squeeze? No, sorry

Ever since I blogged about the status of GNOME 3 in Debian experimental, my web logs show that many people are looking for ways to try out GNOME 3 with Debian Squeeze. No GNOME 3 for Debian 6.0 Don t hold your breath, it s highly unlikely that anyone of the Debian GNOME team will prepare backports of GNOME 3 for Debian 6.0 Squeeze. It s already difficult enough to do everything right in unstable with a solid upgrade path from the current versions in Squeeze But if you are brave enough to want to install GNOME 3 with Debian 6.0 on your machine then I would suggest that you re the kind of person who should run Debian testing instead (or even Debian unstable, it s not so horrible). That s what most people who like to run recent versions of software do. How to run Debian testing You re convinced and want to run Debian testing? It s really easy, just edit your /etc/apt/sources.list and replace stable with testing . A complete file could look like this:
# Main repository
deb http://ftp.debian.org/debian testing main contrib non-free
deb-src http://ftp.debian.org/debian testing main contrib non-free
# Security updates
deb http://security.debian.org/debian-security testing main contrib non-free
Now you should be able to run apt-get dist-upgrade and end up with a testing system. How to install GNOME 3 on Debian testing aka wheezy If you want to try GNOME 3 before it has landed in testing, you ll have to add unstable and experimental to your sources.list:
deb http://ftp.debian.org/debian unstable main contrib non-free
deb http://ftp.debian.org/debian experimental main contrib non-free
You should not install GNOME 3 from experimental if you re not ready to deal with some problems and glitches. Beware: once you upgraded to GNOME 3 it will be next to impossible to go back to GNOME 2.32 (you can try it, but it s not officially supported by Debian).
To avoid upgrading all your packages to unstable, you will tell APT to prefer testing with the APT::Default-Release directive:
# cat >/etc/apt/apt.conf.d/local <<END
APT::Default-Release "testing";
END
To allow APT to upgrade the GNOME packages to unstable/experimental, you will also install the following pinning file as /etc/apt/preferences.d/gnome:
Package: *gnome* libglib2.0* *vte* *pulse* *peas* libgtk* *gjs* *gconf* *gstreamer* alacarte *brasero* cheese ekiga empathy* gdm3 gcalctool baobab *gucharmap* gvfs* hamster-applet *nautilus* seahorse* sound-juicer *totem* remmina vino gksu xdg-user-dirs-gtk dmz-cursor-theme eog epiphany* evince* *evolution* file-roller gedit* metacity *mutter* yelp* rhythmbox* banshee* system-config-printer transmission-* tomboy network-manager* libnm-* update-notifier shotwell liferea *software-properties* libunique-3.0-0 libseed-gtk3-0 libnotify* libpanel-applet-4-0 libgdata11 libcamel* libcanberra* libchamplain* libebackend* libebook* libecal* libedata* libegroupwise* libevent* gir1.2-* libxklavier16 python-gmenu libgdict-1.0-6 libgdu-gtk0
Pin: release experimental
Pin-Priority: 990
Package: *gnome* libglib2.0* *vte* *pulse* *peas* libgtk* *gjs* *gconf* *gstreamer* alacarte *brasero* cheese ekiga empathy* gdm3 gcalctool baobab *gucharmap* gvfs* hamster-applet *nautilus* seahorse* sound-juicer *totem* remmina vino gksu xdg-user-dirs-gtk dmz-cursor-theme eog epiphany* evince* *evolution* file-roller gedit* metacity *mutter* yelp* rhythmbox* banshee* system-config-printer transmission-* tomboy network-manager* libnm-* update-notifier shotwell liferea *software-properties* libunique-3.0-0 libseed-gtk3-0 libnotify* libpanel-applet-4-0 libgdata11 libcamel* libcanberra* libchamplain* libebackend* libebook* libecal* libedata* libegroupwise* libevent* gir1.2-* libxklavier16 python-gmenu libgdict-1.0-6 libgdu-gtk0
Pin: release unstable
Pin-Priority: 990
Package: *
Pin: release experimental
Pin-Priority: 150
Note that I used Pin-Priority: 990 this time (while I used 500 in the article explaining how to install GNOME 3 on top of unstable), that s because you want these packages to have the same priority than those of testing, and they have a priority of 990 instead of 500 due to the APT::Default-Release setting. You re done, your next dist-upgrade should install GNOME 3. It will pull a bunch of packages from unstable too but that s expected since the packages required by GNOME 3 are spread between unstable and experimental.
If you want to read more articles like this one, click here to subscribe to my free newsletter. You can also follow me on Identi.ca, Twitter and Facebook.

13 comments Liked this article? Click here. My blog is Flattr-enabled.

10 June 2011

Gunnar Wolf: Reading revolutions: Online digital text and implications for reading in academe A (very informal) review

It's been a long time since I last took some time to read First Monday A great online publication, if you are not familiar with it, that I would categorize (and no, I'm not probably well-informed in it to be authoritative) as dealing with social, psychological aspects of the cultural shifts the online world has brought upon us (often dealing with topics related to Free Software communities, the reason I first met the publication). Firstmonday is an Open Access champion from early on. It follows an approachable but academic format (this means, it is peer-reviewed, its articles give extensive lists of references, and the articles are not the short reads we have got used to finding on the net, but, quoting from their audience profile, English is not the first language of many First Monday readers; A large percentage of First Monday readers are not a part of academia; Cultures, educational backgrounds, and fields of study vary greatly among First Monday readers.) This means, it's at least a great publication for me to follow :) Anyway After a long time not following it, I have just read Reading revolutions: Online digital text and implications for reading in academe, by Barry W. Cull; First Monday, Volume 16, Number 6 - 6 June 2011 Nice, interesting read. As I was planning on telling about the article to a couple of friends more into the subjects than myself, I'll comment+quote some bits on it. Before going any further: The article makes several references to Maryanne Wolf. No relation to her I'm not lulling my (two? are you both still reading?) readers towards her work ;-) The article talks about the differences social, even dips into some physiological aspects that the activity of reading is sustaining due to the shift from an activity done mainly from books (or other similar printed material) to the computer screen. Of course, we all know from our own experience many of the basic traits Shorter attention spans, a different reading pattern (skimming instead of reading; browsing through several related items instead of in-depth reading a single text as a knowledge unit). The article begins with an overview of reading and humankind. Cull quotes Maryanne Wolf's phrase, despite the fact that it took our ancestors about 2,000 years to develop an alphabetic code, children are regularly expected to crack this code in about 2,000 days . An interesting point I never thaught of is the start of reading as a purely mental activity, detaching reason from verbalization, ~1200 years ago:
Interestingly, these early scribes first did their work by reading out loud to themselves. Not until the ninth century did monastic regulations begin requiring silent reading. By the thirteenth century the practice of men reading silently and alone became commonplace. This shift to silent reading was a profound change, one that Darnton suggested involved a greater mental adjustment than the shift to printed text
. One last important point in this older history, that I'm quoting because I know I'll need the reference later on for one of my texts is about reading as a social activity Yes, also related to the quietness I just mentioned:
Interestingly, with the democratization of the printed text, there was a return to reading aloud. Reading was a solitary silent process only for the educated elite who could afford to buy books. For the rest of the population, as Darnton pointed out, reading was a social activity which took place in workshops, barns, and taverns [and] while children played, women sewed, and men repaired tools.
After this introduction (obviously one of the most interesting parts to me), Cull gives numbers showing how reading is evolving (in the USA and Canada), and quotes some prediction on how the future will end up adapting. Of course, I live in a place with a very different society, so I cannot comment much. Then he confronts some studies regarding specifically leisure reading, as it is a much more trustable factor than just literacy (in a world as highly literate as ours is, many people only read when they have to and have never or very seldom experienced the pleasure of reading just for the sake of it), bringing into the discussion the Internet (and computers in general) usage patterns. I found also very interesting the next section, regarding the pattern changes many libraries are facing now, specially academic/research-oriented libraries:
For several millennia, right up until just two decades ago, the central role of a library was to collect and house physical texts: from clay tablets, to scrolls, to printed books (Battles, 2003; Manguel, 2006). While printed text remains essential to most academic libraries, today s libraries have also become a core conduit via which researchers access scholarly texts online. Just within the last few years, Canadian academic libraries, in a situation similar to libraries throughout the Western world, have reached an interesting tipping point librarians now spend the majority of their collections budgets on electronic instead of printed texts.
( )
Libraries are convinced that digital text, now in its infancy, is likely to have a long future. Not only do they purchase electronic texts, but most academic libraries have also become publishers of electronic texts, whether they are digitizing large portions of their book holdings, or focusing on scanning a relatively small number of archival documents from their unique special collections.
And yes, doing some work with our Institute's library, I can confirm this trend. About e-books: I have got quite into that topic since the Kindle won my heart (and my money!) half a year ago. The little device completly changed my reading habits, I have read lots more since I carry it. And yes, I have never considered a full tablet-like device The article talks long about the difference, about the disadvantage that multitasking means to the human brain (oh, do I suffer from it!) I liked this snippet, quoting Steve Jobs a couple of years ago, regarding an Apple e-book reader question:
When Apple was rumoured to be working on an e book reader a few years ago, CEO Steve Jobs expressed his lack of interest: It doesn t matter how good or bad the product is, the fact is that people don t read anymore, he reportedly said, continuing by stating that 40 percent of the people in the U.S. read one book or less last year. The whole conception is flawed at the top because people don t read anymore (Markoff, 2008). Nicholas Carr (2010) has summed up Apple s involvement in the tablet phenomenon this way: Jobs is no dummy. As a text delivery system, the iPad is perfectly suited to readers who don t read anymore .
The issue for me is, I do enjoy reading, but I am an information addict. I know that if I have parallel information flows, my attention will surely dilute between them. Of course, as I read this article on-screen, it was hard for me to take the needed discipline not to be distracted by IMs or IRC highlights during the whole reading (which was also as an excercise for myself ;-) ). I am surprised to see this on student preferences (and even more surprised to see this data comes from my university):
In a study of students at the Universidad Nacional Aut noma de M xico (UNAM), the majority of students preferred print, and 63 percent reported that they could bear reading a document on a computer screen for no more than one hour (Ram rez Leyva, 2003). When it comes to course textbooks, a marked student preference for paper over e books has recently been found (Woody, 2010).
As we approach the end, it talks about another important topics I have often tried (and often failed) to communicate to my users: That of paratext, the meaning of the different texts, covers, items, layouts, etc. that are not part of the text itself but do shape the way we face it. To some of us, this seems obvious. To others, it is so hard to understand It closes with two more topics I will refer to. One is the permanent connectivity. All the time, more people are connected virtually all of their waking time. This affects not only learning habits but priorities. Will this near constant access to information interfere with students desire to comprehend and remember information, necessary to the educational process of turning it into knowledge? Author and university business school lecturer Don Tapscott recently suggested that students might not have to stress about the details those you can check Finally, regarding the continuity and ellaboration found in texts that are each time more common He quotes Maryanne Wolf:
I am worried about kids who are immersed in digital culture. They will get to college and they will have been Twittering so much that they won t have the patience to read those really long cognitively convoluted and complex sentences. They may not have developed those rich networks which are required in order to read at a high level of sophistication. The effort is what we are going to lose. They are becoming not so much a lazy reader, but an atrophied reader
As you can see, not only this post is meant to tell my couple-of-interested-friends to read the article, but it's mainly meant as a mental placeholder for myself. I will be surely refering to some of these items.

7 June 2011

Vincent Bernat: Mobile browsing & content optimization

Since my laptop features WWAN connectivity, I purchased a 3G mobile broadband plan from SFR, a French mobile phone company. I took the cheapest one available: 250 MB per month for 10 .

Optimizing transparent proxy by SFR While browsing, I was curious about the fact that there were multiple accesses to non-public IP, like 10.141.0.4. I was browsing the mobile version of Le Monde, a French daily newspaper. I enabled Chromium developer tools and watched what happened in the Network tab. All images were served from IP like 10.141.0.4. Looking at the HTML source code, I discovered that <img> tags were rewritten to point to those images. The smaller one were identical to the original one but the larger ones (more than 50 KB) were of reduced quality (and therefore reduced size). <img> tags were not the only one to be rewritten. <style> and <script> tags that were pointing to external resources (with a src attribute) were also rewritten to include the CSS or the javascript inline. Before:
<script src="http://s1.lemde.fr/js/lib/jquery/1.5.1/jquery.js"></script>
After:
<script oldSrc="http://s1.lemde.fr/js/lib/jquery/1.5.1/jquery.js" NG="replaced">
/*!
 * jQuery JavaScript Library v1.5.1
 *
 * Copyright 2011, John Resig
 * Dual licensed under the MIT or GPL Version 2 licenses.
 * http://jquery.org/license
 * [...] */
</script>
This means that each page that includes jQuery, a popular javascript library, as an external resource, will be turned into a page that includes the whole jQuery code which is about 85 KB. Each page will weight more than 85 KB instead of relying on an external and cacheable resource. Nowadays, most browsers support HTTP keep-alive which allows to serve multiple resources into a single request. There are two cases:
  1. If the resource is served from the same server as the HTML page, keep-alive would enable to get almost the same result without hindering the cache: the browser would first request the HTML page, then the javascript resource into the same TCP connection. There is a small penalty of about 100 ms to let the browser request the second resource. The browser may decide to open another connection because the initial document is not loaded yet. Some browsers may even negotiate this additional connection before needing it. In this case, it can use this idle connection at no cost. Otherwise, we may fall in the second case.
  2. If the resource is served from another server (like in the example above) or if the browser does not have an idle connection ready for reuse, it would download the resource in parallel with the help of another connection. It will only costs about 300 ms of additional time due to the establishment of a new TCP connection with the server. Those 300 ms are a very small price to pay compared to the 2 seconds needed to download 85 KB for each page. Moreover, other static resources will likely be downloaded from the same server and therefore, this is 300 ms the first time and about 100 ms for additional resources (round-trip time for a new HTTP request into the same TCP connection).
UPDATED: In a previous version of this article, I confused pipelining and keep-alive mechanisms. While keep-alive allows to reuse an existing connection to download a new resource, only one request at a time by connection could be issued. Pipelining allows to push additional requests in the same TCP connection without waiting for answers. As noted by ric Daspet in a comment, pipelining is disabled for most browsers. However, keep-alive is well supported. Searching on the web for oldSrc and NG="replaced", I find very little information about what is used. It seems that such a feature may also break some scripts, like TinyMCE, trying to get additional resources relative to the path where the script is installed. It appears that SFR is using a product from Flash Networks called Harmony Gateway. I have asked SFR more info about this and if it was possible to disable the use of this feature but got no answer so far. I have retried a bit later and there was no rewriting for Javascript and CSS. Images were still degraded and served by a special server. The next day, my requests did not travel through a transparent proxy anymore (no more Via header in answers). Each try was done at a different location. The proxies may be installed at some locations only and configured differently. Another explanation is that SFR silently disabled this functionality for my account. If it was done correctly, such an accelerator service could be be proposed as an opt-out option since most users would appreciate faster page downloads. If you care about network neutrality, you can disable this option. Of course, it needs to be properly advertised, for example as a free option in your mobile plan.

Setting up our own optimizing proxy Google provides mod_pagespeed as a solution to optimize a website on the fly. This is an Apache module. It is targeted at optimizing your own website but since Apache can act as a regular proxy, it seems possible to enable mod_pagespeed for any website. That s pretty easy to setup if you don t mind using a precompiled version of mod_pagespeed. Download mod_pagespeed. I have downloaded the 64bit version for Debian.
# touch /etc/default/mod-pagespeed
# dpkg -i ~/download/mod-pagespeed-beta_current_amd64.deb
Selecting previously deselected package mod-pagespeed-beta.
(Reading database ... 277699 files and directories currently installed.)
Unpacking mod-pagespeed-beta (from .../mod-pagespeed-beta_current_amd64.deb) ...
dpkg: dependency problems prevent configuration of mod-pagespeed-beta:
 mod-pagespeed-beta depends on apache2.2-common; however:
  Package apache2.2-common is not installed.
dpkg: error processing mod-pagespeed-beta (--install):
 dependency problems - leaving unconfigured
Errors were encountered while processing:
 mod-pagespeed-beta
# apt-get install -f
[...]
Unpacking apache2.2-common (from .../apache2.2-common_2.2.19-1_amd64.deb) ...
Processing triggers for man-db ...
Setting up apache2.2-common (2.2.19-1) ...
[...]
# apt-get install apache2-mpm-worker
[...]
Unpacking apache2-mpm-worker (from .../apache2-mpm-worker_2.2.19-1_amd64.deb) ...
Setting up apache2-mpm-worker (2.2.19-1) ...
Starting web server: apache2.
# a2enmod proxy_http
Considering dependency proxy for proxy_http:
Enabling module proxy.
Enabling module proxy_http.
To to activate the new configuration, you need to run:
  /etc/init.d/apache2 restart
Then, we modify /etc/apache2/mods-enabled/proxy.conf:
ProxyRequests On
<Proxy *>
    AddDefaultCharset off
    Order deny,allow
    Deny from all
    Allow from 127.0.0.1
</Proxy>
Of course, this makes little sense to run the proxy on your laptop since you don t want to download the non-optimized content over your WWAN link. You need to run it on another server and therefore allow your laptop to access it. Also, we need to modify /etc/apache2/mods-enabled/pagespeed.conf to add the directive ModPagespeedDomain * to let mod_pagespeed rewrite any page. You should restart Apache after that. Now, check if the proxy is working:
$ curl -six http://localhost:80 http://www.lemonde.fr/   grep ^X-Mod-Pagespeed
X-Mod-Pagespeed: 0.9.17.7-716
All is working! Let s try a very simple site, like Debian Backports. It features one HTML file and two CSS files: Debian Backports website without pagespeed We downloaded a total of 8 KB (compression included). Let s use our proxy and try again: Debian Backports website with pagespeed mod_pagespeed succeeds in reducing the size of the HTML file and combined the CSS files. Moreover, it minified the resulting CSS file. There seems to be a bug in Chromium that display incorrect transfert sizes (including in the timeline representation) when using a proxy. I have checked with Wireshark that the overall quantity transferred was about 8 KB. The gain is very thin here but you can imagine that you will get better results on larger sites. mod_pagespeed seems to handle well most sites. However, it only optimizes pages that already are in its cache. The first time you grab a page, you will get the unoptimized version. I don t know if it is the expected behaviour.

Next.

Previous.