Search Results: "will"

11 November 2008

Daniel Burrows: aptitude 0.5.0 released

I've released version 0.5.0 of the aptitude package manager (release notes). Once the mirrors have synced, you should be able to download it from Debian's experimental distribution, or manually from the aptitude package page.

[Edit] This release was uploaded to experimental because it is not a final release. aptitude follows the Linux version numbering scheme in which an odd second digit indicates an unstable development version. DO NOT INSTALL THIS VERSION IF YOU WANT STABLE SOFTWARE; IT IS INCOMPLETE, INADEQUATE, AND PROBABLY INCORRECT. To reply to some questions that several people have asked me:

The final release of aptitude 0.6.0 will be split into two packages, one containing only the traditional text interface and named aptitude-curses, and one containing the GUI binary named aptitude-gtk, with both of them providing the aptitude binary through Debian's alternatives mechanism. This release wasn't split because the code to drop GTK+ support was buggy and I discovered this fact halfway through assembling the release. It's fixed now.
To run the traditional curses interface, pass the --no-gui command-line parameter or set the option Aptitude::Start-Gui to false in /etc/apt/apt.conf. Unless you want to test that the curses frontend isn't broken, though, there won't be much for you in this release: the only change affecting it is Xapian support, and that currently breaks incremental search. (see the note above about this being an unstable release)

Also, another feature that I suspect might be bothersome is the behavior of updating the package lists and downloading changelogs on startup. There's no switch for changelogs yet, but you can disable the package list update by setting Aptitude::Update-On-Startup to false.

The main change in this release is the new GTK+ frontend, designed and implemented by Obey Arthur Liu with funding from the 2008 edition of Google's Summer of Code.

The new frontend is is an effort to bring some of the design principles of the curses frontend to a GUI environment, while also exploiting the unique features a GUI gives us and exploring ways to deal with changes in the environment in the nine years since aptitude was first designed. For instance, it is no longer reasonable for the user to actually read the entire package list (when I first installed Debian, you could read through the whole package list in a single sitting). So instead of basing the interface around a list of all the packages, we based it around the ability to search for the packages you're interested in. This version of aptitude also introduces Xapian searching:

daniel@emurlahn:~$ aptitude search "apt package manager"
i   apt                                  - Advanced front-end for dpkg
p   apt-dater                            - terminal-based remote package update manager
p   apt-dater-dbg                        - terminal-based remote package update manager (d
i   aptitude                             - terminal-based package manager
i   aptitude-dbg                         - Debug symbols for the aptitude package manager
p   createrepo                           - generates the metadata necessary for a RPM pack
p   gnome-apt                            - graphical package manager
p   smartpm                              - An alternative package manager that works with
p   smartpm-core                         - An alternative package manager that works with
i   synaptic                             - Graphical package manager
i A update-manager-core                  - APT update manager core functionality

You might ask why I didn't search for just package manager. The reason is simple: aptitude doesn't yet sort by relevance, and that second search gave me screenfuls of packages whose description contains both package and manager, including addressmanager.app (a PIM for GNUstep), compiz-gtk (a piece of eye candy), and wterm (an X terminal emulator). However, it is worth noting that Xapian searches are fully integrated into the aptitude search language, so they can be combined with non-Xapian search terms in just the way you would expect:

daniel@emurlahn:~$ aptitude search "?installed apt package manager"
i   apt                                  - Advanced front-end for dpkg
i   aptitude                             - terminal-based package manager
i   aptitude-dbg                         - Debug symbols for the aptitude package manager
i   synaptic                             - Graphical package manager
i A update-manager-core                  - APT update manager core functionality

So, all the basic functionality that the GUI version needs is in place; the next step is to start polishing it and filling in the gaps that are left. And, of course, suggestions and bug reports are welcome, so we know where the rough spots and the gaps are. Please send your comments to aptitude-devel@lists.alioth.debian.org so that they are part of the public record and so that people besides me can read them. :-)

Happy package managing!

Biella Coleman: Steal this Film

For tomorrow’s class we are going to be discussing piracy and we will do so with the help of Alan Toner one of the filmmakers behind Steal this Film. We will be reading some of his work (and this ethnographic look at the life of a p2p file sharer provides a refreshing look at what it actually looks like), as well as the work of Matt Mason among others. We will also be reading an oldie but goodie that I could not find easily online and given that it is a class on piracy, I thought I would honor the class (and the piece) engage in a little transgression. If find the piracy exhilarating, daunting, questionable, or reproachable, come to NYU Free Culture’s screening of Steal this Film part 2, this Sunday Nov 16th at 7 PM. Alan will also be there to field questions and kick off the debate. I am looking forward to the post-movie discussion.

David Welton: Another "Mysql Doesn't Do that?!" Moment

http://adam.blog.heroku.com/past/2008/9/3/ddl_transactions/

I think the real reason there was resistance to this change is that MySQL doesn't support DDL transactions; and wrapping migrations in a transaction is mostly pointless without this feature. Since MySQL doesn't support it, most Rubyists don't know what DDL transactions are - so here's a quick primer on this incredibly useful and certainly underrated feature. DDL = data definition language, also known as the schema. DDL commands include CREATE TABLE, DROP TABLE, ALTER TABLE, and CREATE INDEX. DDL transaction support means that the following sequence of SQL commands will not modify your database

I never even thought of that. I was fairly confident in my vision of Postgres getting faster and faster, and Mysql become a "real" database... that the two systems were converging on something resembling a common speed/feature set. But things like this still occasionally rear their ugly heads and I'm happy to be a Postgres user.

Joey Hess: spelling policy

Effective immediatly -- If you're annoyed by the spelling of something I've written, I will accept a patch -- preferably generated by git-format-patch, but diff -u is also acceptable. Any other communication about spelling mistakes will be ignored, unless the mistake has ramifications that will cause undue pain and suffering to people who are not English majors. Furthermore, if the spelling "mistake" is that I spelled "-ize" as "-ise", it's not a mistake -- I prefer to use the latter form for obscure reasons, with a few exceptions. PS, I realise that these are entirely arbitrary rules forced upon you willy-nilly. Teh irony..

Clint Adams: Bowling for madducks

A long time ago, I had the idea of improving the interface between shell completion and the programs being completed. The result of this was the bzr shell-complete command (or bzr s-c for short), which was never fully fleshed out, and has since fallen into disrepair. The principles behind this are + the program is the best place to store up-to-date and accurate information + the program already knows all these things (albeit usually in unparseable forms) + duplicating information and effort is annoying Here is an excerpt from bzr s-c, which was intended to give a comprehensive list of subcommands, paired with short descriptions:

diff:show differences in the working tree, between revisions or branches
export:export current or past revision to a destination directory or archive
get:create a new copy of a branch
help:show help on a command or other topic
ignore:ignore specified files or patterns
ignored:list ignored files and the patterns that matched them
info:show information about a working tree, branch or repository
init:make a directory into a versioned branch

There is one subcommand per line, separated from its description by a colon. Next you can invoke something like bzr s-c diff to get the possible options and arguments for the diff subcommand, although the output you would see today is broken and nearly useless. Since I've lost faith in bzr, I'll illustrate what the output might be corresponding to topgit's tg remote if topgit supported this kind of thing:

--populate
REMOTE

This would mean that tg remote can understand the option --populate, which takes no argument, and that the first non-option argument should be a REMOTE. REMOTE would then be defined, for example, in zsh's _topgit function as some kind of git remote which is completed in the same way you might complete a git remote for git. The exciting part then, is that if tg remote starts taking a --decimate option, the topgit completion helper subsystem will start outputting it and _topgit will do the right thing without having to be altered. For tg export, things are a bit more complicated, so let's have it be described in the style of the zsh completion system:

'(--collapse)--quilt:directory:_directories'
'(--collapse -b --branch)' -b,--branch= ':branches:BRANCHES'
'(--quilt)--collapse:branch:BRANCH'

This means that --collapse and --quilt are exclusive, that -b and --branch cannot be used with --collapse, that -b and --branch are equivalent, that -b and --branch take an argument in the form of BRANCHES, that --collapse takes an argument in the form of BRANCH, and that --quilt takes an argument that's a real directory in the filesystem. Then _topgit would have logic to interpret BRANCH as a branch, and BRANCHES as a comma-separated list of branches. A similar idea is the one used by axp. If you invoke axp self completion zsh, it will output zsh completion functions for you. To me this seems more onerous on both the developers and the end users, but I suppose it gives you immediate flexibility that a more generic interface would lack.

Michal Čihař: Gammu test version 1.21.91

Good news everyone, new Gammu testing version is out. Fixes are there for everybody, starting in AT engine initialization, going through FBUS and Nokia drivers up to OBEX handling in Siemens phones. Full list of changes:

Script gammu-config can now also work with whiptail.
Minor fixes in gammu-config script.
Extra spaces from connection or device name are now stripped.
Automatically reinitialize FBUS connection if it is dropped.
Disable OBEX with Motorola Z6c, it is broken (bug #746).
Refresh manufacturer information on second init of AT (bug #746).
Always get charset information on AT phone initialization.
Better handling of OBEX switching for Siemens phones.
Fix flags for 2630 and 3109c (bug #752).
Fix handling of caller groups and picture IDs (bug #752).
Properly set calendar location while adding (bug #747).

You can download from usual place: http://cihar.com/gammu/, Debian users will find packages in experimental soon.

10 November 2008

John Goerzen: Looking back at Wordpress

I've hosted this blog on three different platforms: Drupal, Wordpress, and at present, Serendipity.

Back in 2006, I rejected Wordpress, noting that most of its plugins were incompatible with the current version, its main anti-spam software wasn't Free, there was no central plugin directory. And, while Wordpress supported PostgreSQL, many plugins didn't.

Serendipity, at the time, had none of those problems.

However, I've been having other problems with Serendipity since then. People have repeatedly had trouble with captchas. The RSS feeds have long had subtle incompatibilities with certain aggregators, leading to duplicate posts.

I'm looking back at Wordpress now. It looks like it is a lot more mature than it was 2.5 years ago. Perhaps it's time to switch back.

I hope it will support PostgreSQL better now, but I note that its website seems to list MySQL only these days. Ah well, can't have it all, I guess.

Kurt Roeckx: Finding which sector belongs to which file on a RAID device.

I got this nice message (and some others) in my log today:
smartd: Device: /dev/sdd, 2 Currently unreadable (pending) sectors After running a smartctl -t long /dev/sdd smartctl now reports:

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0032   089   089   000    Old_age   Always       -       8719
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0012   200   200   000    Old_age   Always       -       2
198 Offline_Uncorrectable   0x0010   200   200   000    Old_age   Offline      -       0
[...]
SMART Error Log Version: 1
ATA Error Count: 10 (device log contains only the most recent five errors)
[...]
Error 10 occurred at disk power-on lifetime: 8692 hours (362 days + 4 hours)
  When the command that caused the error occurred, the device was active or idle.
  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 78 e5 89 e0  Error: UNC 8 sectors at LBA = 0x0089e578 = 9037176
  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  25 00 08 78 e5 89 2f 00  45d+06:36:27.880  READ DMA EXT
  25 00 08 70 e5 89 2f 00  45d+06:36:27.798  READ DMA EXT
  25 00 08 68 e5 89 2f 00  45d+06:36:27.308  READ DMA EXT
  25 00 08 e0 77 e5 35 00  45d+06:36:27.295  READ DMA EXT
  25 00 08 60 e5 89 2f 00  45d+06:36:26.653  READ DMA EXT
[...]
Error 9 occurred at disk power-on lifetime: 8692 hours (362 days + 4 hours)
  When the command that caused the error occurred, the device was active or idle.
  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 08 60 e5 89 e0  Error: UNC 8 sectors at LBA = 0x0089e560 = 9037152
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed: read failure       90%      8715         797566305
# 2  Extended offline    Completed without error       00%      8198         -

Notice that 197 Current_Pending_Sector indicates that there are 2 sectors with a problem. Note that the SMART error log shows LBA sectors 9037176 (0x89e578) and 9037152 (0x89e560) but it's limited to 0xffffff. The self test logs shows us the proper LBA error at 797566305 (0x2f89e561) which has a 0x2f in front, and also shows us that it's the second sector in the block of 8 we tried to read. The kernel log also shows:

ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
ata4.00: (BMDMA2 stat 0xc0009)
ata4.00: cmd 25/00:08:60:e5:89/00:00:2f:00:00/e0 tag 0 cdb 0x0 data 4096 in
         res 51/40:00:61:e5:89/00:00:2f:00:00/f0 Emask 0x9 (media error)

So I've tried to reproduce the error with:

# dd if=/dev/sdd of=797566305 skip=797566305 bs=512 count=1 iflag=direct
1+0 records in
1+0 records out
512 bytes (512 B) copied, 1.27727 s, 0.4 kB/s

Notice that it can actually read that block, it just seems to take a while. And the error log now contains:

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 01 61 e5 89 e0  Error: UNC 1 sectors at LBA = 0x0089e561 = 9037153

Because it's part of a RAID device I could just let the whole disk resync, but I was a little curious which files were in it. So I found this document that explains on how to find which files has the problem. But it only has examples covering ext2/ext3 on a partition and LVM and nothing about RAID device. So it looks like someone using RAID 0 will have a hard time finding the file that's having a problem. So following the document we start with:

# fdisk -lu /dev/sdd
Disk /dev/sdd: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x000578f9
   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1   *          63      385559      192748+  fd  Linux raid autodetect
/dev/sdd2       972864270   976768064     1951897+  fd  Linux raid autodetect
/dev/sdd3          385560   972864269   486239355   fd  Linux raid autodetect
Partition table entries are not in disk order

In /proc/mdstat we see:

md1 : active raid5 sda3[0] sdd3[3] sdc3[2] sdb3[1]
      1458717696 blocks level 5, 64k chunk, algorithm 2 [4/4] [UUUU]

So sdd3 is part of md1 and I have an ext3 filesytsem on that. Tune2fs -l grep Block gives:

Block count:              364679424
Block size:               4096
Blocks per group:         32768

mdadm tells me:

# mdadm -QE /dev/sdd3
/dev/sdd3:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 747b071a:a70f723b:c3a9ed44:15845cbf
  Creation Time : Mon Nov 12 20:13:51 2007
     Raid Level : raid5
  Used Dev Size : 486239232 (463.71 GiB 497.91 GB)
     Array Size : 1458717696 (1391.14 GiB 1493.73 GB)
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 1
    Update Time : Mon Nov 10 10:57:28 2008
          State : clean
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0
       Checksum : 4b3147f4 - correct
         Events : 78
         Layout : left-symmetric
     Chunk Size : 64K
      Number   Major   Minor   RaidDevice State
this     3       8       51        3      active sync   /dev/sdd3
   0     0       8        3        0      active sync   /dev/sda3
   1     1       8       19        1      active sync   /dev/sdb3
   2     2       8       35        2      active sync   /dev/sdc3
   3     3       8       51        3      active sync   /dev/sdd3

I've been told that 0.90 stores the superblock at the end of the device, so that makes it a little easier. And this seems to agree:

# cat /sys/block/md1/md/dev-sdd3/offset
0

We have a left-symmetric layout, so in raid5_compute_sector() we see

	stripe = chunk_number / data_disks;
	*dd_idx = chunk_number % data_disks;
[...]
	case ALGORITHM_LEFT_SYMMETRIC:
		*pd_idx = data_disks - stripe % raid_disks;
		*dd_idx = (*pd_idx + 1 + *dd_idx) % raid_disks;

So it looks like the chunks are located on the disk like:

stripe	sda	sdb	sdc	sdd
0	D ₀	D ₁	D ₂	P _0-2
1	D ₄	D ₅	P _3-5	D ₃
2	D ₈	P _6-8	D ₆	D ₇
3	P _9-11	D ₉	D ₁₀	D ₁₁
4	D ₁₂	D ₁₃	D ₁₄	P _12-14

With each of those 64K. The problem is at 797566305 and the partition started at 385560. So that's at sector 797180745 inside partition sdd3. Each sector is 512 bytes and a chunk is 65536 bytes which gets us 128 sectors per chunk. If we want to know which the stride the sector is on we do: 797180745/128=6227974.5703125 This means that the stripe is number 6227974 and that it's sector 0.5703125*128=73 on sdd3 within that chunk. The parity is on disk: pd_idx = 3 - 6227974 % 4 = 1, being sdb. To get the chunk we need to multiply with the number of data disks (3) and since it's the forth disk and the parity is on the second disk we need to add 1 and get: 6227974*3+1=18683923 So calculating that back to sectors we multiply it with the number of sectors per chunk again and add our offset in the chunk: 18683923*128+73=2391542217 To test that our math is any good we try:

# dd if=/dev/md1 of=797566305.2 skip=2391542217 bs=512 count=1 iflag=direct
1+0 records in
1+0 records out
512 bytes (512 B) copied, 5.99103 s, 0.1 kB/s

We see an error message in the kernel log, there are 2 new errors in the error log, so it seems that everything is calculated correctly. The files are also identical. The file system is in blocks of 4096 bytes, so for the file system this is block 2391542217*512/4096=298942777.125 Then we use debugfs:

# debugfs
debugfs 1.41.2 (02-Oct-2008)
debugfs:  open /dev/md1
debugfs:  icheck 298942777
Block   Inode number
298942777       <block not found>

Which isn't making any sense to me, since I get that error about every 2 hour when a certain cronjob runs. 298942777 / 32768 (Blocks per group) = 9123.00955. So I wonder if ext3 is keeping some meta data there and debugfs doesn't tell that? Anyway, at some point I was running a smartctl -t long /dev/sdd again because I couldn't reproduce the error anymore, the cronjob started, and I then found alot of errors in my log again and now it included:

raid5:md1: read error corrected (8 sectors at 797180744 on sdd3)

It's so much fun that disks always behaves the same trying to read the same sector. Smartctl now reports:

  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0012   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   200   200   000    Old_age   Offline      -       0

I assume the kernel removed the pending sector by writing to it. The other pending error was already gone for some time, I assume new data got written to that one.

Uwe Hermann: OpenOCD, a Free Software JTAG utility with ARM and MIPS support

Just FYI, I've recently updated the OpenOCD Debian package in unstable. OpenOCD is a Free Software JTAG utility which currently supports quite a large number of JTAG adapters and various CPUs/targets (many ARM and now also some MIPS ones). It's being used by a number of Free Software related projects such as OpenMoko and many others. Here's an example of how you usually use the (new) OpenOCD with a cheapo parallel port JTAG device. First, start the OpenOCD server, providing it an interface config file and a target config file (you can copy/adapt them from /usr/lib/openocd/ interface,target /*.cfg, or use those files directly if they work for your target, of course).

  $ openocd -f parport.cfg -f lpc2148.cfg

Then, in another xterm for example, connect to the now-running OpenOCD telnet server. Here you can now run various commands to probe, control and program the JTAG device(s). Try help for a list of commands. As an example, for flashing a binary onto some LPC2148 eval board you would do something like this:

  $ telnet localhost 4444
  Trying 127.0.0.1...
  Connected to localhost.
  Escape character is '^]'.
  Open On-Chip Debugger
  > reset init
  JTAG device found: 0x4f1f0f0f (Manufacturer: 0x787, Part: 0xf1f0, Version: 0x4)
  srst pulls trst - can not reset into halted mode. Issuing halt after reset.
  target state: halted
  target halted in Thumb state due to debug-request, current mode: Supervisor
  cpsr: 0x800000f3 pc: 0x7fffd2a2
  requesting target halt and executing a soft reset
  target state: halted
  target halted in ARM state due to debug-request, current mode: Supervisor
  cpsr: 0x800000d3 pc: 0x00000000
  > flash write_image /home/foo/program.bin 0
  wrote 1236 byte from file /home/foo/program.bin in 0.533683s (2.261701 kb/s)
  > resume 0

The final resume 0 will start to execute your program on the ARM LPC2148 microcontroller. Check out the openocd info page (info openocd on the command line) for lots more documentation.

Benjamin Mako Hill: Wikimedia and GFDL 1.3

I spent more time than I would like to admit massaging the process that ultimately led to the release of the the GNU Free Documentation License 1.3 (GFDL) by the Free Software Foundation (FSF). Hours counted, it was probably one of my biggest personal projects this year. The effect is to allow wikis under the GFDL to migrate to the Creative Commons BY-SA license or, as Wikimedia's Erik M ller has proposed, to some sort of dual-license arrangement. There are many reasons for this change but the most important is that the move reduces very real barriers to collaboration between wikis and free culture projects due to license compatibility. BY-SA has become the GPL of the free culture world and Wikimedia wikis were basically locked out from sharing with a larger community, and vice-versa; projects will no longer have to choose between sharing with Wikipedia and sharing with essentially everyone else. The GFDL has done a wonderful job of helping get Wikimedia projects to where they are today and M ller's proposed switch seems, in my opinion, the best option to continue that work going forward. The FSF gets a lot of credit (and a lot of flack) for what it does. Offering to "let go" of Wikipedia -- without question the crown jewel of the free culture world -- represents a real relinquishing of a type of political control and power for the FSF. Doing so was not done lightly. But giving communities the choice to increase compatibility and collaboration by switching to a fundamentally similar license was and is, in my opinion, the right thing to do. Everyone who has worked hard to make this happen deserves the free culture movement's thanks. This list includes Richard Stallman, Brett Smith and Peter Brown of the FSF; James Vasile and Eben Moglen of the SFLC; Erik M ller, Mike Godwin and Shunling Chen of the Wikimedia Foundation. The FSF in general, and RMS in particular, deserves a huge amount of credit for what it has decided to not do in this case and for giving up control in a way that was responsible and accountable to its principles and to GFDL authors and in the interest of free culture movement more generally. It has not been easy or quick. If you support or appreciate work like this, please support the FSF and express this while doing so. Doing so is an important way to support these essential and almost inherently underappreciated efforts.

Christoph Haas: screenshots.debian.net goes beta

Roberto Sanchez and I proposed to create a service to provide screenshots for Debian packages. And with a little work, a lot of Pylons frameworking power and a tad of jQuery spice I created the ‘debexpo’ software that provides a nice-looking web site doing exactly that. The service is already up and running at screenshots.debian.net. This is still the phase where I have to sort out a few minor technical, legal and organisational issues. And thanks to a friendly fellow Pylons’ user I have now a place to host the service (my vpslink-sponsored server is already overloaded). Once all the subleties are sorted out I will announce the service to debian-devel-announce and hope to get mentioned in the weekly news.

Biella Coleman: Epic Lulz will be had (aka: I am giving a talk)

Screenshot-coleman-scientology-slides - OpenOffice.org Impress -1, originally uploaded by the biella.

“Old and New Net Wars over Free Speech, Freedom and Secrecy or How to Understand the Lulz battle against the C0$ Gabriella Coleman, Assistant Professor, New York University Thursday November 13: noon - 2:00pm
Columbia University Communications Colloquium
270B IAB (International Affairs Building update Open the public! You may need ID to get into the building but I am not sure. This is the case with NYU…

Biella Coleman: Epic Lulz will be had (aka: I am giving a talk)

Screenshot-coleman-scientology-slides - OpenOffice.org Impress -1, originally uploaded by the biella.

Julien Blache: pommed v1.22: various improvements, Late 2008 laptops

I’ve just released pommed v1.22, which has been collecting improvements and reworks over the past few weeks; this release also adds partial support for the Late 2008 laptops. Partial support means the LCD backlight is not supported yet. The method for controlling the backlight has to be figured out; if you hear of anything, let me know. You’ll need a recent kernel for full support on these machines, which means either a late 2.6.28-rc or possibly an early 2.6.29-rc. It all depends on how fast the patches get applied to mainline. Patches are needed for the keyboard quirks, trackpad support, applesmc support and sound support, as usual. On the improvements front, wmpomme has become event-driven, eliminating the fixed-rate update and associated wakeups and visual ugliness in some configurations; pommed has got some code fixes, though nothing major; gpomme has got a new Japanese translation. Hopefully the next release will bring full Late 2008 support.

Wouter Verhelst: Belgian Griffins - Lille Vikings: 30-0

Soccer fans will see the above score as immense, but this isn't soccer; it's American Football. Yesterday, the Belgian National American Football team played a practice match against the Lille Vikings, and won by four touchdowns and a field goal. Unfortunately they missed one of their conversions, hence 30 points rather than 31. They actually even scored a safety in the final seconds of the match, but the officials did not count it. Still, that's a good score, so congratulations go out to all Griffins. The game took place on the new field of the city of Andenne, in Namur—in fact, it was part of the official opening ceremony—and will be the home field of the Andenne Bears team that plays in the LFFAB, the french-languaged American Football league of Belgium. This field is now probably the best field of its type in all of Belgium, and I imagine we will be seeing it more often from now on. Anyway, the main reason I was there was bifold. The first was called Joris Verhelst:

Number 85, my brother. The second was my D50. Unfortunately I didn't get a chance to get a good action shot of my brother. He's a Tight End, and did get his share of the game; but when he did, either I was too far away, or someone else was in the picture, or—as happened all too often yesterday, since the lighting was rather dim—the picture was too blurry to make out anything interesting. All in all, it was a nice day, and the ending was really great:

See you next time, Griffins!

9 November 2008

Adam Rosi-Kessel: Roslindale Scoop: Himalayan Bistro Coming to the Neighborhood

I have it on very good authority^* that our favorite Indian/Nepali restaurant, Himalayan Bistro, is coming to Roslindale (from West Roxbury), into the oft-turned-over location last assigned to NuVo, and Gusto before that. Perhaps the third “-o” restaurant will be a charm. Assuming they serve up the same superlative food and atmosphere as the West Roxbury location, I’m sure this one will be a keeper. (My only concern is that having the two restaurants so close to each other may result in self-competition.) ^*Authority = Himalayan Bistro delivery guy. You don’t get much more authoritative than that. In other Roslindale Restaurant news, we heard via RVMS recently that Robyn’s Bar and Grill has changed hands to the owners of the Halfway Cafe in Dedham, which could be a change for the better. Roslindale did not need another pizza/sports-bar, and still does not need one, although maybe that’s all the Robyn’s location will ever be. Even more recently, however, we heard an unconfirmed rumor from a neighbor that the sale fell through and Robyn’s is not changing owners. In other potentially disappointing news, the Roslindale Emac and Bolio’s has changed owners again, this time to a co-owner of the Blue Star Restaurant. I want to like Blue Star, but they can’t seem to get the most basic short-order cooking tasks right. A diner must, first and foremost, deliver greasy food fast, hot, and at the same time for the entire group. At worst, serve the kids first. It also helps for the food to be consistent. Blue Star in our experience (many visits) only occasionally satisfies the basic diner requirements, much less delivers any sort of creative urban-chic reinterpretation of the classic retro diner. (By contrast, Deluxe Town Diner in Watertown is our current Greater Boston favorite — it’s always a good sign when a greasy spoon also actively caters to vegans.) Which brings me to Emac and Bolio’s, which apparently will be renamed “Select Caf .” First, why give up the goodwill associated with the Emac’s trademark? Second, the initial signs of the change-over are not promising. A handwritten sign posted in the window announces that the cafe is under new management, and (1) no credit or debit cards will be accepted and (2) no Emac and Bolio’s gift certificates will be honored. I’d love to be proved wrong about this, but I’m pessimistic that this is going to be the Change We Need in Roslindale — namely, a top rate artisanale snobby espresso shop. We can only hope that Simon’s, Flatblack, Cafe Nation, Diesel, or another of their ilk will come occupy one of the vacant storefronts on the courtyard. That we have any vacancies at all in the square, much less several, is a crime against humanity. Technorati Tags: Roslindale, Restaurants

Adeodato Sim : Five films (#3)

After a summer during which I didn’t get to watch many movies, I’m back on track now. Here we go:

A Raisin in the Sun: this is a magnificent film, almost a masterpiece. Story of a family when a large sum of money comes to disrupt their lives. You should watch it.
Happiness: this one is definitely a masterpiece, but it’s quite likely you should not watch it: many people find it very disturbing. If you think you could take it, though, I say go for it.
There Will Be Blood: it is a good film, and it wasn’t a waste of my time, but I do believe it’s hyped. If you ask me, I think it’s so high in the IMDB list because of its last scene.
Ataque verbal: if you’re into Spanish cinema, and are fond of conversations, you may enjoy it quite much: a film consisting of 7 conversations. It isn’t perfect, but it’s different and cool (I’ve watched it twice, actually). The director, Miguel Albaladejo, has some other fine films as well.
The Remains of the Day: England, 1930; Anthony Hopkins is a butler, and Emma Thompson the housekeeper of the house. Ivory at his best. I had watched this film a couple years ago; having enjoyed it quite a lot, I went and read the book recently. Reading the book was a wonderful experience, so I decided to watch the film again. It is a very good and faithful script, but I think I enjoyed it this second time better. Regarding the book, I wrote some bits about it here; from now on I hope to write something about books I read in this space. By the way, the cover of the book has some amusement in it, see if you can spot it.

Simon McVittie: Spec-writing On A Plane

(Written on Friday 16th, posted on Monday)

As I write this, I'm on a plane off the coast of the Netherlands, on the way back from a couple of days designing APIs with Rob McQueen and the RTCom people at NRC Helsinki (who we work with on the chat and VoIP functionality for Nokia internet tablets).

We've had a very useful couple of days, mainly designing the next-generation channel requesting and dispatch API for Telepathy (the same APIs I've been doing preparatory work for on the Telepathy mailing list in the last couple of weeks). API sketches for review and comment are either on are on Merge Monkey or on the way there.

Before even reaching Nokia, Rob and I did a lot of design on the plane over to Helsinki, and I took some notes: here they are, by way of a glimpse at how the Telepathy design process works when we don't have a whiteboard[1] :-)

Geoloc looks good in principle, needs some more work

add more docstrings

add a way to ask people for uncached info (RequestLocations)

move access control to Co.I.Presence, call it RichPresenceAccessControl?

DeliveryReporting looks good, ish

improve wording of delivery-echo and rationale

when sending a delivery report, add a reason for the status (using Channel_Text_Send_Error?)

Uniqueness requirement for (channel, handle_type != 0, handle) is in the way - let's drop it

MSN should use "authentic" 1-1 chats, and use Ch.I.UpgradeableToRoom (name tbd) to migrate the underlying switchboard to be a room

XMPP should use Ch.I.UpgradeableToRoom too, but the 1-1 channel may also stay open

Mutable handle => handle 0 must stay (for requestotron's benefit)

bundles ftw

renaming: we want SelfHandleChanged in the core

TpC* - add ability to subscribe to ifaces ay construct time (blocks Ready) or later

Dispatching: when registering a c'handler, specify everything it can handle

If a new bundle can be handled in its entirety by the same handler (call it 'the bundle handler'), do it. Otherwise split it up completely (there is no 'bundle handler').

If a channel in an existing bundle can be handled by the bundle handler, send it there, else send to the default handler.

Group - add SelfHandleChanged, maybe? Also Connection

Add HandleOwners: prop a uu , HandleOwnersChanged(a uu , au)

Use TargetHandleType, TargetHandle

Stringified initiator handle as a separate property

Maybe even a FirstMessage property, although this stretches the definition of "immutable"

org.freedesktop.Telepathy.Client interface

Nokia's Mikhail Zabaluev kindly took notes for some further discussion we had with him and Alberto Mardegan about the dispatch API, which he already posted to the mailing list.

We hope this API will make it possible to integrate Telepathy into the desktop better - it should make it a lot easier to share connections between processes in a useful way and hand out incoming channels to the appropriate handlers, which was always a major goal for Telepathy. It should also let us and others implement exciting new notification mechanisms (hi to the people on IRC and the mailing list wanting to patch their media players to integrate with Telepathy! :-)

[1] When we do have a whiteboard, the design process is: "Robot101 scribbles on a whiteboard, smcv writes down <tp:rationale> elements"

Simon McVittie: Implementing D-Bus services with telepathy-glib

(Part 1 of an ongoing series: shiny things in telepathy-glib)

16:46 < epmf> tp-glib is made of magic
—#telepathy, 2008-04-15

Rob pointed out today that I'd been promising to blog about telepathy-glib for several months and still haven't done so. I think there's too much to cover in one blog post, so I'll start with the oldest part - implementing a service (typically a Telepathy connection manager).

While telepathy-glib is (obviously) intended for Telepathy implementors, I think the ideas we've been implementing are likely to be useful for any D-Bus API, particularly flexible/complex APIs where an object implements many interfaces.

Introduction to telepathy-glib telepathy-glib started out as a collection of code extracted from our XMPP connection manager, telepathy-gabble, but at some point it grew beyond that into a wrapper around/partial replacement for dbus-glib. We've been quite pleased with it, really :-) My first major round of work on telepathy-glib, during the first half of 2007, was to split it out from telepathy-gabble (versions up to 0.5.9 were released as part of Gabble 0.5.x tarballs, in fact), leading to the release of the 0.6.x stable branch in late September. This was very helpful for implementing connection managers - Salut, Idle and telepathy-sofiasip (our connection managers for link-local XMPP, IRC and SIP) all started off by forking/cargo-culting from Gabble, and achieved huge code reductions by porting to telepathy-glib. Also, Will Thompson used it in his Summer of Code project, telepathy-haze, to go from nothing to a working and useful Telepathy implementation based on libpurple in a matter of a few months. The second major round of work, which I'll discuss in a later article, added client code to telepathy-glib, obsoleting the older/worse libtelepathy and allowing client programs like Empathy and telepathy-stream-engine to be written using telepathy-glib. In the process, I accidentally reinvented DBusGProxy :-) More details to come later!

Some background: service-side code generation

The secret Collabora code-generation process

(credit: daf)

Supporting D-Bus in the GObject world has always involved quite a lot of code generation. The core API of dbus-glib is heavily reliant on varargs functions, which aren't type-safe and are easy to get wrong. dbus-glib contains a program called dbus-binding-tool, which is meant to generate reasonably sane GObject APIs for D-Bus objects. Unfortunately, it seems to be intended to generate a whole GObject at a time. Early versions of telepathy-gabble used dbus-binding-tool plus a script called gengobject.py to generate API stubs for the exported objects; developers then filled in the blanks, and hey presto, we had a GObject on D-Bus. This was fine up to a point, but had a couple of major drawbacks. Whenever we changed the D-Bus API (quite common during the early development of Telepathy), there was a very laborious and error-prone merging process. We ended up with the following process:

copy the D-Bus introspection XML from the telepathy-spec repository into a directory xml/pristine (actually, the canonical form for the spec was Python for a while, and we had a script that exported a contrived object onto D-Bus and introspected itself to get the XML - but that's another story!)
preprocess the introspection XML to add the dbus-glib CFunctionName annotation, resulting in a directory xml/modified
run dbus-binding-tool and a Python script gengobject.py to generate C source in xml/src
three-way-diff the old version of xml/src, the new version of xml/src, and the real C code in src to create a new version of src
pray that the diff process hadn't randomly exchanged functions' implementations
update the resulting code in src so it worked
check the whole mess back into darcs
hope that nobody else had made changes that would result in darcs conflicts

If you haven't yet gathered, this was a bit of a nightmare. Also, it was very easy to return the wrong thing from a method without noticing - because all the APIs were varargs, the compiler didn't notice, and the first we'd know about it was a mysterious segfault under certain circumstances.

How telepathy-glib fixes code generation telepathy-glib improves on this by taking advantage of this innocuous-looking feature in dbus-glib:

commit 355ef78d98d6fc65715845d56232199162cab12a
Author: Steve Fr cinaux <steve istique net>
Date:   Thu Aug 17 11:48:20 2006 +0200
    Interface support for bindings.

Instead of running dbus-binding-tool or creating GObjects with D-Bus interfaces, we put the entire Telepathy spec through glib-ginterface-gen.py, a distant descendant of Gabble's gengobject.py. For each interface in the spec, we generate a GInterface that mirrors the D-Bus interface. Any GObject that implements the GInterface automatically gets the D-Bus interface if it's exported onto D-Bus. There are a few non-obvious refinements, though:

The signature of the method implementation is always in the mode that dbus-glib calls "asynchronous", where the method implementation can either send a reply message before or after it returns. For "slow" methods, this is the only thing you can do. For instance, many Telepathy D-Bus methods can't return anything until some TCP round-trips to the server have happened. This is also the only thing you can do if you want to extract extra information, like the sender's unique name, from the method-call message (dbus-glib's "synchronous" API doesn't allow this to be done). For "fast" methods, sending a reply message before returning is just as easy as using the "synchronous" API (it's just a difference of syntax), and has an API consistent with that of the "slow" methods, making it easier for service authors.
The layout of the GInterface vtable is private, and (auto-generated) accessor functions are used to fill in implementations. This means our ABI doesn't change just because we re-ordered functions in the spec.
If no implementation is provided for a method, we just raise an appropriate error (org.freedesktop.Telepathy.Errors.NotImplemented), rather than suffering an assertion failure. This means we can safely add methods to an interface.
While we're generating code anyway, we generate some static inline wrapper functions which wrap dbus_g_method_return(), to have type-safe method returns. You can easily check that a method replies with correct types, by checking that the implementation of Foo() replies by calling tp_svc_some_interface_return_from_foo().
Similarly, we generate wrapper functions to emit signals, to get type-safe signal emission.

Not all interfaces are stable enough to be included in telepathy-glib's stable API and ABI, so some of our other projects include a copy of the telepathy-glib code-generation tools, and generate their own "mini-telepathy-glib" (traditionally in a /extensions/ directory) for their implementation-specific (drafts of) interfaces. This isn't yet as polished as it ought to be (mainly because we don't want to freeze the augmented-introspection-XML format that we write the Telepathy spec in, because it's rather ad-hoc and hackish in places) but it works quite well in practice. To see what this looks like in practice, have a look at the examples in telepathy-glib, or at a Telepathy connection manager like telepathy-gabble.

Mixins and base classes Another feature of telepathy-glib which makes life easier for connection manager authors is that it provides "mixins" and base classes which implement some of the GInterfaces. There's absolutely nothing magic about the base classes - they don't have any access to private implementation details of the GInterfaces, they just implement them like everyone else does. The "mixins" are only slightly more magical - they store some extra state inside the structures representing GObjects and their classes, and use it to provide default implementations of some or all of the methods on a particular interface (or two interfaces, in the case of the new TpMessageMixin). I'll leave it up to Rob to explain those in greater detail, since I seem to remember that he invented them :-P The examples and regression tests in telepathy-glib are quite good examples of how these work in practice.

Simon McVittie: A magical xrandr incantation

Here's a handy incantation I use with the Intel X.org graphics driver on my laptop.

Put this script somewhere convenient:

#!/bin/sh
if xrandr   grep '^VGA connected' >/dev/null
then
    #echo "VGA connected"
    xrandr --output VGA --above LVDS --auto
    xrandr --output LVDS --auto
else
    #echo "VGA not connected"
    xrandr --auto
fi 2>&1   logger -t xrandr-hotkey

and bind it to your "switch video mode" key, e.g. Fn+F7 on my Thinkpad X60s. For instance, I run xbindkeys, so I saved it as autoxrandr and put this in ~/xbindkeysrc:

"exec /home/smcv/bin/autoxrandr"
    XF86Display

Now if you press the "switch video mode" key with a monitor plugged in, you'll get the external monitor appearing above the laptop panel (my preferred configuration - I've balanced my monitor on an N800 box to get the bottom of the screen level with the top of my laptop's screen). If you press that key again after unplugging the VGA cable, the laptop will turn off the VGA output and pull all the windows onto the built-in screen.

Next.

Previous.