Search Results: "ana"

15 November 2008

Stefano Zacchiroli: from Vim to Emacs - part 2

One month with Emacs and counting - Part 2
Other posts in the from Vim to Emacs series: part 1
A while ago I've blogged about my switch from Vim to Emacs, promising a blog post series, quite a mouthful :-) nevertheless, it's time to continue the series. The first part was about why I think that nowadays Emacs is ready to be switching to. This second part is about flawed Vim design choices which substantially contributed to my choice. (Of course all this post is under a IMO-/rant-disclaimer, it can't be otherwise, so take it cum grano salis) Flawed Vim design choices Vim started as a wonderful editor, following the path of Vi, which in turn has got right a good deal of design choices. One is the often debated modality. In spite of being a HCI PITA in general, modality is not a problem at all for software you are using every single day (and the editor is the 2nd program I use the most, with the terminal being the 1st), because you learn it as a habit. In the case of Vim, modality is good because it let you use powerful yet simple motions in command mode, which can be easily combined with operator-pending commands. This can be considered yet another incarnation of the UNIX philosophy, casted inside the editor itself. ... which brings us to the good old joke which describes Emacs as a great operating system, lacking only a decent editor. In fact, Vim is no longer the nice Unix philosophy player which it used to be, and Emacs plays much more nicely with external tools than Vim. As arguments, let's consider a few (not so) recent Vim evolutions: Well, I acknowledge that Emacs got this choice right, and that it's an important one: Emacs supports interaction with external processes, and plugin authors have been happy to exploit it to create really cool stuff (sticking to the UNIX philosophy). Please welcome: Flyspell, M-x grep (shameless smartass plug: same number of keystroke of :grep), Grand Unified Debugger, a sampling of language-specific major modes with top-level support. (But I recommend having a look at least once at the coolness of other stuff built-on top of external process support, like Flymake, mentioned by dancer not a long ago.) Even in this respect, I acknowledge that Emacs got this right. It has chosen a single, now full-fledged, programming language (Elisp), which is offered both to the final user (fact: the average Emacs user knows much better how to program her own editor than the average Vim user) and to extensions developer. The standard library of the language is organized with a consistent naming (even though not always intuitive) and documented in an organized manner. You might not like the language, but at least is has been taken seriously and managed as such. Finally, being a Lisp dialect, you might need what you learn with Elisp elsewhere. Vim script is too committing: it is useful only inside Vim. I believe that contributed significantly in its scarce diffusion among Vim users. The fact that Elisp was born together with Emacs and that it is also used internally for the editor implementation is not relevant here: I do care about what is offered to me as an user and as an extension developer. What I see on the Emacs side in this respect is much better that what I see on the Vim side. Moreover, from the point of view of the users, is the difference between what is inside the editor and what is outside (i.e., the extensions) that relevant? I think it is not, and that brings me to the final Vim choice which I consider flawed and which has the potential of seriously limit its future evolutions: Where to from here in this blog post series? For sure some tips on how to migrate for hard-core Vim users, then we'll see ... If you did enjoy the read please let me know commenting in the discussion page (as you did in the last post, thanks!).

Yves-Alexis Perez: Xfce 4.6 Beta 2 Hopper

Hey, Xfce 4.6 Beta 2 Hopper has just been released and, guess what, packages are already available. Still not in unstable (or even experimental), that will wait post-lenny. You can install it from my repository, and, as usual, there's no Debian support on them. The known_issue page is currently quite empty, and this beta looks really quite polished. Packages currently available are i386 and amd64, powerpc will be available when I have time to put the mac at build. This beta features mostly bug fixing, and some features too: The official announcement can be found there.
Have fnu!

14 November 2008

Eddy Petrișor: If you own/have access to a Nokia E71 made for Romania, please contact me

I am trying to contact an owner of a Nokia E71 who is able to input Romanian diacritics on this type of phone. This person shouldn't worry, there is nothing destructive/dangerous involving his/her phone, should he/she contact me.

(Skip up to the "second approach" paragraph, if you don't care for the technical details or my story.)


Some time ago, during my latest blog silence period, I bought a white Nokia E71 phone from a Cosmote dealer.

I am really satisfied with it, except for two or three small issues[0], but the most annoying is that my phone was not made for the Romanian market, so I can't type Romanian diacritics[1].


Why does this matter? Because since I bought this phone I managed to do some translation work on the E71, on the daily commute to work. Still, the lack of diacritics forces me to go through another hop to add diacritics to my .po files when I get home.


So I was thinking on ways to fix this issue. After some google-ing I came up with the following possible fixes:
Since my phone was made for Singapore and I want to change to a Romanian product code, I have to wait for an update to be visible for Romania. Although the firmware update arrived for Singapore products, for phones made for Romania the firmware update is unavailable, so I am stuck on the first front.



Enter the second approach and my request for help.

I have looked into the files of my phone and I think I have found the files which define the keyboard layouts available on my phone. I haven't made out the details of the files, but, as a first step, I thought that I should try to replace one of those layout files I don't need with a Romanian layout file.

The problem is that I don't have a copy of such a file, but on a phone that works this file should be present and should be easily accessible. For short, I would need a copy of the firmware that is running on the phone, so I can try to replace on my phone the relevant files.



If you own a Nokia E71 and you're able to input Romanian diacritics on it, please contact me via a comment on my blog, or an email at eddy.petrisor+e71@gmail.com. Thanks in advance!


[0] the phone is really close to perfection on my scoreboard
[1] the correct "s comma" and "t comma" are not visible in any font present in the phone, but as long as they are correctly encoded I am fine with that. Surely with a proper font and an external editor I can fix that problem.

Christoph Haas: screenshots.debian.net goes live

After just three days of beta operation I could already make screenshots.debian.net go live. Within the first day I fixed a few typos, optimized database queries and added proper HTTP headers for caching. We got a huge load spike after my announcement on debian-devel-announce. I’m very happy and proud to have this service running and appreciate the help from the SQLAlchemy, Pylons and Debian community. Michael Vogt has already added screenshots support to the Synaptic package manager. Martin Stigge has joined the admin team to speed up approving screenshots in the moderation queue. (It would nice if someone else volunteered who is from .us or .au to deal with uploads while we sleep.) And above all I’d like to thank ‘zepolen’ from car.gr for allowing me to host the web application on his server. I think neither of us estimated so much interest in the service. May his server not melt under the load. :) Honestly the first feedback wasn’t very motivating (developers tend to point out errors first) but I enjoy seeing how quickly the news is spreading. And thanks to everybody for uploading further screenshots. I’ll keep on watching the moderation queue and enjoy how others blog about it. :)

MJ Ray: Spammers Silenced by Service Suppliers

Maybe, like me, you’ve noticed that you’ve had less junk email this week and you’ve been wondering why. News sites are reporting that a large spammer-friendly hosting service in California has been disconnected by its service providers after they were sent evidence about its activities. (Check out the “Next” links on the report to see how the story develops.) For the technically-minded, Changes in Spam Levels this week Posted by simonw illustrates the level of disruption and may grow an interesting discussion from server managers - it seems the reduction is less than the 75% reported in some news services, but still significant. This is great news for all good internet users. It’s disappointing if the spam hosting service won’t have to pay any of the costs they’ve inflicted on other computer users in some way. The only practical negative that I’ve noticed so far is that much of the stopped spam was pretty easy to identify and filter out, so the reduction in spam reaching my “unsure” mailbox hasn’t been anything like 50%. Still, less spam hitting the filters means less computer power used, which means less electricity and network data transfer used, which means lower costs for us. Yippee! And finally, I smiled at this comment over on the WebmasterWorld discussion:-
“Our spam email has dropped so much in the past 2 days that I was beginning to wonder if there was something wrong with our email accounts.”

13 November 2008

David Welton: Ubuntu Intrepid Regression: Beware of Wireless and WPA

A lot of people, myself included as of the upgrade I did last night, seem to be having trouble using certain wireless chipsets with WPA: https://bugs.edge.launchpad.net/ubuntu/intrepid/+source/wpasupplicant/+bug/272185 There are a lot of people following this one, and you have to think that for every person who looked it up, googled it, and dug around to find that page, registered with the site, and so on, there are several more who simply gave up and went back to Windows or something. I thought a couple of weeks was probably a good amount of time for the bugs to be shaken out of Intrepid, but I thought wrong. Here is a set of search terms that can be used to look through "regression-potential" bugs, which if I'm not mistaken, ought to mean those things that worked in previous versions of Ubuntu but now no longer work. In my case, there's an extra little bit of annoyance because I am running on a Dell laptop that I bought with Ubuntu preinstalled, and you'd think the Ubuntu guys would have a few sitting around to do some testing with prior to release. I spent some time digging around in the code for the network manager, wpasupplicant, and a brief look at the kernel code, but there was nothing that made much sense, so I guess I'll just have to wait for the fix like everyone else... wish I could help, somehow, though.

Decklin Foster: Film at 11

Managers are dicks. (ObBookMeme: "What's hard is sorting the result list so that the good results show up near the top.") (Something I am working on now that Njiiri has search, but who gives a fuck, right?)

Tore S. Bekkedal: Following memes for fun and prophets.

Cute meme du jour: Well, OK. Page 56, fifth sentence:
“The functions of other software interrupt service routines are quite variable; The I/O postprocessing interrupt service routine has a specific function to perform but is data-driven by the I/O request packets (IRPs) in its work queue.”
I know at least one of the slightly more than one people (I count too, right?) who read my blog (Hi, Ian!) might well realize what book this came from: VAX/VMS Internals and Data Structures, by Ruth Goldenberg and Lawrence Kenah. Yes, the book really was the closest to me - I have an overdesk shelf, and it was the furthest out. The book, by the way, is a fascinating read; I don’t know of any other book that lays out the design of an entire OS kernel in the really quite elegant way that this book does. I got mine signed by Ruth, too, which is pretty damn cool. :) One highlight of this book are the quotes at the beginning of each chapter - sometimes funny, sometimes profound, frequently both. I decided to list those from the first part here, for the enjoyment of both my readers.
Part I
Chapter 1, System Overview:
For the fashion of Minas Tirith was such that it was built on seven levels, each delved into a hill, and about each was set a wall, and in each wall was a gate.
- J. R. R. Tolkien, The Return of the King
Chapter 2, VAX Interrupts and Exceptions
“By indirections find directions out.”
- Shakespeare, Hamlet, 2, i
3, Hardware Interrupts
While I nodded, nearly napping, suddenly there came a tapping,
As of someone gently rapping, rapping at my chamber door.
Edgar Allan Poe, The Raven
4. Software Interrupts
And now I see with eye serene
The very pulse of the machine.
William Wordsworth, She Was A Phantom Of Delight
5. Condition Handling
“Would you tell me, please, which way I ought to go from here?”
“That depends a good deal on where you want to get to,” said the Cat.
Lewis Carroll, Alice’s Adventures in Wonderland
6. System Service Dispatching
Between the idea
And the reality
Between the motion
And the act
Falls the Shadow
T. S. Eliot, The Hollow Men
7. ASTs (Asynchronous Software Traps, ed.)
What you want, what you’re hanging around in the world
waiting for, is for something to occur to you.
Robert Frost
8. Synchronization Techniques
“Time,” said George, “why I can give you a definition of time. It’s what keeps everything from happening at once.”
Ray Cummings, The Man Who Mastered Time
9. Event Flags
I claim not to have controlled events, but confess plainly that events have controlled me.
Abraham Lincoln, Letter to A. G. Hodges, April 4, 1864
10. Lock Management
‘Tis in my memory lock’d
And you yourself shall keep the key of it.
Shakespeare, Hamlet, 1, iii
11. Time Support
Love, all alike, no season knows, nor clime,
Nor hours, days, months, which are the rags of time.
John Donne, The Sun Rising
12. Scheduling
It is equally bad when one speeds on the guest unwilling to go, and when he holds back one who is hastening. Rather one should befriend the guest who is there, but speed him when he wishes.
Homer, The Oddysey
13. Process Control and Communication
I was alone and unable to comunicate with anyone. I did not know the names of anything. I did not even know things had names. Then one day, after she had tried a number of approaches, my teacher held my hand under the water pump on our farm. As the cool water ran over my hand and arm, she spelled the word water in my other hand. She spelled it over and over, and suddenly I knew there was a name for things and that I would never be completely alone again.
Helen Keller

Sean Finney: (experimental) ps3 packages for debian

when not being distracted by other uses of the system, i've managed to spend some more time on packaging some more stuff that's useful on a debian ps3. namely: the last of which i think will be of the most interest to some folks, as it provides an spu-based xvideo extension, which brings back the possibility of playing HD videos etc on the ps3. now if you're interested in checking out these packages, read on, starting with this: !=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!= ZOMG WARNING!!!1!!ONE!! THESE ARE EXPERIMENTAL BLEEDING EDGE PACKAGES. I MAKE NO GUARANTEE WHETHER THESE PACKAGES ARE PROPERLY BUILT, WILL INSTALL, WILL WORK, WILL NOT DESTROY YOUR SYSTEM, WILL NOT CRAWL OUT OF YOUR SYSTEM AND MAKE LONG DISTANCE PHONE CALLS WHILE EATING THAT SNACK YOU WERE SAVING IN THE FRIDGE, ETC. !=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!=!= okay, so now that we've disposed with that formality, i'm glad if you're interested and still here :) so... where were we? oh yes. put this in your sources.list: 
deb http://people.debian.org/~seanius/ps3-experimental-packages sid main
source packages are also available if you want to poke around or backport the packages to etch or ubuntu or something else wierd like that: 
deb-src http://people.debian.org/~seanius/ps3-experimental-packages sid main
now, if you have a properly configured system, you should get a warning: 
Reading package lists... Done
W: GPG error: http://people.debian.org sid Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY CA78CB3E6E76D81D
W: You may want to run apt-get update to correct these problems
but i promise this key is valid, and mine. but hey, you don't have to take my word for it :) 
sudo apt-get install debian-keyring
gpg --no-default-keyring --keyring /usr/share/keyrings/debian-keyring.gpg --export --armor CA78CB3E6E76D81D   sudo apt-key add -
then you can install the packages... assuming you want the xorg video driver: 
sudo apt-get install xserver-xorg-video-spu
and update the appropriate Device entry in your xorg.conf, so that the normal fbdev is commented out and instead you use the spufbdev driver. example: 
Section "Device"
        Identifier      "Generic Video Card"
        Driver          "spufbdev"
        Option "ShadowFB" "false"
EndSection
and then 
sudo /etc/init.d/gdm restart
(or the equivalent if you're using something else). that's it! more packages may trickle in here as well, at least until and they find their way into debian proper. i'm also in contact with the upstream authors of this stuff, and they're generally very responsive, so let me know if you have any problems.

12 November 2008

MJ Ray: Webmaster Jargon for Website Owners

I wrote to a site owner last week and I thought I was writing to a webmaster. The site owner complained about some of the jargon and, while explaining who I thought I was writing for, I explained some of it because I think more website owners might benefit from these three explanations:- “Expat-like terms” - made available in a way that is freely sharable, modifiable and redistributable, similar to the Expat software package, whose terms are published at http://www.jclark.com/xml/copying.txt - this is often used as a clear, simple example for encouraging wide distribution of electronic resources (software). “clandestine Google Analytics” - Google Analytics is a service from Google, Inc for tracking users through a website in various ways. I believe the Data Protection Act means that English websites should obtain informed consent from users by publishing a Privacy Policy on their site which discloses what the GA service will be used for and linking through to GA’s own Privacy Policy. Some websites attempt to run Google Analytics on users’ computers without explaining why and without any Privacy Policy. That is what I mean by “clandestine”. “valid xhtml” - validating against the eXtensible HyperText Markup Language standards published by the World Wide Web Consortium (W3C) - the underlying language of the web. There is a test service provided at http://validator.w3.org/ and passing it is a key stepping stone towards making an accessible website. There’s not really such a thing as “invalid xhtml” - if it doesn’t pass validation, it’s not xhtml. So I guess I’m guilty of using a tautology sometimes - sorry about that. Is it worthwhile knowing those three phrases? Are there other key technical phrases which you think site owners should know?

11 November 2008

Daniel Burrows: aptitude 0.5.0 released

I've released version 0.5.0 of the aptitude package manager (release notes). Once the mirrors have synced, you should be able to download it from Debian's experimental distribution, or manually from the aptitude package page.
[Edit] This release was uploaded to experimental because it is not a final release. aptitude follows the Linux version numbering scheme in which an odd second digit indicates an unstable development version. DO NOT INSTALL THIS VERSION IF YOU WANT STABLE SOFTWARE; IT IS INCOMPLETE, INADEQUATE, AND PROBABLY INCORRECT. To reply to some questions that several people have asked me:
  1. The final release of aptitude 0.6.0 will be split into two packages, one containing only the traditional text interface and named aptitude-curses, and one containing the GUI binary named aptitude-gtk, with both of them providing the aptitude binary through Debian's alternatives mechanism. This release wasn't split because the code to drop GTK+ support was buggy and I discovered this fact halfway through assembling the release. It's fixed now.
  2. To run the traditional curses interface, pass the --no-gui command-line parameter or set the option Aptitude::Start-Gui to false in /etc/apt/apt.conf. Unless you want to test that the curses frontend isn't broken, though, there won't be much for you in this release: the only change affecting it is Xapian support, and that currently breaks incremental search. (see the note above about this being an unstable release)
Also, another feature that I suspect might be bothersome is the behavior of updating the package lists and downloading changelogs on startup. There's no switch for changelogs yet, but you can disable the package list update by setting Aptitude::Update-On-Startup to false.
The main change in this release is the new GTK+ frontend, designed and implemented by Obey Arthur Liu with funding from the 2008 edition of Google's Summer of Code. aptitude 0.5.0 screenshot The new frontend is is an effort to bring some of the design principles of the curses frontend to a GUI environment, while also exploiting the unique features a GUI gives us and exploring ways to deal with changes in the environment in the nine years since aptitude was first designed. For instance, it is no longer reasonable for the user to actually read the entire package list (when I first installed Debian, you could read through the whole package list in a single sitting). So instead of basing the interface around a list of all the packages, we based it around the ability to search for the packages you're interested in. This version of aptitude also introduces Xapian searching: 
daniel@emurlahn:~$ aptitude search "apt package manager"
i   apt                                  - Advanced front-end for dpkg
p   apt-dater                            - terminal-based remote package update manager
p   apt-dater-dbg                        - terminal-based remote package update manager (d
i   aptitude                             - terminal-based package manager
i   aptitude-dbg                         - Debug symbols for the aptitude package manager
p   createrepo                           - generates the metadata necessary for a RPM pack
p   gnome-apt                            - graphical package manager
p   smartpm                              - An alternative package manager that works with
p   smartpm-core                         - An alternative package manager that works with
i   synaptic                             - Graphical package manager
i A update-manager-core                  - APT update manager core functionality
You might ask why I didn't search for just package manager. The reason is simple: aptitude doesn't yet sort by relevance, and that second search gave me screenfuls of packages whose description contains both package and manager, including addressmanager.app (a PIM for GNUstep), compiz-gtk (a piece of eye candy), and wterm (an X terminal emulator). However, it is worth noting that Xapian searches are fully integrated into the aptitude search language, so they can be combined with non-Xapian search terms in just the way you would expect: 
daniel@emurlahn:~$ aptitude search "?installed apt package manager"
i   apt                                  - Advanced front-end for dpkg
i   aptitude                             - terminal-based package manager
i   aptitude-dbg                         - Debug symbols for the aptitude package manager
i   synaptic                             - Graphical package manager
i A update-manager-core                  - APT update manager core functionality
So, all the basic functionality that the GUI version needs is in place; the next step is to start polishing it and filling in the gaps that are left. And, of course, suggestions and bug reports are welcome, so we know where the rough spots and the gaps are. Please send your comments to aptitude-devel@lists.alioth.debian.org so that they are part of the public record and so that people besides me can read them. :-) Happy package managing!

John Goerzen: Review: Silicon Mechanics

After some hilariously frightening reactions from Dell support to simple problems, and HP becoming aggressively competitive on price, we've been using HP servers for a few years now. The hardware is good, and the support, while reasonable, always... pauses... when I mention that we're running Debian. I try not to let it slip if I don't have to.

We put in some HP blades a couple of years ago, and I was annoyed to discover that they have discontinued that enclosure and all the blades in it. I decided this was a good time to look at their newer options, as well as at other companies.

Back in July, I had noticed a Silicon Mechanics booth at OSCon. I noticed their slogan "experts included." That sounds great; we've got software experts here, but not hardware experts, and I'd enjoy dealing with a company that knows more about their hardware than I do. I went up to their booth and asked what they'd say about us running Debian on their hardware. "That would be just fine." "So you'd fully support it when I'm running Debian?" "Sure." "What about management software - do you have any of that which I'd find annoying to port to Debian?" "Our servers don't need any management software other than what comes with your kernel." Good answers.

So, when it came time for us to decide what to do about getting a new server in here, I figured I'd call up Silicon Mechanics and see what they'd recommend. They put me on a conference call with a sales rep and an IT engineer, and wound up recommending a 1U server for us to start with, and an iSCSI storage device to address some of the storage needs we have (both for that server and others). I had heard of iSCSI only vaguely, and asked how it worked, and what the performance would be like compared to our 2Gb FC SAN. I got back intelligent (and correct) answers.

They probably spent 2 hours with me on the phone before we placed an order. I was incredibly happy with their service, level of expertise, and helpfulness. They even did a webinar to demo the management interface on the storage unit for me.

Today, the 1U server arrived. I unboxed it and set it on my desk to configure. First item: set an IP address for the IPMI card. That's the device that lets me connect to it over a web browser and interact with the console, power cycle it, etc. as if I was there. I set an IP, but somehow couldn't seem to figure out the username and password for the web interface.

So I called Silicon Mechanics support at the number that was included on the fridge magnet (!) that came with the shipment. Phone rang once. Then a live, capable American answered. No menus, no fuss. I asked my question. He apologized, saying, "I should know that, but I'll have to look it up... hold on just a bit." I had my answer about 90 seconds later. He offered to send me the full docs for the IPMI card if I wanted as well.

So I've been very impressed with them so far. From what I've heard, their iSCSI enclosure ought to be quite something as well. They even helped us spec out a switch that supports trunking for use with it.

I'll give them a "highly recommended".

Clint Adams: Bowling for madducks

A long time ago, I had the idea of improving the interface between shell completion and the programs being completed. The result of this was the bzr shell-complete command (or bzr s-c for short), which was never fully fleshed out, and has since fallen into disrepair. The principles behind this are + the program is the best place to store up-to-date and accurate information + the program already knows all these things (albeit usually in unparseable forms) + duplicating information and effort is annoying Here is an excerpt from bzr s-c, which was intended to give a comprehensive list of subcommands, paired with short descriptions: 
diff:show differences in the working tree, between revisions or branches
export:export current or past revision to a destination directory or archive
get:create a new copy of a branch
help:show help on a command or other topic
ignore:ignore specified files or patterns
ignored:list ignored files and the patterns that matched them
info:show information about a working tree, branch or repository
init:make a directory into a versioned branch
There is one subcommand per line, separated from its description by a colon. Next you can invoke something like bzr s-c diff to get the possible options and arguments for the diff subcommand, although the output you would see today is broken and nearly useless. Since I've lost faith in bzr, I'll illustrate what the output might be corresponding to topgit's tg remote if topgit supported this kind of thing: 
--populate
REMOTE
This would mean that tg remote can understand the option --populate, which takes no argument, and that the first non-option argument should be a REMOTE. REMOTE would then be defined, for example, in zsh's _topgit function as some kind of git remote which is completed in the same way you might complete a git remote for git. The exciting part then, is that if tg remote starts taking a --decimate option, the topgit completion helper subsystem will start outputting it and _topgit will do the right thing without having to be altered. For tg export, things are a bit more complicated, so let's have it be described in the style of the zsh completion system: 
'(--collapse)--quilt:directory:_directories'
'(--collapse -b --branch)' -b,--branch= ':branches:BRANCHES'
'(--quilt)--collapse:branch:BRANCH'
This means that --collapse and --quilt are exclusive, that -b and --branch cannot be used with --collapse, that -b and --branch are equivalent, that -b and --branch take an argument in the form of BRANCHES, that --collapse takes an argument in the form of BRANCH, and that --quilt takes an argument that's a real directory in the filesystem. Then _topgit would have logic to interpret BRANCH as a branch, and BRANCHES as a comma-separated list of branches. A similar idea is the one used by axp. If you invoke axp self completion zsh, it will output zsh completion functions for you. To me this seems more onerous on both the developers and the end users, but I suppose it gives you immediate flexibility that a more generic interface would lack.

10 November 2008

Riku Voipio: When Linux doesn't mean freedom

I actually think it's a bit of an insult if people think of Motorola's EZX or MAGX (and now Android) phones as "Linux phones". Because all the freedoms of Linux (writing native applications against native Linux APIs that Linux developers know and love, being able to do Linux [kernel] development) are stripped.
-
http://laforge.gnumonks.org/weblog/2008/10/23/#20081023-android">
http://laforge.gnumonks.org/weblog/2008/10/23/#20081023-android">
http://laforge.gnumonks.org/weblog/2008/10/23/#20081023-android">Harald Welte

This is something that has worried me too for quite a while. For years we have now had Linux phones available in form or another (well, mostly in far away countries you happen not to be at). Yet with the exception of openmoko none of them allow native application development. Yet all the same time you can go to the nearest phone shop and grab HTC windows CE phone or a Nokia Symbian phone, which actually give you the freedom to writing and running your own software on them.

Now we have the situation that HTC's most locked device* is their only Linux device - The Android G1.

Incidentally, you can install Debian/armel on T-mobile G1, but only until Google engineers manage to fix the hole that gives you r00t.

*The only one that doesn't allow native app development.

Gunnar Wolf: 4 Rodada Nacional Tri-Estatal M xico-Pachuca

Yesterday I took part of the fourth inter-state bike ride, requesting the construction of proper and safe cycling ways. Tens of Mexican bike-rider organizations were part of the organization, althought the effort is basically grouped under the Ciclopistas y Ciclov as Interestatales project - For further details on the route we took, you can look at the M xico-Pachuca road map, although it was not followed literally.
I am very happy I decided to take part of this - I went with Adri n, but after the first rest/grouping we lost track of each other, and decided the best would be for each to go at his own pace. He managed to do the 95Km ride, I decided to stop at about 85 (I set my target to the first houses of Pachuca, so I could say I arrived ;-) ) and took a ride from the barredoras that were trailing us. Still, 85Km in about six hours (including the two rests) are well over what I expected to endure. And although the conditions were not ideal (i.e. there was some re-pavement jobs in a large portion between Tizayuca and Pachuca, which led to poor road conditions in some areas, and the dry, horrid smell of tar in others; we cycled along the vehicles in the very busy highway, so we didn't exactly get a dose of clean air), it was great. And I thought each of my muscles would hurt like hell today, but no, they are just feeling lazy ;-)
The GPS tracking on Nokia SportsTracker, The GPS tracking on OpenStreetMap

David Welton: Generating Rails Fixtures From Large Datasets

I've been working with Tomaso Minelli doing some Rails work, and one thing we needed to do was set up some tests. The situation we are in is that we already have a large set of data to work with in our development environment, but getting that data into .yml files is no easy task. Since we have a fairly large number of interdependent tables, dumping one of them alone is pretty useless, unless you manage to get the other tables it links to, and so on and so forth. Dumping all the data would make testing quite slow, and really is too much to be useful. Here's the rough solution we arrived at: 

@fixtures =  
@seen =  
# Do something with a fixture.  In this case, we store it for later.
def handle_fixture(record)
  @fixtures[record.class.table_name]  = []
  @fixtures[record.class.table_name] << record
end
# Get some sort of unique way of identifiying a record - class name and id.
def unique_record(record)
  return record.class.to_s + " " + record.id.to_s
end
# Grab all the relations of a class and add them to the queue.
def add_fixture(record)
  ur = unique_record(record)
  if @seen[ur]
    puts "LOOP!"
    return
  end
  @seen[ur] = true
  puts "Fixture for # record.class.name  - ID # record.id "
  handle_fixture(record)
  record.class.reflections.each do  k, r 
    begin
      if r.macro == :has_many  
          r.macro == :has_and_belongs_to_many
        related = record.send(r.name).send('find', :all, :limit => STARTING_LIMIT)
      else
        related = record.send(r.name)
      end
      related = [related] unless related.is_a?(Array)
      related.each do  related_record 
        if related_record
          puts "    New Record # related_record.class.name  # related_record.id "
          @todo << related_record
        end
      end
    rescue => e
      puts "    ERROR: # e :\n# e.backtrace.inspect "
    end
  end
end
@todo = []
namespace :db do
  namespace :fixtures do
    desc 'Create YAML test fixtures starting from a single model using reflections'
    task :modelbase => :environment do
      STARTING_CLASS = Foobar
      STARTING_FIND  = :all
      STARTING_LIMIT = 1
      tmp = 0
      totale = STARTING_CLASS.find(STARTING_FIND, :limit => STARTING_LIMIT).size
      STARTING_CLASS.find(STARTING_FIND, :limit => STARTING_LIMIT).each do  m 
        tmp += 1
        puts "# tmp  su # totale : # m.id "
        @todo << m
      end
      while true
        if @todo.length == 0
          break
        end
        add_fixture(@todo.pop)
      end
      puts "\n\n# '='*80 \nScan Finished\n# '='*80 \n\n"
      @fixtures.each do  name, fixture 
        if fixture.size > 0
          i = "000"
          filename = "# RAILS_ROOT /test/fixtures/# name .yml"
          puts "Create # filename  for class # fixture.class.name "
          FileUtils.mkdir(File.dirname(filename)) unless Dir[File.dirname(filename)].size > 0
          File.open("# RAILS_ROOT /test/fixtures/# name .yml", 'w') do  file 
            file.write fixture.inject( )    hash, record 
              hash["# record.class.table_name _# i.succ! "] = record.attributes
              hash
             .to_yaml
          end
        else
          puts "Fixture for # name  is EMPTY"
        end
      end
    end
  end
end
It's written in a somewhat quick and dirty way, but it gets the job done. What it does: It takes a single class - hopefully a well connected one - fetches a few of them, and goes from there, exploring all the other objects that the initial ones are connected to, and then dumping them in .yml files in the fixtures/ directory. It could certainly use more work:
  • Additional starter classes, for ones that aren't connected to others.
  • Intelligent examination of data and data sets...
  • For example you could check to see what classes aren't connected to the others and use them as starter classes.
  • If you wanted to waste a lot of cycles, you could also try and aim for a representative set of examples. For an object that has_many something elses, you could try and find one with zero, one with one, and one with a lot, to cover various cases. That could quickly get out of hand for largish data sets though.
  • I'm sure there are various other tweaks, knobs, and levers that could be added to work more efficiently. I'd love to see them!

Matthew Garrett: Android

As a brief introduction to this - I first read through the Android code when interviewing with Google for an opening with the Android kernel team. I didn't get the job, and while I don't think anything that follows here is as a result of residual bitterness you might want to take some of it with a pinch of salt.

Anyway. Android is a Linux kernel with a heavily customised user layer that bears little resemblance to any traditional Linux system. I write in C because using pointer arithmetic lets people know that you're virile so this Java-like thing is of little interest to me and I'm going to ignore it and just look at the kernel, because after all that's what I'm paid to be good at.

The short summary of the rest of this post is that I'm not planning on giving up my iphone yet.

The Android kernel source lives in a git tree here. It can pretty much be logically split into two parts - the port to Qualcomm's MSM7xxx series chips and the special Android customisations. A bunch of the MSM7xxx port code has been merged through the ARM tree and is now upstream, and Brian Swetland seems to have been fairly active in looking after that. Full marks to Google there. This code is handy outside the Android world and benefits anyone wanting to run Linux on similar devices.

The Android-specific code is more... interesting.

As I mentioned, the Android application layer isn't Unix in any real sense. The kernel reflects this. It ranges from pragmatic (if hacky) approaches like encoding security policy and capabilities in group IDs that are hardcoded into the kernel through to implementing an in-kernel IPC mechanism (apparently related to the OpenBinder implementation from Palm, but judging by the copyrights a complete rewrite). To an extent, I'm fine with this. Something like Binder is pretty clearly not going upstream, so the fact that it engages in bizarre design decisions like sticking chunks of its interface in /proc is pretty irrelevant. What's more interesting is the code that should be generalisable.

I work on power management, so I'm always interested in what kind of power management functionality and interfaces people want. Plumbers included a nice discussion with someone from an embedded company I can't remember, culminating in us deciding that the existing cpufreq interface did what they wanted and so no new interfaces needed to be defined. Google was going to be an interesting case of a large company hiring people both from the embedded world and also the existing Linux development community and then producing an embedded device that was intended to compete with the very best existing platforms. I had high hopes that this combination of factors would result in the Linux community as a whole having a better idea what the constraints and requirements for high-quality power management in the embedded world were, rather than us ending up with another pile of vendor code sitting on an FTP site somewhere in Taiwan that implements its power management by passing tokenised dead mice through a wormhole.

To a certain extent, my hopes were fulfilled. We got a git server in California.

Android contains something called the android_power driver (ignore the references to CSMI, which is a piece of OMAP-specific hardware for communicating with the baseband - I'm not sure what they're doing in there). As far as I can tell, this is an interface that handles the device being locked and unlocked, and associated powering down of certain bits of hardware. Except it's shit. Devices register device handlers and a level where they wish to be suspended. There's no direct concept of intra-device dependencies so you end up with stuff like:
android_early_suspend_t early_suspend;
android_early_suspend_t slightly_earlier_suspend;
to deal with the fact that the MSM framebuffer driver depends on the MDDI driver having brought the link bank up and so needs its suspend method called before the MDDI driver but its resume method to be called after, and the only way to handle that is to register two methods - the "slightly earlier" one which has a suspend method but no resume, and the "early" one which has a resume method but no suspend one.

Of course, this also means that all of your device runtime power management policy ends up in the kernel. Userspace indicates what state it wants to go to and the kernel decides what's going to get powered down. This kind of coarse grained approach means that as your hardware setup becomes more complex you hit combinatorial explosion. Expressing all the useful combinations of hardware state simply becomes impractical if all you're exposing is a single variable. What would be more useful is the ability for userland to interact with individual pieces of hardware.

The amusing thing is that in many cases Linux already has this. Take a look at the backlight and LCD class drivers, for instance. They provide a trivial mechanism for userspace to indicate its desires and then modify the device power state. It's true that there are other pieces of hardware that don't currently have interfaces to provide this kind of information. And that's where cooperation with the existing community comes in. We've already successfully fleshed out interfaces for runtime power management for several hardware classes, with the main thing blocking us being a lack of awareness of what the use cases for the remaining classes are. But linux-pm has seen nobody from the Android team, and so we end up with a lump of code solving a problem that shouldn't exist.

When Robert Love gave his presentation on Android at Lugradio Live in SF back in April, he talked about how one of the reasons that Google weren't releasing the source for their userland stack until they shipped phones was to prevent it being seen as throwing unfinished code over the wall and then ignoring the community. It's unfortunate that this is almost exactly what's happened with their kernel. Right now the fact that Android is based on Linux is doing almost nothing to benefit the larger Linux community. What could have been a valuable opportunity for us to gain understanding of an interesting problem has instead ended up as yet another embedded vendor kernel, despite all the assurances I got from various people at Google.

Disappointing.

9 November 2008

Adam Rosi-Kessel: Roslindale Scoop: Himalayan Bistro Coming to the Neighborhood

I have it on very good authority* that our favorite Indian/Nepali restaurant, Himalayan Bistro, is coming to Roslindale (from West Roxbury), into the oft-turned-over location last assigned to NuVo, and Gusto before that. Perhaps the third “-o” restaurant will be a charm. Assuming they serve up the same superlative food and atmosphere as the West Roxbury location, I’m sure this one will be a keeper. (My only concern is that having the two restaurants so close to each other may result in self-competition.) *Authority = Himalayan Bistro delivery guy. You don’t get much more authoritative than that. In other Roslindale Restaurant news, we heard via RVMS recently that Robyn’s Bar and Grill has changed hands to the owners of the Halfway Cafe in Dedham, which could be a change for the better. Roslindale did not need another pizza/sports-bar, and still does not need one, although maybe that’s all the Robyn’s location will ever be. Even more recently, however, we heard an unconfirmed rumor from a neighbor that the sale fell through and Robyn’s is not changing owners. In other potentially disappointing news, the Roslindale Emac and Bolio’s has changed owners again, this time to a co-owner of the Blue Star Restaurant. I want to like Blue Star, but they can’t seem to get the most basic short-order cooking tasks right. A diner must, first and foremost, deliver greasy food fast, hot, and at the same time for the entire group. At worst, serve the kids first. It also helps for the food to be consistent. Blue Star in our experience (many visits) only occasionally satisfies the basic diner requirements, much less delivers any sort of creative urban-chic reinterpretation of the classic retro diner. (By contrast, Deluxe Town Diner in Watertown is our current Greater Boston favorite — it’s always a good sign when a greasy spoon also actively caters to vegans.) Which brings me to Emac and Bolio’s, which apparently will be renamed “Select Caf .” First, why give up the goodwill associated with the Emac’s trademark? Second, the initial signs of the change-over are not promising. A handwritten sign posted in the window announces that the cafe is under new management, and (1) no credit or debit cards will be accepted and (2) no Emac and Bolio’s gift certificates will be honored. I’d love to be proved wrong about this, but I’m pessimistic that this is going to be the Change We Need in Roslindale — namely, a top rate artisanale snobby espresso shop. We can only hope that Simon’s, Flatblack, Cafe Nation, Diesel, or another of their ilk will come occupy one of the vacant storefronts on the courtyard. That we have any vacancies at all in the square, much less several, is a crime against humanity. Technorati Tags: ,

Simon McVittie: Implementing D-Bus services with telepathy-glib

(Part 1 of an ongoing series: shiny things in telepathy-glib)
16:46 < epmf> tp-glib is made of magic

—#telepathy, 2008-04-15

Rob pointed out today that I'd been promising to blog about telepathy-glib for several months and still haven't done so. I think there's too much to cover in one blog post, so I'll start with the oldest part - implementing a service (typically a Telepathy connection manager).

While telepathy-glib is (obviously) intended for Telepathy implementors, I think the ideas we've been implementing are likely to be useful for any D-Bus API, particularly flexible/complex APIs where an object implements many interfaces.

Introduction to telepathy-glib telepathy-glib started out as a collection of code extracted from our XMPP connection manager, telepathy-gabble, but at some point it grew beyond that into a wrapper around/partial replacement for dbus-glib. We've been quite pleased with it, really :-) My first major round of work on telepathy-glib, during the first half of 2007, was to split it out from telepathy-gabble (versions up to 0.5.9 were released as part of Gabble 0.5.x tarballs, in fact), leading to the release of the 0.6.x stable branch in late September. This was very helpful for implementing connection managers - Salut, Idle and telepathy-sofiasip (our connection managers for link-local XMPP, IRC and SIP) all started off by forking/cargo-culting from Gabble, and achieved huge code reductions by porting to telepathy-glib. Also, Will Thompson used it in his Summer of Code project, telepathy-haze, to go from nothing to a working and useful Telepathy implementation based on libpurple in a matter of a few months. The second major round of work, which I'll discuss in a later article, added client code to telepathy-glib, obsoleting the older/worse libtelepathy and allowing client programs like Empathy and telepathy-stream-engine to be written using telepathy-glib. In the process, I accidentally reinvented DBusGProxy :-) More details to come later!
Some background: service-side code generation
The secret Collabora code-generation process

(credit: daf)

Supporting D-Bus in the GObject world has always involved quite a lot of code generation. The core API of dbus-glib is heavily reliant on varargs functions, which aren't type-safe and are easy to get wrong. dbus-glib contains a program called dbus-binding-tool, which is meant to generate reasonably sane GObject APIs for D-Bus objects. Unfortunately, it seems to be intended to generate a whole GObject at a time. Early versions of telepathy-gabble used dbus-binding-tool plus a script called gengobject.py to generate API stubs for the exported objects; developers then filled in the blanks, and hey presto, we had a GObject on D-Bus. This was fine up to a point, but had a couple of major drawbacks. Whenever we changed the D-Bus API (quite common during the early development of Telepathy), there was a very laborious and error-prone merging process. We ended up with the following process:
  • copy the D-Bus introspection XML from the telepathy-spec repository into a directory xml/pristine (actually, the canonical form for the spec was Python for a while, and we had a script that exported a contrived object onto D-Bus and introspected itself to get the XML - but that's another story!)
  • preprocess the introspection XML to add the dbus-glib CFunctionName annotation, resulting in a directory xml/modified
  • run dbus-binding-tool and a Python script gengobject.py to generate C source in xml/src
  • three-way-diff the old version of xml/src, the new version of xml/src, and the real C code in src to create a new version of src
  • pray that the diff process hadn't randomly exchanged functions' implementations
  • update the resulting code in src so it worked
  • check the whole mess back into darcs
  • hope that nobody else had made changes that would result in darcs conflicts
If you haven't yet gathered, this was a bit of a nightmare. Also, it was very easy to return the wrong thing from a method without noticing - because all the APIs were varargs, the compiler didn't notice, and the first we'd know about it was a mysterious segfault under certain circumstances.
How telepathy-glib fixes code generation telepathy-glib improves on this by taking advantage of this innocuous-looking feature in dbus-glib: 
commit 355ef78d98d6fc65715845d56232199162cab12a
Author: Steve Fr cinaux <steve istique net>
Date:   Thu Aug 17 11:48:20 2006 +0200
    Interface support for bindings.
Instead of running dbus-binding-tool or creating GObjects with D-Bus interfaces, we put the entire Telepathy spec through glib-ginterface-gen.py, a distant descendant of Gabble's gengobject.py. For each interface in the spec, we generate a GInterface that mirrors the D-Bus interface. Any GObject that implements the GInterface automatically gets the D-Bus interface if it's exported onto D-Bus. There are a few non-obvious refinements, though:

  • The signature of the method implementation is always in the mode that dbus-glib calls "asynchronous", where the method implementation can either send a reply message before or after it returns. For "slow" methods, this is the only thing you can do. For instance, many Telepathy D-Bus methods can't return anything until some TCP round-trips to the server have happened. This is also the only thing you can do if you want to extract extra information, like the sender's unique name, from the method-call message (dbus-glib's "synchronous" API doesn't allow this to be done). For "fast" methods, sending a reply message before returning is just as easy as using the "synchronous" API (it's just a difference of syntax), and has an API consistent with that of the "slow" methods, making it easier for service authors.

  • The layout of the GInterface vtable is private, and (auto-generated) accessor functions are used to fill in implementations. This means our ABI doesn't change just because we re-ordered functions in the spec.

  • If no implementation is provided for a method, we just raise an appropriate error (org.freedesktop.Telepathy.Errors.NotImplemented), rather than suffering an assertion failure. This means we can safely add methods to an interface.

  • While we're generating code anyway, we generate some static inline wrapper functions which wrap dbus_g_method_return(), to have type-safe method returns. You can easily check that a method replies with correct types, by checking that the implementation of Foo() replies by calling tp_svc_some_interface_return_from_foo().

  • Similarly, we generate wrapper functions to emit signals, to get type-safe signal emission.

Not all interfaces are stable enough to be included in telepathy-glib's stable API and ABI, so some of our other projects include a copy of the telepathy-glib code-generation tools, and generate their own "mini-telepathy-glib" (traditionally in a /extensions/ directory) for their implementation-specific (drafts of) interfaces. This isn't yet as polished as it ought to be (mainly because we don't want to freeze the augmented-introspection-XML format that we write the Telepathy spec in, because it's rather ad-hoc and hackish in places) but it works quite well in practice. To see what this looks like in practice, have a look at the examples in telepathy-glib, or at a Telepathy connection manager like telepathy-gabble.
Mixins and base classes Another feature of telepathy-glib which makes life easier for connection manager authors is that it provides "mixins" and base classes which implement some of the GInterfaces. There's absolutely nothing magic about the base classes - they don't have any access to private implementation details of the GInterfaces, they just implement them like everyone else does. The "mixins" are only slightly more magical - they store some extra state inside the structures representing GObjects and their classes, and use it to provide default implementations of some or all of the methods on a particular interface (or two interfaces, in the case of the new TpMessageMixin). I'll leave it up to Rob to explain those in greater detail, since I seem to remember that he invented them :-P The examples and regression tests in telepathy-glib are quite good examples of how these work in practice.

Simon McVittie: Encrypted root filesystem on a Debian laptop

I've been meaning to document this for ages... My laptop is set up with an encrypted root filesystem using LVM and dmcrypt. It also has a small Debian stable system for recovery purposes (I don't have a CD drive, so if everything goes horribly wrong, I can't just boot from a live CD). Here's how to set up something similar:

Step 1. Set up the recovery system Start the Debian lenny (beta) installer as usual. (I originally used etch, but these instructions are for lenny - either should work.) When you get to "Partition disks", choose "Manual". Here's the partitioning scheme to use:
  • main partition for LVM, taking up the whole disk minus 1GB or so. select "Do not use", for now
  • 1GB recovery partition at the end of the disk (this will also be the main system's /boot, since /boot needs to be unencrypted anyway). Use the defaults: ext3 mounted at /.
Finish partitioning and write changes to disk, and wait for the base system to install. Say yes to installing Grub to the MBR for now (it might be possible to do the install more cleanly by installing Grub to the recovery partition's boot sector instead - I haven't tested that). Reboot into the freshly installed recovery system and satisfy yourself that it works.

Step 2. Set up the main system Now boot the Debian installer again. This time it's for your installed system. At the "Partition disks" stage, choose "Manual" again. Select the main partition and choose "Use as: physical volume for encryption". Next select "Configure encrypted volumes". Your main partition will now be randomized - this is slow - and some time later you'll be asked for a passphrase. You'll have to type this in at each boot. Select the contents of the "disk" Encrypted volume (hda1_crypt) and choose Use as: physical volume for LVM. Now "Configure the Logical Volume Manager" and create a Volume Group. I always use the laptop's hostname as the VG name (this reduces confusion if you ever plug the disk into another machine for disaster recovery). Create a Logical Volume called swap, the size you want your swap space to be. If you plan to use suspend-to-disk, this needs to be at least as large as your RAM. Create a Logical Volume called root, for the root filesystem. If you want separate "partitions" for things like /home, now is a good time to create them too; if you want to use my schroot howto, leave some unallocated space in the VG for that. Set your swap LV to be used as a swap area, and your root LV to be used as ext3 mounted at /. If you wanted extra LVs, set them up too. Also set your recovery partition to be used as ext3, mounted on /boot, and not reformatted. It should now look something like this (smcvcrypt is the name of a KVM virtual machine in which I've been testing these instructions, normally you'd have the laptop's hostname there). Screenshot from d-i Proceed with the installation. Install Grub to the MBR - this will temporarily make the recovery system unbootable, but shrug never mind. Finish the installation and reboot into your new main system.

Step 3. Make them dual-boot Within the main system your recovery system is also visible, at /boot. Bind-mount /dev onto /boot/dev, chroot into /boot, and run "update-grub /dev/hda2", where hda2 is the partition where the recovery partition is. Leave the chroot. Edit /boot/grub/menu.lst and put this right at the end: 
title Go to recovery system
root (hd0,1)
chainloader +1
(Replace (hd0,1) with what Grub thinks the recovery partition is - the second number is the partition number starting from 0, so /dev/hda5 would be (hd0,4) and so on.) Also edit /boot/boot/grub/menu.lst and put this right at the end: 
title Back to main system
root (hd0)
chainloader +1
Also, still in /boot/boot/grub/menu.lst, go to the top of the file and change the colour scheme to something else (I used a red background) to indicate that this boot menu is for the recovery system. Reboot and try it out. You should now have an extra boot menu option, "Recovery system". Selecting it will switch to the recovery system's boot menu, which has an option to switch back, and so on. Each boot menu also has some entries for kernels, any of which will boot with the appropriate root filesystem (encrypted root for the main system, unencrypted for the recovery system). Success!

Next.

Previous.