Search Results: "ag"

31 May 2009

Dirk Eddelbuettel: Ubuntu Developer Summit in Barcelona

Due to some things falling into place, I had an opportunity to attend the first two days of last week's Ubuntu Developer Summit in beautiful Barcelona. Somehow, I had never managed to attend a Debian conference either, so it was good to meet a few of the old Debian hands now moving Ubuntu along, as well as a few of the Ubuntu folks. I also gave a short presentation on R in Debian / Ubuntu and the plans for the upcoming Ubuntu release. More on that another time. All told, a well-organised conference in a nice setting -- two stone throws from the legendary Camp Nou. Unfortunately, I had to leave by Wednesday so I missed what was undoubtedly quite a scene in Barcelona following Barca's dismantling of Man U in this year's Champions League final.

Russell Coker: Expectations and Fandom

Russ Allbery has written about the hostile reactions of sci-fi fans toward the delay of a book release [1]. Russ makes some good points regarding the issue of whether paying $100 for some books entitles a reader to have the rest of the series written (in summary - no, the author owes the fans nothing). Russ also notes that when someone makes a public promise to do something (such as writing a book) they will probably be feeling bad about being unable to live up to that, and harassing them about it is not going to make them feel any better - or make them better able to do the work. Charles Stross has written a blog post about this topic, he makes some interesting points and the comments illustrate most of them [2]. One comment claims that the author of the delayed book in question flaunts his NFL watching while the same person also criticises the author for writing a blog. This level of contempt for the author amazes me. In case anyone is interested I watch The Bill and Desperate Housewives every week. Anyone who wishes to criticise me for watching soapies instead of documentaries is welcome to do so. But I will not make any apologies for spending some of my time doing things other than coding. If I watch Desperate Housewives instead of fixing a bug that inconveniences you then you just have to deal with it. I believe that taking a regular break from work is a basic human right, apart from which there is evidence to show that working excessive hours decreases the quality and quantity of work that is completed [3]. It seems to me that members of the free software community are in a better position regarding fans than most creative people, and that this is largely due to how transparent the development process is. The most influential members of the user (fan) base are software developers. While people will be disappointed when software is not released on time or has some bugs many users know what is involved and understand that no testing will find all bugs. Even some basic shell scripting can teach people the difficulties of programming and convince them to have a little more patience than they might otherwise have. So the typical user of free software will have some idea of what is required to write software while the typical reader of a novel has no idea of what is involved in writing a book. Another benefit for the free software community in this regard is the open development process. When users are unhappy about release delays (as some Debian users were before Lenny was released) they can easily look at the bug list. If a user is pushy they can be invited to fix some bugs themselves, if they investigate the matter and discover that the bugs are too difficult for them to fix then they will probably feel compelled to stop hassling the people who can fix the bugs. Complaints from users regarding the release process can be turned around with the software is late because YOU haven t fixed any bugs . Please note that submitting bug reports is NOT criticism, it is in fact helping the development process (particularly when the bug reports contain information on how to fix the problems). This exposes another significant difference between writing software and writing a book. Even if I was a talented author I would not be able to offer any assistance to an author who had trouble completing a work simply because I wouldn t have the same vision for the plot of a book. But it s not uncommon for software development projects to be forked if the users are unhappy with some aspect of how they work, and it s also not uncommon for a programmer to voluntarily give one of their projects to another developer - sometimes developers try and convince other people to take projects from them when they don t have enough time. Even without changing the ownership of a project, anyone can submit a patch. The patches that I have received from users of my software have generally been of a high quality and it has taken little work to merge them. A final difference is that any programmer can produce a private fork of a project. I have lost count of the number of times that I have produced custom versions of free software for my own personal use or for the use of my clients. Sometimes the modifications that I made were unsuitable for wider use (IE they broke things for usage cases that I didn t care about) so I just privately made the software do what I needed. This makes the upstream developer (or team) and other contributors irrelevant to fixing the problem in question. Now this is not to say that the free software community is perfect. But it seems that certain types of bad behavior that we see demonstrated in the sci-fi fandom circles are uncommon and the people who demonstrate them are significantly outnumbered by people who oppose such actions. Publicly criticising a programmer for watching the NFL in their spare time is akin to holding a sign saying kick me . As I am a fan of science fiction (and hope to write some in the near future) I wonder whether the sci-fi community could be changed to have a culture more like that of the free software community. The web site 365tomorrows.com publishes a short sci-fi story every day [4], they have been doing this for a few years now. They started out publishing work by a small team of authors with the aim of writing a story a day for a year. Now they also take submissions from users and they have a forum for budding authors to discuss their work. The site Protagonize.com is based on the idea of collaborative fiction [5]. I wonder whether the increasing number of sci-fi fans who join such forums will change the nature of the community. Maybe when a book is delayed the fans could take comfort in an increasing amount of fan-fiction based on characters from that book, I expect that the fan fiction will probably not impress all authors - and some may be offended by having their characters used in pornographic fan fiction. But that might just lead to a market shift towards authors who are not overly concerned about such things. When attending sci-fi book signings I ve noticed fans who appear to have stalking tendencies. I have been wondering how much of this is another symptom of the same problem and how much of it is due to science fiction attracting some people who don t understand the concept of personal space and who haven t considered the mathematics of interactions between famous people and fans - as one of many fans the famous person will not remember your fan letters or whether they have met you before, it s not possible. I look forward to the day when a well known author can have lunch with some fans after a book signing and not be afraid of weird fans. I wonder if Cory Doctorow [6] already has such cool fans, Cory seems to be one of the leaders in the new open way of writing and releasing sci-fi so it seems likely that he would be among the first to reap the benefits. I m sure that most authors would like to have lunch or dinner with their fans if they could be assured that the probability of getting stuck with a nutty fan was extremely low.

Wouter Verhelst: COBOL

According to The Registry, COBOL is now slightly over 50 years old. I wish I could say I never wrote a single line of COBOL, but I can't. In college, I was taught COBOL for three long years. Even though I've managed to purge most of that knowledge from my brain (good riddance), I still have nightmares about the horrible way in which one writes COBOL. I wish I could say COBOL is dying, but it isn't. People still write new COBOL today. I mean, seriously, there's a standard called 'COBOL 2002. 'Nuff said. I wish I could say COBOL isn't a problem, but it's the sole source of most of the Y2K issues way back when. Its features invite many of the kind of horribly wrong coding styles which cause bugs; Y2K is just one. I wish I could say COBOL code is pretty, but it isn't. Just to give one example: the first 7 characters of every line of code in each and every COBOL program are 0x20. That is, space. Nothing. Utterly useless. There is no reason for this in modern environments, yet it's still there. Note why I dislike significant whitespace. Another example is the fact that you can't just say 'foo = 1 + 1'. Instead, you have to say 'MOVE 1 TO FOO', and then add a second statement that says 'ADD 1 TO FOO'. I'll just use english if I want to write a novel, thank you. I wish I could say I didn't care about COBOL anymore, but for some weird reason, I do. And just in case you were confused: no, I do not have warm, fuzzy feelings for COBOL. If there was a button that would nuke all the COBOL code on the planet, would I push it? Probably not immediately. I'm sure it would take time for me to recover from the sheer joy of knowing that this button actually exists, before I'll be physically capable of keeping my hands steady for the amount of time required for me to actually be able to push it. COBOL is proof of either the absense of any deity, or the existence of the devil. Given the chance, no sane person, let alone a sane deity, would allow the creation of such an abomination. At least I don't have to deal with it anymore. That part is not going to change. Ever.

Russell Coker: LG U990 Viewty

I have just got a LG U990 Viewty mobile phone [1]. It s a 3G phone and came free on the $29 monthly cap plan from Three (minimum monthly spend is $29 - but this is free if you use $29 per month). My previous plan was the $29 cap but had a minimum spend of $20 per month, as I never happened to use less than $29 per month I am not paying more. For a modern mobile phone the actual phone functionality is a sideline. If a device was strictly designed to be a phone then I think it would be very similar to the Nokia phones that were available 5+ years ago - the Nokia I had in 1999 performed every phone function that I desired of it. Like most modern phones the LG U990 Viewty suffers in it s phone functionality from the desire to make it do non-phone tasks and from the desire to cripple it to meet the desires of the carriers (not the desires of the users). For example the home screen will always have at least two buttons for paid Three services and I have no configuration option to remove them. Replacing them with speed dial options for a couple of numbers that I regularly call would be handy. As the main screen is a touch-screen there is no excuse for this, they should allow the software to be reconfigured with more useful options. Eventually the Android will kill most of the other phones and this problem of phones being designed to suit the telephone companies instead of the users will be solved. One of the most annoying mis-features of the phone is that it doesn t properly handle address book entries with multiple phone numbers. One of my friends turns his mobile off when he is at home, so I regularly call his mobile and then immediately call his home number if the mobile is unavailable. With my previous two mobile phones I could press the dial button to bring up the list of previous calls and then use the arrow buttons to select from the other numbers that are attached to the same address-book entry, so with three button presses I would be dialing his other number. With the Viewty I have to go back to the address book. The compelling feature of the Viewty is the camera and display. It has a 5MP camera which makes it the second highest resolution camera-phone offered by Three - the LG Renoir (KC910) has 8MP but needs a $99 plan for it to be free. It also has a 240*320 resolution touch-screen display (in a quick search in January when I bought my phone the best resolution display I could find on a phone is 240*400 in the LG Renoir). While the camera is documented as being 5mp there are no specs available about the resolution of the CCD. I want to use the native resolution of the CCD for pictures (I think interpolation is a waste of space). The CCD might actually be 5mp, a picture of the 1400*1050 resolution screen on my Thinkpad allows me to read all the text even when small fonts are in use, so the CCD resolution must be significantly greater than 1400*1050. This is a really important feature as the Viewty will work well for making screen-shots for bug reports about crashed computers (several of my clients have expressed interest in getting one after seeing such a demonstration). One annoying problem is that the camera software takes a while to load, my trusty Sony digital camera starts a lot faster and the LG U890 phone I used for the past two years is also a lot faster and more convenient. This won t work well for photographing unexpected events. When I am traveling by public transport I will photograph the relevant pages of my street directory as I can zoom in to photos of the maps and read the street names, it saves some weight when traveling. The 2G micro-SD card (which incidentally cost $10 from OfficeWorks) will allow me to store a lot of maps. One interesting feature is the video recording capabilities. It can do 640*480 resolution at 30fps (which is pretty good) and 320*240 resolution at 120fps (they claim that you can film a balloon popping). In my quick tests the standard 640*480*30fps mode works well, but the 120fps mode requires much brighter light than most of my test environments, so I have not yet got it working properly. The phone has a reasonable voice recording function, it can record considerably more than 34 hours of audio and the quality is reasonably good if you use an external microphone. It is however quite poor if you use the built-in microphone for a dictaphone function, it seems that quality is poor at any distance. I had wanted to record my LCA mini-conf talks with my phone but unfortunately forgot to bring the adapter for the external microphone. It s a pity that the phone doesn t have a standard microphone socket as I have misplaced my Viewty microphone, when designing the phone they should assume that misplacing attachments is a common occurrence and design it to use common parts. I recently spoke to a journalist who uses his mobile phone to record interviews. He said that his phone supported phone calls, voice recording, and taking pictures - all the essential tasks for his work. It seems that the Viewty would be better than most phones for journalistic work apart from the issue of low quality voice recording when you have misplaced your external microphone. The text editor is unfriendly in the keyboard mode (I have not tried hand-writing recognition), one thing I don t like is the fact that the letters jump when you press them. This does allow changing a letter by moving the stylus before releasing the press (some people consider this a great feature). There are no cursor control keys (which is a serious omission), and the keyboard doesn t resemble a real keyboard. My iPaQ is far better for writing (I once wrote a full-length magazine article on an iPaQ). The stylus is quite strange and interesting. In the picture of the front of the phone the stylus is compacted with it s lid on. In the picture of the back of the phone the stylus is extended with the lid off. The end of the stylus clips in to the lid so that removing the lid drags the central part out of the body. It s an interesting design and with the string on the lid allows the stylus to be attached to the phone when it s not being used. But I have never used it. Even with an iPaQ (which had a proper stylus that attached firmly inside the body of the device) I often used a fingernail on the touch screen. I have not felt the need to ever use a stylus with my Viewty. One final noteworthy thing is the support for Google services. It seems to have client support for YouTube, Google Maps, GMail, and Blogger. This seems to be a major win for Google, the Viewty is one of the most popular phones at the moment and I expect that lots of people who buy them will now have an incentive to use the Google services. Between these sorts of deals and the Android I think that it will be necessary to have some sort of anti-trust action against Google. Google are generally doing good things for the users. I have been quite satisfied to use Google search, Google advertising on my blog, and Gmail. Also I have been moderately happy with Blogger (it was good when I started blogging) and Google Maps is useful on occasion. So generally I am happy with Google, but monopolies are bad for the users so I think that if things continue on their current trend then Google may have to be split into several little Googlets some time in the next few years.

30 May 2009

Stefano Zacchiroli: kick-starting turbogears 2 packaging

TurboGears 2 in debian ... soon ! After a long incubation, a few days ago TurboGears 2 has been released. Historically, I've been preferring TurboGears over Django for being closer to the open source philosophy of reusing existing components. Since the long 2.0 release was marking a gap with Django, I was eager to test the 2.0, and I was delighted to find it in Debian. Unfortunately, it doesn't seem to be in good shape yet. In particular it lacks several dependencies before it can even be used to quickstart a project, in spite of its presence in unstable. To give an idea of the needed work, after having installed it manually (via easy_install), my virgin /usr/local/lib/python2.5/site-packages/ has been polluted by 26 egg-thingies. I decided to spend some week-end time to start closing the gap, because we cannot lack such an important web framework in Debian (and ... erm, yes, also because I need it :-) ). Here is the current status of what has been ITP-ed / packaged already (by yours truly): Where to Considering, foolishly, the last package as being already done, the way to go is still long:
    zack@usha:~$ ls /usr/local/lib/python2.5/site-packages/ grep -v repoze.what
    AddOns-0.6-py2.5.egg
    BytecodeAssembler-0.3-py2.5.egg
    Catwalk-2.0.2-py2.5.egg
    easy-install.pth
    Extremes-1.1-py2.5.egg
    PEAK_Rules-0.5a1.dev_r2582-py2.5.egg
    prioritized_methods-0.2.1-py2.5.egg
    sprox-0.5.5-py2.5.egg
    sqlalchemy_migrate-0.5.2-py2.5.egg
    SymbolType-1.0-py2.5.egg
    tg.devtools-2.0-py2.5.egg
    tgext.admin-0.2.4-py2.5.egg
    tgext.crud-0.2.4-py2.5.egg
    TurboGears2-2.0-py2.5.egg
    tw.forms-0.9.3-py2.5.egg
    WebFlash-0.1a9-py2.5.egg
    zope.sqlalchemy-0.4-py2.5.egg

Possibly, some of them are already in Debian hidden somewhere, but sure there is still work to be done. If you want to help, you are more then welcome. The rules are simple: all packages will be maintained under the umbrella of the Python Modules Team, but you should be willing to take responsibility as the primary maintainer. Please get in touch with if you are interested, as I'm in turn already in touch with some other very kind volunteers (thanks Enrico and Federico2!) to coordinate who-is-doing-what. Preview packages available What has already been packaged, including a temporary workaround for python-transaction, will be available in experimental after NEW processing. The idea is that nothing will go to unstable until TurboGears 2 will be (proven to be) fully functional. In the meantime, packages are available from my personal APT repository (signed by my key), here are the friendly /etc/apt/sources.list :
    deb http://people.debian.org/~zack/debian zack-unstable/
    deb-src http://people.debian.org/~zack/debian zack-unstable/

Versions are tilde-friendly and shouldn't get in your way when the official packages will hit unstable. Packaging multiple-egg / multiple-upstream packages In all this, I've faced an interesting problem with the python-repoze. who,what -plugins packages. They correspond to a handful of plugins, each of which is about 20/30 Kb. I didn't consider appropriate to prepare 5 different packages, due to archive bloat potential. Hence, you get the usual problems of multiple-upstream packages. To counter some of them, as well as some egg-specific packaging annoyances with multiple upstream, I wrote a couple of very simple helpers: I'm no Python-packaging-guru so, lazyweb, if you spot in this choice something I utterly overlooked, or if you have improvements to suggest, please let me know. Repacking eggs An interesting problem will be faced in trying to integrate the above approach with Python modules that are shipped only as .egg files (i.e., no tarballs, ... yes there are such horrifying things out there). To preserve uniformity, we would need uscan to support --repack ing of .egg files as they were simple .zip archives. Since eggs are .zip archives ... why not?

Michael Prokop: Directory specific shell configuration with Zsh

Now being an official Debian developer I ve the possibility to use $DEBEMAIL=mika@debian.org for my Debian packages. But whereas I want to use that for all my official Debian packages I still want to use $DEBEMAIL=mika@grml.org for my grml related packages. I m a lazy sysadmin and don t want to manually adjust my changelogs depending on the type of package (especially since this might be error-prone) so I needed a generic solution. Grml s Zsh configuration (Debian package grml-etc-core >=0.3.71) provides a solution for Zsh (version 4.3.3 or later): based on directories you can set your own profiles. Demonstration:
% cd /grml/git-grml
chpwd(): Switching to profile: grml
% echo $DEBEMAIL
mika@grml.org
% cd /grml/git-debian
chpwd(): Switching to profile: debian
% echo $DEBEMAIL
mika@debian.org
The configuration is pretty simple:
# define profiles based on directories:
zstyle ':chpwd:profiles:/grml/git-grml( / /*)'       profile grml
zstyle ':chpwd:profiles:/grml/git-debian( / /*)'     profile debian
zstyle ':chpwd:profiles:/home/mika/Customers( / /*)' profile customers
# configuration for profile 'grml':
chpwd_profile_grml()
 
  [[ $ profile  == $ CHPWD_PROFILE  ]] && return 1
  print "chpwd(): Switching to profile: $profile"
  export DEBEMAIL=mika@grml.org
  export GIT_AUTHOR_EMAIL="mika@grml.org"
  export GIT_COMMITTER_EMAIL="mika@grml.org"
 
# configuration for profile 'debian':
chpwd_profile_debian()
 
  [[ $ profile  == $ CHPWD_PROFILE  ]] && return 1
  print "chpwd(): Switching to profile: $profile"
  export DEBEMAIL=mika@debian.org
  export GIT_AUTHOR_EMAIL="mika@debian.org"
  export GIT_COMMITTER_EMAIL="mika@debian.org"
 
# configuration for profile 'customers':
chpwd_profile_customers()
 
  [[ $ profile  == $ CHPWD_PROFILE  ]] && return 1
  print "chpwd(): Switching to profile: $profile"
  export TELEPHONE=[...]
  export MAIL=[...]
 
Kudos to Frank for implementing this lovely feature in grml s zshrc. Look for chpwd_profiles() in grml s zsh config for further documentation on this feature.

Christine Spang: microblogging

At some point in the recent past I gave in and signed up for Identica and Twitter. I'm now simultaneously posting to both these services and Facebook via Gwibber. (I grabbed the Ubuntu package from Launchpad since the Debian ITP has been stalled for some time.) In one way, I've been microblogging for some time via Zephyr. But I sometimes feel limited by the fact that it only puts me in touch with the MIT community (and only a subset of that, even), so perhaps Twitter/Identica will prove a better solution.

Dirk Eddelbuettel: JPM Chase Corporate Challenge 2009

The 28th annual JP Morgan Chase Corporate Challenge race took place a couple of days ago May 21. Participation was down from the record of 23,000 runners set last year at around 17,125. With splendid weather, it is always a nice way to start the Memorial day weekend. We fielded a small but spirited team of nine runners. I finished with a decent (hand-stopped) time of 22 minutes and 27.93 seconds for the 3.5 miles -- or a 6:25 min/mile pace. That is among the fasters times but not quite the fastest compared to the other six times I have run this. Most importantly, everybody seems to have had a blast. And we did set a record for longer post-race party which sets a nice precedent for 2010.

Marc 'Zugschlus' Haber: The grub drama

This is a rant. A rant which goes to the grub maintainers, and one that could go nearly identically to many people in the KDE environment or many other open source projects. I really like grub. I really like grub 0.97 despite that it s been unmaintained for years and not booting on two of my important machines. I should like grub 2 because its configuration looks more straightforward and for its better features - direct booting of .iso images, from LVM and RAID. But actually, I have learned to hate grub 2 since it is not finished and badly documented, and that its existence is already being used as an excuse for grub 0 s development having stopped years ago (and it being renamed to grub-legacy to clearly show that it s the unloved child) - and things looks like this is not going to change any time soon. grub 0.97 is the de facto standard in booting Linux, its most prevalent advantages probably being its capability of reading filesystems and its command shell, which allows booting kernels and other software that one wasn t planning to boot when the system was shut down. There is a small fraction of people who still use lilo despite its disadvantages, for example on achitectures that are not supported by grub, if one wants to have the boot FS on LVM, if one just prefers lilo for non-technical reasons or on systems that grub doesn t work on. The last three systems, by the way, that I still run lilo on are Pyramid pizzaboxes with supermicro-made mainboards from 2002, which just refuse to boot if grub is used and the vendor just says that grub 0.97 is still beta and that they do not support grub on their systems: Use Lilo. Grub s last upstream release, 0.97, dates back to 2005, and upstream has stopped developing this grub release years ago. grub 2, the new way to go, had its first release in 2003, and the last stable release that people are supposed to use dates back to februar 2008. grub 2 is clearly a step in the right direction, but there are still bugs, and the most prevalent show stopper against grub 2 is the almost complete absence of any documentation. I have, for example, not yet found good documentation about the new config file format, which is a completely different deal from what we are used to with grub 0. The commands manual pages do little more than list the (obvious!) command line options without giving much more information about what the actually do, let alone usage examples. The documentation (which used to be really excellent for grub 0), has been moved to a wiki, which has some useful content but cannot keep up with the excellent documentation that grub 0 used to have. Additionally, grub2 does not seem to have a grub shell which can be called from within the host system, which makes installing grub 2 in exotic situations much harder than it used to be with grub 0. I am also quite disappointed that despite the grub upstream web pages claiming that grub 2 is under active development, the last release is more than fifteen months in the past while there are still so many shortcomings. The Debian maintainers of grub 2 are regularly packaging grub 2 subversion snapshots, which is a good idea in face of upstream s release policy which is quite different from what s being told in the wiki. I am, however, reluctant to rely on a development snapshot of a software to boot productive systems. Guys, you cannot declare a widely used, well-documented software legacy when the replacement is still in a nearly unuseable shape and the docs are hidden in the helpful files named .c and .h. This is simply unacceptable. It s a drama that the docs are still in such a pitiful shape fifteen months after your last release. But, grub is not the only free software project that so blatantly scores developer fun over the needs of the end user. I know that it is a boring task to fix bugs and to work around design shortcomings in the existing version of software, and that it is much more fun to hack away on new designs and new code, but dropping old versions before the new version is up to par with the old one is annoying the users and driving people away to other software packages. This applies to grub, a quite low-level program, but also to larger and more complex projects like KDE who has recently pulled the same stunt by dropping development and support for KDE 3.5 while KDE 4 is still missing many many features and KDE 3.5 s stablity, thus driving people (back) to GNOME or alternative desktop environments. This we give a damn about people who actually use our software, our compilers run without users as well attitude is a show of disrespect for the user base and makes me very sad as this does indeed show that free software is in no way superior to its commercial counterparts. Sadly, I do not have a solution for this issue since I know that free software developers are usually volunteers who do not have to report to anybody, and that they do not need to care for their user base. But I hope that people begin to reconsider their attitude and think about what kind of horrible damage is being done to the user base and to free software s reputation.

Steve McIntyre: age++

On Thursday it was my birthday, and I fulfilled a promise I made to myself many years ago. I've always been a car nut, and when I was about 6 years old and had posters of Lamborghinis on my bedroom walls I promised myself that by the time I reached 35 I would have one. That would be long enough in the future that it might happen, but I'd still be young enough to still be able to appreciate it. Well, as it got to the beginning of this year I realised I was running out of time and my bank balance didn't quite look large enough to buy a Lamborghini. So, I cheated a little and I hired one for the day on Thursday. Toy! More pictures here. I had one car booked a few weeks ago, but unfortunately the company involved had to let me down as the car developed problems a couple of days before I was due to have it and needed to go away for repairs. I'm not going to complain too much about them, as it wasn't their fault and they did their level best to make things right: offering a different car, a refund or a later replacement hire. But dammit! I wanted the Lamborghini and I wanted it on my birthday! I quickly rang around on Wednesday to find somebody else, and thankfully the folks at Signature Car Hire were very helpful, sorting out a car with just a few hours' notice. Yay! When they guys turned up on Thursday morning to deliver the car to my office, I couldn't stop myself grinning like the 6-year old boy whose promise I was keeping. Here was a gorgeous car, and it was mine for the day!!! I planned a route for the day to take in some nice driving roads and, importantly, some tunnels. I've watched Top Gear on the TV and I read Autocar every week, and they had both said that taking a fast sports car through a tunnel with the roof down was a good thing to do. I now know that's an understatement! As a whole, the car was awesome: the acceleration, the handling, the brakes were all wonderful. But, as I found out on that day, the noise from the engine was just astonishing. I went through several tunnels that day, heading down past Baldock to Hatfield and back, and at full throttle I could not help but laugh out loud at the music coming from the 5-litre V10 orchestra just inches behind me. It's an experience I'm never going to forget. grin. I also gave short rides to quite a few of the people at work; oddly I had lots of friends that day... :-) I'd also planned other things for the day. I bought myself a new camera in advance to make sure I got some nice photos of the day. Then at the end of the day I went out to the pub with a huge group of friends and I was forced to drink lots of beer. It was horrible, horrible I tell you.

Joost Yervante Damad: how to set a serial port at MIDI speed in linux

Linux serial ports only work at standard speeds by default. MIDI runs at 31250 baud, which is not a standard speed. However there are tricks to get custom speeds, but documentation is quite fuzzy. This is a simple recipe that worked for me with an FT232 USB-Serial board.

Check the baud base of the device:

$ setserial -g -a /dev/ttyUSB0
/dev/ttyUSB0, Line 0, UART: unknown, Port: 0x0000, IRQ: 0
Baud_base: 24000000, close_delay: 0, divisor: 0
closing_wait: infinite
Flags: spd_normal low_latency

As you can see the baud base is 24000000 here.
Next calculate the divisor by dividing the baud_base you see here by the speed you want.
In my case 24000000/31250=768.

Apply the new setting:

$ setserial -v /dev/ttyUSB0 spd_cust divisor 768

Next start your serial application, you might want to make sure it is already set to the correct speed before you do the above changes else it might destroy your settings. The correct speed is 38400 baud, which is now aliased to 31250.

With minicom just use the menu (CTRL-A Z) to change the settings.

Screen can be used like this:

screen /dev/ttyUSB0 38400

Exit screen by pressing CTRL-A CTRL-\

Sweet.

Andrew Pollock: [life] On obtaining an E-3 visa from the US Consulate in Vancouver

I've already had one enquiry, so I guess I'd better get writing. Despite the lawyers being really down on using this Consulate, I have to say that from my experience, it's been the easiest so far. Or I'm just getting used to the rigmarole. The one screw up was I didn't read the instructions thoroughly enough, and we rocked up to the Scotia Bank branch to deposit the visa application fee in their bank account, and I thought the fee was quoted in Canadian dollars. Turns out, it was quoted in US dollars. It wasn't the end of the world that we didn't have cash, they could sell us US dollars on a credit card, except the bank would only accept Visa card, not Mastercard. I haven't had to deal with such bullshit since the great Cabcharge/Visa card war in Australia in the late 90's. Moral of that story: read the instructions multiple times. Carry one of each flavour of credit card when traveling internationally, just in case. That minor transactional hiccup aside, everything went fine. The Consulate itself does all of the pre-interview shenanigans on one floor (the bulk of the waiting around, the fingerprinting and photographing) and then they shoot you up to another floor for the "interview". Annoyingly, they run you through the metal detectors again on the higher floor. The interview was basically "So you work for Google? How about the food?" and then he told us to come back the next day to pick up the passports. I think we were in and out in about three, three and a half hours. I have to say, the ability to collect the passports from the Consulate later is a real boon. When you're in a foreign country, the last thing you want to deal with is any glitches in the postal system, or trying to get mail delivered to a hotel. This alone makes me like the Vancouver Consulate the best. We went back at the 2:15pm allotted "passport pick up time", and they came down to the security checkpoint on the ground floor and you show your receipt and they give you your passports back. They tell you to step outside and check it, and come back if there's any problems, so I presume you can at least talk to a human at that point if there are any problems. This is my third E-3 visa. They did not ask to see any of my supporting paperwork showing ties with Australia or anything else that I had in my giant wad of paperwork, just the approved Labor Condition Application, which they kept and returned with the passport the next day.

Christian Perrier: Yet another silly name change for a country

After Venezuela changed its official "short" name from "Venezuela" to "Venezuela, Bolivarian Republic of", it's now the turn of Bolivia to take the same silly decision. From now, Bolivia's "short" name is "Bolivia, Plurinational state of". Don't take me wrong. I'm fine with such changes in official country names...when it comes at the official long name of the country. Most long names have some kind of political meaning. For Bolivia, the change is meant to reflect the consideration of the current government of the country for all "minorities" of Bolivia. That's for sure definitely any country's decision. But, for $ DEITY 's sake, why change the *short* name? For everybody, Bolivia is "Bolivia" and Venezuela is "Venezuela" and not some funky long and politically correct (or incorrect) name! So, once again, the iso-codes package maintainers will use a new "common name" for another country so that lists of countries that display in various places do not show ridiculously long names for some countries while others have the name...everybody uses. Remember "Taiwan" back in 2004?

Francois Marier: Troubleshooting Postgres Performance Problems

There are great resources to help tune PostgresQL for performance, but say you've got a Postgres database which suddenly becomes very slow. How can you figure out what's going on?
General system stateA slow database server will typically be running out of:
So the first thing you should do is to get an overview of the system state using common UNIX tools such as:
If you notice that memory is exhausted, you could play with the following postgresql.conf settings:
shared_buffers
work_mem
effective_cache_size
If you see a lot of I/O, then try adjusting these settings:
checkpoint_segments
autovacuum_naptime
wal_buffers
Finding slow queries
On the database side, start by determining whether:
To do this, make sure that the following setting is turned on in your postgresql.conf:
stats_command_string = on
Then fire up psql dbname and look at the currently executing queries:
SELECT usename, current_query, query_start
FROM pg_stat_activity;
(You can also add the procid column to the query if you want to map a query to the process ID you see in top.)

If you can't get anything useful out of pg_stat_activity, you may want to enable slow query logging by adding this to your postgresql.conf:
log_min_duration_statement = 100
(All queries which take more than 100 ms to execute will be printed to the main Postgres log file.)
Examining a particular queryOnce you have identified a slow query, you can time it by running this command before executing the query manually:
\timing
To get an idea of where Postgres spends its time when executing that query, look at the query plan:
EXPLAIN your_query;
The numbers you see there are estimates. To run the query and get actual numbers, use this instead:
EXPLAIN ANALYZE your_query;
If you still can't figure out why a certain query takes so long, have a look at the ratio between live and dead rows in the relevant tables:
ANALYZE VERBOSE tablename;
Having too many dead rows is often a sign of insufficient vacuuming. You might want to turn autovacuum on if it isn't already, or to make it a bit more aggressive by tweaking these settings:
autovacuum_vacuum_scale_factor
autovacuum_analyze_scale_factor

Troubleshooting database performance is a bit of a black art, many thanks to Mark Kirkwood for sharing his Postgres wisdom with me.

If there's anything else you have successfully used to find the cause of your performance woes, please feel free to leave a comment.

29 May 2009

Russell Coker: Links May 2009

An interesting opinion post in the NY Times describes the research on early education and how it can affect IQ [1]. Among other things children from poor families who are adopted into upper middle-class families tend to end up with higher IQ scores. The article notes that half the population in 1917 would be regarded as mentally retarded by today s standards - finally we have an explanation for WW1! Two Dominos employees tarnish the brand s image with a prank video on Youtube [2]. The next obvious step is for activists to seek jobs at such companies for the purpose of influencing companies. The animal rights protesters outside KFC stores haven t achieved much, but if they worked for KFC and made some nasty videos they would really encourage a change of action. I predict that chain stores will be spending a lot more on security and background checks for their employees in the near future. Cory Doctorow has written an amusing article titled Big Entertainment Wants to Party Like It s 1996 about how the entertainment industry is killing itself by conducting back-room negotiations about new copyright laws [3]. Nate Silver gave a TED talk about racism in elections [4]. The most interesting point was demonstrating statistically that people who don t meet people of other races tend to be more racist. It seems to me that the use of the X-Face: header in email and the use of HackerGotchi in Planets can help reduce the level of racism on the Internet. Cory Doctorow writes about his Geeky writing [5]. His idea for an organised system for donating books to libraries will hopefully be fully implemented soon - it should be easy to do and the incremental costs will decrease as the scope increases. Cory Doctorow writes about the perverse laws that protect criticism of copyright works but stifle praise [6]. In a similar note he has documented a plan for trademark and copyright holders to allow fans to create derivitive works while preserving the original rights AND sharing the profits [7]. So if Cory s idea became popular someone who wanted to create some art work based on a Coke bottle (which is trademarked) could pay the Coca-Cola company a reasonable rate, include an appropriate disclaimer, and things would work out well for everyone. Also this would allow small artists to develop new products that could be used by the large companies (I m sure that anyone who legally released an artwork that turned out to be an effective advert for Coke would receive a lucrative job offer). Bruce Schneier s blog has an interesting article about the poor quality of software used for breath alcohol detectors [8]. It s a great concern that innocent people are being punished due to bad software, but it s only a small part of the problems with the legal system. Mary Roach gave a TED talk 10 Things You Didn t Know About Orgasm [9]. Not as insightful as the usual TED talk, but strange and interesting.

Erich Schubert: Java hacks: Generics and toArray

Arrays and Generics in Java do not mix very well. In order to create an array, you need to know the object class the array is supposed to store.Arrays in Java are special: they can efficiently store primitive data types. The expected difference in efficiency between byte[] and Byte[] is pretty big (of course some good VM might optimize) for obvious reasons (think of: references, garbage collection, pointer sizes, ...).This is probably why you need to know the type before creating an array (because an array of primitive types such as byte will be different from one that stores objects of some kind).In particular, the following Java code
  String[] foo = (String[]) new Object[0];
results in a run time error ("[Ljava.lang.Object; cannot be cast to [Ljava.lang.String;"). But it gets more confusing when you introduce generics:
public static <T> T[] test()  
  T[] te = (T[]) new Object[0];
  System.err.println(te.length);
  return te;
 String[] foobar = test();
will print "0", then throw the same run time error in the foobar line.What happens here is that in the test() method, T actually is replaced with "Object" at compile time. Thus the array type works just fine, and so does the call to te.length. Upon returning, it is then cast into a String[] array and fails.Now here comes a crazy Java hack:
public static <T> T[] test(T... ts)  
  T[] te = (T[]) java.lang.reflect.Array.
      newInstance(ts.getClass().getComponentType(), 0);
  System.err.println(te.length);
  return te;
 String[] foobar = test();
The exception is gone, foobar is of the proper type now!A result of discovering this hack are these two methods:
public static <T> T[] newArrayOfNull(int len, T... ts)  
  // Varargs hack!
  return (T[]) java.lang.reflect.Array.
      newInstance(ts.getClass().getComponentType(), len);
 public static <T> T[] toArray(Collection<T> coll, T... ts)  
  // Varargs hack!
  return coll.toArray(ts);
 
Notice how elegant the last method looks - and it finally allows you to do toArray(collection) instead of collection.toArray(new WhateverClassTheCollectionHas[0]).Note that this is still a hack, and may or may not work with all Java compilers, JREs and/or Java versions.Update: Note that this 'hack' is also not transitive. The context calling toArray needs to know the object type at compile time. So it doesn't save you much more than writing "new KnownClass[0]" etc.Update: So I'm actually not using this - it's just a hack, and often quite hackish. The problem is that when you call e.g. toArray in an Generics context, it will actually create an array of "Object", so it makes much more sense to verbosely specify the class you want to use for the arrays (and get some reliability in use back).

Neil Williams: IRC, timewasting and life balance

I've never particularly liked IRC because it is as intrusive as a telephone - flashing away and demanding attention. I much prefer email, which I can answer when I choose to allocate the time to handle it. I have a lot of different tasks ongoing and an interruption about a completely different topic makes it difficult to retain the train of thought required to pursue the original task. I've cut back on my IRC time since Christmas but I'm cutting back further.

It's part and parcel of managing my software work without it excluding other things I like doing too. The good weather is a great excuse to get away from the laptop/desktop, so I'll be doing a lot more impromptu hikes and day trips. I'm resurrecting other non-computer activities too - it's been too long.

So, don't expect to see me on IRC - except during a conference etc. If you need me, as ever, use email and preferably one of the many mailing lists to which I'm subscribed. If the query applies to any of my packages, a bug report is far better than an email or I could forget. (Telephone is for business/close friends only, again except during conferences.)

Ingo Juergensmann: Please sign the epetition against Internet censorship in Germany

If you are living in Germany, please help preventing Internet censorship in Germany by signing the ePetition now! If you already signed it, ask your friends and family to do so as well! :-)

Biella Coleman: My return

After a fairly tiring (though exciting) semester of teaching, I have packed my books, lots of files, and other things I Can t Live Without, and shipped myself via Jet Blue to Puerto Rico where I will spend my summer. My reason for being here is simple: I want to see my mother who continues to live in a state of minimalism at a nursing home with Alzheimers. Coming home is never easy. Seeing her for the first time after an absence of months is especially hard and expect will continue to be as such, so long as I have periods away from her like I do. I never imagined that I would live between two cities as I do, but in general I am grateful I have a job that allows me to bridge these two places. When I am in NYC, I am so ridiculously busy (being an assistant professor is not unlike being a medical resident and doctor except the tempo slows down during our summers, at least to some extent) that I don t for better or for worse have my mother always perched on my mind. But being here is a different story. Not only do I have to structure my own time (so my mind can wander easily into those nether regions), but the house I stay in is nothing but filled with memories of my mom, most especially from the last 4 years she spent here with Alzheimers Along with visiting my mom, most of my time will be occupied in front of the computer working on the first draft of my book manuscript, which is due on September 15, 2009, 2 days before my birthday. I think I can hack it given that I have lots of time on my hands and when I get into work mode I can get a lot done. But the first month or so is going to be really really rough. I always find the transition into stare at your monitor mode for the entire day sort of tough. Puerto Rico also has a host of magnetic distractions like the garden and the beach, which I don t love but in fact adore. I also don t really associate the tropics/this house/the summer with mental work, but I will spend the next few weeks reversing that association or else I will be in deep trouble! Right now I am setting up the office, struggling to find the right a/c unit for the space, and in a few days I hope to be writing away

Biella Coleman: My return

After a fairly tiring (though exciting) semester of teaching, I have packed my books, lots of files, and other things I Can t Live Without, and shipped myself via Jet Blue to Puerto Rico where I will spend my summer. My reason for being here is simple: I want to see my mother who continues to live in a state of minimalism at a nursing home with Alzheimers. Coming home is never easy. Seeing her for the first time after an absence of months is especially hard and expect will continue to be as such, so long as I have periods away from her like I do. I never imagined that I would live between two cities as I do, but in general I am grateful I have a job that allows me to bridge these two places. When I am in NYC, I am so ridiculously busy (being an assistant professor is not unlike being a medical resident and doctor except the tempo slows down during our summers, at least to some extent) that I don t for better or for worse have my mother always perched on my mind. But being here is a different story. Not only do I have to structure my own time (so my mind can wander easily into those nether regions), but the house I stay in is nothing but filled with memories of my mom, most especially from the last 4 years she spent here with Alzheimers Along with visiting my mom, most of my time will be occupied in front of the computer working on the first draft of my book manuscript, which is due on September 15, 2009, 2 days before my birthday. I think I can hack it given that I have lots of time on my hands and when I get into work mode I can get a lot done. But the first month or so is going to be really really rough. I always find the transition into stare at your monitor mode for the entire day sort of tough. Puerto Rico also has a host of magnetic distractions like the garden and the beach, which I don t love but in fact adore. I also don t really associate the tropics/this house/the summer with mental work, but I will spend the next few weeks reversing that association or else I will be in deep trouble! Right now I am setting up the office, struggling to find the right a/c unit for the space, and in a few days I hope to be writing away

Next.

Previous.