Search Results: "reg"

6 December 2008

Mike Hommey: On open source and mobile phones

No software is perfect, and most users happen to have some itches they would like to scratch on them. I don’t know anyone who hasn’t thought some day “why can’t that software do <some feature>”. But not everybody is able to scratch these itches. For a starter, most people aren’t developers, or aren’t able to modify source code to reach some known goal, be it fixing a bug or implementing a missing feature. But even when they are able to do so, most applications can’t be fixed or improved easily: most widespread applications are still proprietary software where access to the source code is either impossible or prohibitively expensive. That’s the good thing about open source and free software: you are able to itch these scratches yourself ; and as a free software user and developer, I often do so in the software I use. I must say it’s even frustrating when you are used to this possibility but end up on damn stupid bugs or limitations you can’t fix in proprietary software. I have quite a bunch of such examples, the most recent ones being with VMware ESX, but that’s another story. Mobile phones are everywhere nowadays. In some countries, there are even more mobile phones than people. A huge amount of people are using them daily. Or maybe should I say “endure”. When you go to shop for a laptop, digital camera, or digital portable music player, you can find demonstration models you can fiddle with, see what the user interface looks like, how it behaves, see if whatever feature you like on such devices is present and how easily you can do what you want. With a mobile phone, you don’t have all that (at least in France and Japan, where I experienced this). You’re lucky if one of the sellers owns the exact model you’d like to test and lends it to you for a minute. Most of the time, the “thing” to watch is a plastic model, not even remotely related to the real thing in colour, weight… and obviously, it has as much use as a brick (not even so, actually, as you can’t, for instance, break windows with it). And most of the time, you end up regretting the choice you made, because the UI is so unuseable, or so slow after a while, or so buggy, etc. More and more phones nowadays have multimedia as their main feature. While it can be nice, especially when you see how some are pretty good at it, and when you consider it avoids you to carry both a phone and a multimedia device. Sadly, they usually forget to be good phones, too. Sometimes even, multimedia(-related) features are crippled by the carriers. As an example, my phone, while able to read mp3 music, has been limited by the carrier to refuse to use them as ring tones, while, according to the maker and to various forums, the phone is perfectly able to do so. It’s even worse than that, because even converted as 3gp, which it otherwise allows as ring tones if you download them at a ridiculous price, it still doesn’t allow to put your music as a ring tone. You got it, features in phones are also limited for big money-making. Fortunately, there is a hole in my phone where I can set a 3gp as ring tone if I do it from the music player instead of the file browser. Most articles I read about Android, or the iPhone SDK, relate how they are great opportunities to enable developers to provide nice applications for users, using the provided frameworks. But the thing is : they’re only applications. An application is not the core of the phone. An application won’t take care of ring tones, or vibration types. And that’s what open source in mobile phones should also allow users to do. That’s what it all should be about. As a user and free-software developer, I would like to enhance my phone like I can enhance my laptop. For instance, my phone will only allow crappy polyphonic tones for SMS or e-mail reception. Digital music is only available when you receive a call. Why ? There is obviously nothing preventing digital musing when receiving SMS or e-mail, except a stupid software restriction that could be fixed if the firmware were open-source. Very few phones in France appear to allow japanese input, let alone proper display of japanese characters in e-mail or on the web. My previous phone was able to display japanese correctly. My current phone doesn’t. Neither of them is able to input japanese (which is not unexpected in France, actually). I know the iPhone does both. Anyways, these are informations very hard to get a hand on before buying a model, and something that open-source would allow me to fix. Even better, it would allow me to take the pieces from the asian version of the same (or similar) phone which probably has the functionality. The phone I had when I was living in Japan had a nice feature that I haven’t seen in any phone in France (but again, this kind of information is very hard to get): different vibration types. For example, you had a vibration type making short vibrations, another one long vibrations, yet another one 3 short vibrations, a short pause, then 3 short vibrations again, etc. This is very useful when you disable ring tones during a meeting or some other occasions where you don’t want to bother the people around you with your phone ringing: depending on how the phone vibrates, you know whether you have to take the call or can ignore it, or if it’s only an e-mail, without taking a look at the phone. Again, an open-source firmware would allow me to implement this. I’m sad so much of what open-source can bring to mobile phones is so badly publicized.

Erich Schubert: On Facebook

I've become a Facebook user. Caused by most of my dancing friends being there: Facebook is a convenient and easy way to keep some contact with all those people that aren't like your core friends, but still friends, and that are all around the world.I've also toyed around with the Facebook API, and actually written two small applications for it. From a technical point of view, I'm actually quite impressed by Facebook.Yes, Facebook seems to have quite some load trouble these days. Sometimes it's very slow. Sometimes it just malfunctions (with being kicked out and having to re-login being just the least). These days, messages were distorted every now and then, some app messages were always missing the last few words.What seriously does impress me is the architecture they use with third party applications. It's designed around cachability (so profile pages aren't slowed down by slow or broken third party applications) and they do jump some hoops to allow application writers do a lot of things while preventing them to disturb other applications or the core functionality.For all I know, Facebook is the first major thing to do CSS and JavaScript rewriting. Data produced by third party applications is fed through a very smart parser and rewriter that allows an impressively large subset of CSS and JavaScript to be used without the developers having to pay attention to not producing conflicts. In CSS, rules are prefixed with a selector to restrict them to their applications scope. In JavaScript, object references are uniquified, and the convenience functions you have for interacting and accessing nodes (including functions to do common things such as modify CSS class assignments) take care of all that. Access to the raw JavaScript methods is filtered, so you can't e.g. use parentNode to get access to objects outside of your scope. At least in theory.Much of this is for the benefit of users: applications are not allowed to do annoying animations unless the user has just interacted with them; apps also can't modify or disturb others, or read data from other applications via DOM.Well, of course there might be one or another security issue still there; some of these things might also be related to the performance issues of Facebook recently. And of course there are bugs. Lots. A couple of things still need to be thought through properly (e.g. aggregation of feed messages with multiple targets, localization functionality for applications, finer grained control of data access). But their CSS and JavaScript rewriting is really cool.

4 December 2008

Jon Dowland: planet spamming

I've just migrated hosts for my website and managed to spam at least one aggregator in the process. (I think this is because git does not preserve timestamp metadata for blobs but I haven't investigated properly yet). Apologies to those affected. Luckily, I have my feed configured to only show the most recent 5 items, so the impact was quite small. I think this should be the default for applications that generate feeds for precicely this reason: It is for blosxom (#373570) and pyblosxom (#373658), and ikiwiki specifies a limit, at least. Asides from feeds, I think blog web pages that display an enormous number of posts on the front page (or inside category pages) are demonstrating some pretty bad usability. I thought that Jakob Nielsen may have included this in his Weblog Usability: The Top Ten Design Mistakes, but he didn't. Thomas Watson writes we have celebrated with finally scrapping our blog archive. Instead we are now just showing the latest 50 articles on the front page, followed by a link to the previous 50., which pretty much the opposite of what I'd advise. The point he makes, though, is that the alternative (an ever-expanding "archive", accessed by calendar) is bad form, too. That is one of the points in Nielsen's list. Jonathan Boutelle describes an interesting compromise which he calls the "Mullet style" - a short list of full posts followed by a list of older posts, containing only their titles and date.

3 December 2008

Jose Carlos Garcia Sogo: Fanatics did it again

Once again, a person died today murdered by terrorists in Spain. He was a businessman, who had a construction company in his town (Azpeitia) at Basque Country region. Close his house, he was murdered by gunshots. Just because his company was a contractor in the construction of AVE infraestructure in Basque Country, which will connect it with Madrid and France. Once again, a family has been hit forever. I was going to include a black strip with the article, but it’s better to see an image.

2 December 2008

Aigars Mahinovs: iPhone review from a free software guy

I got an iPhone on the very first day it was available officially in my country. I have used it every day since then and am now ready to decide how good it really is. Summary: it is a great piece of technology and the only tradeback is the locked down platform. I have abandoned my Palm and am not looking back. Hardware: the hardware is just perfect. 4 buttons (Home, Power, Volume Up and Down) and 1 switch (Mute) are all very clear, easy to find and press and frankly are enough. The touchscreen is reasonably fast and very precise nicely corrected by software that guides your presses toward UI points of interest if such are within your finger. The casing is absolutely solid - there is no gaps, no wobble, nothing. All glossy surfaces attract fingerprints, but a couple passes with a microfiber cloth (included) clean it up easily. The loudspeaker is phenomenal - all sound comes from a single small hole, but it can easily fill a small room with music. Call sound quality is impeccable. You can plug in any headphones, but if you use the included headphones you get a microphone with a button that can pickup and hand up calls, start/pause music and skip to the next track. I also love that the only charger included is a very small power brick that provides a power-only USB port and a USB charging cable. That is a good way to reduce number of power bricks for gadgets. The iPhone 3G (that is what I have) also has a GPS build-in, but let us be clear - it is not good enough to replace a car GPS, the iPhone GPS is slow (takes several seconds to refresh the location and tens of seconds to find it first time) and the precision is measured in meters or even tens of meters. The WiFi is very good at getting a connection and doing it very fast. Oh and there is an acceptable camera. Out of the box software: the software included surely is better than on any other phone I have seen (including Palm). The UI widgets are nicely written specially for a touchscreen interface - the best example is having click wheels instead of drop down menus. It is much easier mentally to flick a wheel up or down then to click on a list, click to scroll the list and then click to select an entry. And the click wheel makes it much easier to correct off-by-one errors. Switches that look like regular physical switches instead of checkboxes is another very useful UI concept. I think a lot of this can be reused on the desktop as well. iTunes: and here comes the bad spot, the lock-in part. To get anywhere with the iPhone you need to associate it with an installation of iTunes. It fails to work in Wine or even in a virtual machine - apparently there are some errors related to avahi or something. Unfortunately, this means that you need wither Windows or MacOS to do nything with your iPhone. iTunes is the only way to backup, restore and upgrade your iPhone, it is the only way to get contacts and music on the iPhone (there is an iTunes Store app in the newest version that allows purchasing music over the air). Even after doing a jailbreak (see further) it is currently not possible to add music to the iPhone from any third party applications, because the iTunes database has a hashed checksum that the build-in iPhone music player (and iTunes) check before doing anything. If the checksum is wrong the only thing you can do is a full restore of the phone from backup. Crypto geeks - please help decrypt that hash, so that we can sync iPhones to amaroK. The iTunes itself is a bit weird - to get music onto the phone you need to create a playlist and then you can sync that playlist over. One of the playlist options is a smart playlist with a random feature and a limit on number of matches. That is the way to get random music from your collection onto the phone. I do dislike the fact that iTunes syncs all the contacts from Google to the phone and not only contacts for a certain group or only those that have a phone number, for example. Search and favorites do help here. Third party software (iTunes Store): it is very easy to install new software on the phone over the WiFi and there is plenty to choose from. You do need a Mac to develop this software which restricts the developer base a bit. The main bases that I needed (or did not know I needed) are covered: Last.fm player, TwitterFon, LockBox password storage, Stanza ebook reader, Shazam music identificator, Wordpress client, BigOven cookbook and lots of fun games. Have not used any paid-for apps due to the fact that iTunes store is not available in Latvia and the official documentation recommends registering with a fake address in UK or US. :) Most of the software is rather stable and useful, but there is also a lot of fluff. User reviews and ratings help a lot, can we have those for Debian packages? Jailbreak and third party software (Cydia): thanks to some anonymous hackers, we have the ability to break the software jail on the iPhone and install software that Apple did not approve. The process is made very simple, but again it does require you to boot into Windows or MacOS X. The main thing the jailbreak process does is install an installer on the phone. Currently the best one is Cydia which is built on top of … apt-get and dpkg! After that is done, you can install a terminal, a ssh server and BossPrefs advanced preferences. There are also other interesting programs there - a different dock, file manager, caching youtube player, quake, themed app launcher with categories and startup page (winterboard). Theoretically the ssh server allows you to manage your music over WiFi from any platform, but unfortunately until the iTuned database hash algorithm is decoded, we can only copy music from the phone to the PC and not back. It is possible to copy files over using ssh and then play them using third-party players, but those players are very crude and featureless. To summarize: an iPhone is a great phone and a decent application platform. To make using an iPhone with Linux easier we need to make iTunes run in Wine (and allow it to see the iPhone) and reverse engineer the iTunes database hashing algorithm so that we can get music on our phones with Linux tools. I also wish there was a way to develop iPhone apps in Linux, but that really is reaching for the stars.

1 December 2008

Russell Coker: Gmail and Anti-Spam

I have just received an email with a question about SE Linux that was re-sent due to the first attempt being blocked by my anti-spam measures. I use the rfc-ignorant.org DNSBL services to stop some of the spam that is sent to me. The purpose of rfc-ignorant.org is to list systems that are run by people who don’t know how to set up mail servers correctly. But the majority of mail that is blocked when using them comes from large servers owned by companies large enough that they almost certainly employ people who know the RFCs (and who could for a trivial fraction of their budget hire such people). So it seems more about deliberately violating the standards than ignorance. The person who sent me the email in question said “hopefully, Google knows how to make their MTA compliant with RFC 2142“, such hope is misplaced as a search for gmail.com in the rfc-ignorant.org database shows that it is listed for not having a valid postmaster address [1]. A quick test revealed that two of the Gmail SMTP servers support the postmaster account (or at least it doesn’t give an error response to the RCPT TO command that is referenced in the complaint). However Gmail administrators have not responded to the auto-removal requests, which suggests that postmaster@gmail.com is a /dev/null address. However that is not a reason to avoid using Gmail. Some time ago Gmail took over the role of “mail server of last resort” from Hotmail. If you have trouble sending email to someone then using a free Gmail account seems to be the standard second option. Because so many people use Gmail and such a quantity of important mail is sent through that service (in my case mail from clients and prospective clients) it is not feasible to block Gmail. I have whitelisted Gmail for the rfc-ignorant.org tests and if Gmail starts failing other tests then I will consider additional white-lists for them. Gmail essentially has a monopoly of a segment of the market (that of free webmail systems). They don’t have 100%, but they have enough market share that it’s possible to ignore their competitors (in my experience). When configuring mail servers for clients I make sure that whatever anti-spam measures they request don’t block Gmail. As a rule of thumb, when running a corporate mail server you have to set up anti-spam measures to not block the main ISPs in the country (this means not blocking Optus or Telstra BigPond for Australian companies) and not block Gmail. Not blocking Yahoo (for “Yahoo Groups”) is also a good thing, but I have had a client specifically request that I block Yahoo Groups in the past - so obviously there is a range of opinions about the value of Yahoo. Someone contacted Biella regarding an email that they couldn’t send to me [2]. I have sent an email to Biella’s Gmail account from my Gmail account - that should avoid all possibility of blocking. If the person who contacted Biella also has a Gmail account then they can use that to send me email to my Gmail account (in the event that my own mail server rejects it - I have not whitelisted Gmail for all my anti-spam measures and it is quite possible for SpamAssassin to block mail from Gmail). It turns out that the person in question used an account on Verizon’s server, according to rfc-ignorant.org Verizon have an unusually broken mail server [3]. If your ISP is Optus, BigPond, Verizon, or something similarly broken and you want to send mail to people in other countries (where your ISP is just another annoyance on the net and not a significant entity that gets special treatment) then I suggest that you consider using Gmail. If nothing else then your Gmail account will still work even after your sub-standard ISP “teaches you a lesson” [4].

Matthew Garrett: Thintel

ICH8 docs, section 5.13.5.1:

To eliminate the audible noise caused by aggressive voltage ramps when exiting C4 states at a regular, periodic frequency, the ICH8 supports a method to slow down the
voltage ramp at the processor VR for certain break events.


Sounds good. Section 9.8.1.6:

C4_TIMING_CNT: Bit 6 Slow-C4 Exit Enable When 1, this bit enables the Slow-C4 Exit functionality.

Well, hey, we can quirk this on and make life better for everyone oh wait section 9.8.1.4:

Bit 0 C-STATE_CONFIG_LOCK: When set to 1, this bit locks down the C-State configuration parameters. The following configuration bits become read-only when this bit is set:
...
The entire C4 Timing Control Register (C4_TIMING_CNT)


Thanks, Intel. Thintel.

Michael Prokop: Schluchtenscheisser!

grml release delayed through a CBS commedy series. News at 11<Rhonda /> There we go: there’s a new grml release (you know, the Debian based Linux Live-CD for sysadmins and texttool users). The release with codename Schluchtenscheisser and version number 2008.11 is available in different variants. The 32bit versions grml, grml-medium and grml-small as well as grml64, grml64-medium and (new!) grml64-small are available through grml.org/download/. For details regarding the changes in the new release check out the release announcements. Happy grml-ing, dear Schluchtenscheissers - and to the rest of the world. 8-)

30 November 2008

Asheesh Laroia: Scrape the Web: Strategies for programming websites that don't expect it

I just received this email:
From: Greg Lindstrom
To: Tutorial List

Hello,

On behalf of the PyCon Tutorial Selection Committee, I'd like to inform you that you have been selected to present at least 1 tutorial at PyCon 2009 in Chicago. We had 50 proposals for the 32 available slots and many good proposals had to be rejected.
It means: I get to stand in front of people I don't know in Chicago and talk about web scraping! (Unless the talk is canceled because no one signs up.) So I guess it means I'll be going to PyCon 2009! It's in Chicago, from March 25 to April 2. If you'll be there, too, drop me a line!

Jon Dowland: x40 suspend

Do you own a thinkpad X40, or a similar device - specifically one which needed the S3 hacks to get the backlight back on after resume? If you aren't sure, lshal grep quirk should tell you (if you have hal). You're looking for something like power_management.quirk.s3_bios = true (and s3_mode). Kernel 2.6.26 added a lot of quirk handling into the drivers, meaning an end to user-space hacks. pm-suspend, from the pm-utils package, assumes that all of these quirks have been resolved, but it seems that the S3 ones for my hardware have not. This means broken suspend/resume out-of-the-box. This is fixed in a version of pm-utils to be released in a week or thereabouts, after the freeze begins. The X40 at one point seemed to be the Debian hacker's laptop of choice. I bought mine after glowing recommendations from Steve McIntyre, Stephen Gran, Mark Hymers, Amaya Rodrigo, Daniel Stone, Matthew Garrett, Rob McQueen and several others. It's one of the best pieces of hardware I've ever had the pleasure of using, and I'm concerned that we might even be going backwards in terms of supporting this device from release-to-release. If you do have such a device, can you please try the following:
  • install the 2.6.26 kernel
  • boot into single user mode
  • modprobe i915
  • echo mem > /sys/power/state
  • resume your laptop
Please try this and report to me whether the backlight came back on - either in the discussion page for this post, or via email (jon.backlight@alcopop.org). The kernel bug to try and get this fixed is http://bugzilla.kernel.org/show_bug.cgi?id=10985, but it might not be possible to backport a fix for this to the Lenny kernel. The bug against pm-utils regarding assuming 2.6.26 is quirk-free is https://bugs.freedesktop.org/show_bug.cgi?id=16453 (Debian bug http://bugs.debian.org/488144).

Biella Coleman: Hacking Spaces, the Spaces of Hacking

Space and place has long been important to hackers. Whether it was/is the university lab, the workplace, the hacker con, or the particularly high-tech city, hackers congregate and meet face to face, often and everyday. One semi-new development has been the explosion and proliferation of hacker labs and spaces, such as Noisebridge located in the Mission district of San Francisco Foulab in Montreal. I recently got back from San Francisco and was able to spend a few nights at Noisebridge and was jaw droppingly impressed. The space is, well, spacious and nice (and located right by the Bart, a real +++++), but more important is that it is a thriving collective with all sorts of geeky participants and they have just souped up the space with all sorts of equipment, from the usual suspects (lots of computers) to lots of electronic gear such as oscilloscopes. What I also was impressed by was not only the blizzard of events but the open and accessible nature of the organization, which seemed to sit in some contrast to NYC’s hacker collective, NYC Resistor. Like so many organizations in this metropolis, they apparently are lacking in space to grow and the word on the streets (which I cannot confirm or deny as I have had very little contact with them but have heard this repeatedly) is that the organization has had a tough time letting in new members. Some folks are understanding of this given their space limitation, others have been less kind, and have referred to the group as a clique. There are already a few other initiatives under way to find a larger space so as to accommodate a more open, participatory atmosphere for a hacker space (sign up for the email list here but I imagine that over time the culture and developments of NYC Resistor will also change as new spaces develops and do hope that this creates the conditions for more access rather than less (and again, I know next to nothing of the situation though I suspect space plays a real limiting role as it does with nearly everything else in NYC). And thankfully Rose White, a NYC-based sociology graduate student at CUNY, is paying close attention to the rise and development of these hacker spaces. She is well underway doing her dissertation dissertation on these hack spaces and I really look forward to her work. update: The Axis of Awesome is a hacker space in L.A. and as far as I am concerned, dons the best name.

Lucas Nussbaum: State of hardware support in Linux

I’m getting increasingly annoyed by the state of some aspects of hardware support in Linux. My laptop (old Dell Latitude D610, Intel-based with ATI graphics) used to suspend/resume correctly with Linux 2.6.24 (ie, > 95% success rate). Later changes made the success rate drop significantly, and added a problem with kacpid taking 100% CPU because of an interrupt storm. And now, with 2.6.28-rc6, it’s completely broken. (partially documented in bug 11563, but I admit I gave up on this bug, because I’m going to change my laptop soon). My desktop used to wake-on-lan correctly with 2.6.24 (but required some hacks, because it wouldn’t wake up if the NIC was DOWNed before shutdown), but changes in the r8169 driver broke it. (documented in bug 9512). As a result, I’m forced to run old kernel versions on the two systems I have at home. I can understand that those issues aren’t considered high priority (not everybody use WoL), but the fact that in both cases, they are regressions, worries me a bit. How many things are routinely broken during each kernel release cycle? Hardware support is difficult, of course, but are we really doing everything we could to make it suck less? Some things I really would like to see:

28 November 2008

Andrew Pollock: [tech] Wising up about SMART

I've always installed the smartmontools package on my servers because I figured it'd maybe give me some early warning about an impending disk failure. Getting proper SMART access was one of the motivating factors in moving my home-brew SAN from IDE-to-USB disks to native SATA. Since then, I've had two drives fail on me. One of the original four that I bought, and the replacement one that Seagate shipped to me. (Incidentally, I love how your only recourse to a replacement drive failing with Seagate is they'll ship you another (reconditioned) replacement drive. I can see the shipping expenses racking up very quickly). I decided to do the expedited return when the second drive failed, just because it meant I didn't have to be without a disk in my RAID-10 for a week while I waited for them to receive my drive and turn around and ship me a replacement drive, and it also meant I didn't have to mess around with packaging it myself. (Incidentally, I wonder what happens in Seagate's reconditioning process? Will someone else get my failed drive (i.e. the same serial number) after they replace the platters or whatever it was that failed? Would it be interesting to set up a website where people could register the serial numbers of drives they returned to Seagate, and how they died, and at how many power-on hours? How many times does a drive get "reborn"?) Whilst I was waiting for the replacement drive to arrive, I messed around with the failed one (I removed it from the RAID array), fiddling with the SMART self-tests that smartctl can initiate for you. The faulty drive failed the long self-test at the same LBA every time I ran it. It was around this time that I realised that the default configuration that comes with the Debian smartmontools package isn't all that useful with SATA disks. It ships with
DEVICESCAN -m root -M exec /usr/share/smartmontools/smartd-runner
which, as it turns out, doesn't detect SATA drives properly. I got
Nov 24 22:00:13 minotaur smartd[2990]: Device /dev/sda: ATA disk detected behind SAT layer 
Nov 24 22:00:13 minotaur smartd[2990]:   Try adding '-d sat' to the device line in the smartd.conf file. 
Nov 24 22:00:13 minotaur smartd[2990]:   For example: '/dev/sda -a -d sat' 
logged. It seems to me that if the software can detect this situation, it should be able to just try the "-d sat" behaviour automatically. So it seems that this whole time, I haven't been getting any SMART monitoring of my SATA drives. hddtemp has been happily logging the temperatures for me though. So I took the advice in the comments of /etc/smartd.conf and removed the DEVICESCAN entry in favour of explicitly naming the disks I wanted to monitor. I also discovered that I can have smartd kick off scheduled short and long self-tests on whatever schedule floats my boat. So now I've got something like this:
/dev/sda -d sat \
         -M exec /usr/share/smartmontools/smartd-runner \
         -s (L/../.././01 S/../.././(00 12))
This runs a long self-test at 1am every day, and a short one at midnight and midday. Hopefully this will help predict the next drive a bit earlier. I'm now debating whether to set up an active Nagios monitor for SMART, or to see if I can write a passive one that runs as part of the smartd-runner infrastructure that the smartmontools package provides (and is pretty neat). I guess it's already setup to email me, so it doesn't really matter whether it's Nagios emailing me, or smartd itself.

Jonathan McDowell: Perl regex help

Dear Lazyweb, please put me out of my misery.

I have a string that looks something like:

a b c foo 0 1 2 fred 3 4 5 5 4 3 2 1 0 quux
I want to output:
a
b
c
foo
 0 1 2 
fred
 3 4 5 
 5 4 3   2 1 0 
quux

(actually I want each line to be an array element, but that's obviously easy)

Can I do this with a Perl regex? Originally I thought I was going to have to walk the string like I would in C, but some playing around (and local help) got me to:

my $match = qr/(?:\s*[^\ \ \s]+\s* \ (?R)\ )/;
while ($cur =~ m/($match)\s+(.*)/o)  
        print $1, "\n";
        $cur = $2;
 

which sort of gets me there, except I get
0 1 2 fred 3 4 5 5 4 3 2 1 0
as one line instead of 4.
What obvious thing am I missing here?

MJ Ray: Bristol Cooperatives and Social Enterprise Trade Fair on Monday

Cooperatives-SW is working in partnership with the Regional Infrastructure for Social Enterprise (RISE) to hold a Trade Fair with support from CDA (BRAVE Ltd.), Social Enterprise Works, Co-opportunity and Watershed Media Centre. It will be held on Monday 1 December at the Watershed in Bristol. TTLLP will be there and I should post lots of news about it on Tuesday. I’ll try to microblog the event, but that depends how busy it is. Thanks to the VAT rate cut on the same day, I’ll be the only one there, leaving workers in the office just in case any of our customers have problems with their OSCommerce or Drupal online shops. (It seems there shouldn’t be any, but we hope for the best and prepare for the worst.) If you’d like to meet up on Monday, send me a comment however you prefer. Update: If you want to attend the event, registration details are on this page - it looks like registration is required, but I’ll be disappointed if they turn people away from the exhibition space between 1 and 2pm if there’s room.

27 November 2008

Russell Coker: Links November 2008

Netatia has an interesting series of articles about running a computer for two people [1]. It is a bit of a kludge, they have a single X server that covers both displays and then use Xephyr to divide it into two virtual screens. The positive aspecct of this is that it shuld allow a single wide monitor to be used by two sessions as displays are getting wider regardless of the wishes of manufacturers and consumers [2] this should be useful. It’s a pity that no-one has solved the problem of having multiple video cards, sound cards, and input devices to allow a single desktop system to be used for 6 or more people. It seems that the problems that need to be solved are only the support for multiple video cards, mouse-wheel support, and sound support. Paul Ewald gave an interesting TED talk about changing the conditions for diseases so that they evolve to be benign [3]. The first example is Cholera which if spread by water will benefit from being as toxic as possible (to cause the greatest amount of diorrhea - killing the host not being a problem), but if spread by human contact benefits from leaving it’s host well enough to walk around and meet people. This and the other examples he cites seem like strong reasons for universal health-care provided by the government. If clean water is provided to all the poor people then cholera will evolve to be less harmful, and if a rich person (such as myself) is unlucky enough to catch it then the results won’t be so bad. He also notes that less harmful bacteria will often result in the victim not seeking anti-biotics and therefore less pressure for the disease to evolve resistance to anti-biotics. Therefore the people who really need them (the elderly, the very young, and people who are already sick) will find them to be more effective. Paul Stamets gave a great TED talk about fungus [4]. One of his discoveries was that fungi can be used for breaking down petro-chemicals (they can eat oil). It would be interesting to see this tested on a large scale with one of the oil spils or with the polluted land around an ooil refinery. Also he has patented a method for using fungus to kill wood-eating ants (such as the ones that briefly infested his home). Robert Full gave an interesting TED talk on robot feet [5]. I found the bit about leg spikes particularly interesting (I had always wondered why insects have spikey legs). Alan Kay gave a very interesting presentation on using computers to teach young children about science [6]. An OLPC is referenced. It makes me want to buy an OLPC for everyone I know who has young children. The start of the talk is a little slow. Dan Barber gave a very interesting TED talk about organic and humane production of foie gras in Extramuda [7]. Apparently it tastes a lot better too. Incidentally I don’t list all the TED talks I watch, only the better ones. Less than half the TED talks that I see announced seem interesting enough to download, and of those less than half are good enough that I will recommend them. The ones that I don’t recommend don’t suck in any way, it’s just that I can’t write a paragraph about every talk. Of recent times my video watching has been divided about equally between “The Bill” and TED talks. Here’s an interesting article about Sarah Palin and “anti-elitism”: The prospects of a Palin administration are far more frightening, in fact, than those of a Palin Institute for Pediatric Neurosurgery. Ask yourself: how has “elitism” become a bad word in American politics? There is simply no other walk of life in which extraordinary talent and rigorous training are denigrated. We want elite pilots to fly our planes, elite troops to undertake our most critical missions, elite athletes to represent us in competition and elite scientists to devote the most productive years of their lives to curing our diseases. And yet, when it comes time to vest people with even greater responsibilities, we consider it a virtue to shun any and all standards of excellence. When it comes to choosing the people whose thoughts and actions will decide the fates of millions, then we suddenly want someone just like us, someone fit to have a beer with, someone down-to-earth in fact, almost anyone, provided that he or she doesn’t seem too intelligent or well educated.[8] Sarah will be representing the Republican party in 2012, the desire for leaders of average intelligence (or less) will still be around then. It will be interesting to see how many votes she gets and amusing to see her interviewed. The proceedings of the “Old Bailey” - London’s Central Criminal Court have been published [9]. It’s interesting to read some of the historical information about the legal system at the time. It made me appreciate how civilised the UK (and other countries that I have visited) are now. Bruce Schneier writes about the feture of ephemeral communication [10]. He concludes with the point “until we have a Presidential election where both candidates have a complete history on social networking sites from before they were teenagers we aren’t fully an information age society“. Of course as he notes the rules are written by the older people, currently I don’t think that any candidate for high office (cabinet minister or above) anywhere in the world can have a good history on the Internet. During the course of a decade or more on the net it’s impossible not to write something that can be used against you and no reasonable person could avoid changing their views on some issues in such a time period. That’s enough to lose an election with the way things currently work.

Adrian von Bidder: Unintended Consequences

Looking at access logs: Because of a recent entry in this blog, I have the doubtful honour to come up third (right now) in a somewhat unexpected query on Google...

Christian Perrier: How to get work done without doing anything

It's interesting to be here in Casar de Caceres for the joined i18n+QA+FTPmasters Extremadura meeting. Today, I've been sitting all day long in the biblioteca, which is used as hacklab, and just watched all great people around playing around with the concept of "tdebs". That weird beast can be described in two words as a concept of deb packages that would be able to distribute translation data in a more independent way then "regular" deb packages. Don't smash me as the concept is currently being twisted around in all directions by the nice folks here, to bring it to a great concept well suited for "side" material of our packages. To get an idea, you can read the original concept that popped up during the 2006 i18n Extremadura meeting, but got transformed drastically by ideas exchanged at Debconf 8 All this is made possible because the concept is now a very important issue for Emdebian and is strongly pushed by Neil Williams, whose nickname shall now be "MisterT", by general agreement of the folks here. What's really interesting right now is to see live discussions happening all the way round and see the concept slowly emerging from this. Something we could probably never do in regular mailing lists discussions or on IRC. We should really grant tons of rewards to the Extremadura region, and particularly out tireless C sar who again organised that meeting. It's still hard to say what will emerge from this and we willprobably need some more such live meetings to have real implementations of these tdeb beasts, but this is really how I like to see free software development going on. And now people begin to think about coding stuff and prepare patches, which is a good sign, in general.

Russell Coker: Slicehost vs Linode

Six months ago I investigated the options for Xen virtual servers [1]. I ended up receiving an offer of free hosting and not needing that, but the research was useful. There is a good range of options for Xen servers with different amounts of CPU power, RAM, bandwidth, and disk space. There are a couple of things that seem to be missing, options to upgrade from virtual servers to physical servers, and information on dedicated disk and database performance - but I’ll explain that later after some history. About a week ago a client needed a Xen virtual server in a hurry, their main server (a Xen system that I run on hardware that they rent) was getting a bit overloaded and needed to have one of the largest DomUs moved off. I ended up recommending Linode [2] based on my research and comments I received. The Linode server is working quite well and the client is happy, one nice feature of Linode is the choice of server rooms that they offer. I was able to choose a room in the same region as the other servers that the client owns and thus get ping times that are sometimes less than 2ms! Due to a missing feature in a program that I’m maintaining for the client a large number of MySQL queries are being made. Due to a problem I’m having with MySQL it won’t let me create a slave database server so all the queries go over the VPN and use a large amount of data. This combined with the other traffic that should be going over that link means that about 600G per month is being used, fortunately that is rather cheap. Linode staff handled this very well, after the server had exceeded it’s quota by 120G they asked my client to confirm that the traffic was legitimate and then suggested an upgrade to a plan that could handle the traffic (which went smoothly). Now I have another week to add the feature in question before I meet the quota again. Shortly after getting the new virtual server running at full capacity David Welton wrote a detailed review of Linode and Slicehost for the issues that matter to his use [3]. His conclusion seems strongly in favor of Linode. But now I am looking at getting a Slicehost [4] virtual server for the same client (for a different project) because Slicehost is owned by Rackspace [5], and the new project if successful will need a set of powerful servers and Rackspace seems like a reasonable company to host that. The problem with Rackspace is that they (and every other ISP I’ve researched so far) seems to offer little in regard to customers who need serious disk IO. I am planning some servers that will have a write bottleneck on a MySQL database (or maybe multiple shards), so serious disk capacity is needed. At least I would like to be able to get disk storage by the tray (12-14 disks) with the controllers having RAID-6 support. Rackspace only offers RAID-5 (according to the “livechat” person), and we didn’t get as far as discussing how to add more trays. What would be ideal is if there was an ISP that had both virtual servers and physical servers (so I could start with a virtual server and move to a physical server when things are working well), and also serious storage options. They would offer internal disks, external RAID arrays, and NetApp Filers [6] (or some equivalent device). It would be really nice if I could just instruct the ISP to add another NetApp Filer to my back-end network and have it done for me (I’m certainly not going to visit the US to install new hardware). It’s been over a day since I submitted a sales request to NetApp asking whether they partner with any ISPs and I haven’t received a response. OpenSolaris with ZFS also sounds good for disk IO performance (they have similar features to NetApp). Unfortunately the support for OpenSolaris among ISPs is not that great (while everyone offers Linux and Windows), and I haven’t used any recent version of Solaris. So using OpenSolaris would require finding someone with the skills to manage it who can work for my client - as opposed to a NetApp device that would be just like any other NFS server, SAN, or iSCSI server. But I’m not ruling OpenSolaris out, if someone knows of a good ISP that hosts OpenSolaris machines and supports adding dozens of disks and decent amounts of NVRAM for ZFS then I would be interested to investigate it. Joyent has some interesting OpenSolaris virtual server plans [7], they are a little pricey but offer large amounts of data transfer. They don’t provide any information on disk IO capacity (other than saying that they use ZFS for good performance). I’ve just downloaded Nexenta (Debian with the OpenSolaris kernel) [8] and will test it out over the next few days. One of the reasons I’m tending towards Rackspace at the moment (with Slicehost as the entry point) is that they seem cooperative to customer requests. My discussions with them (on a web based “livechat” and on the phone) have indicated that they may be able to do something special for me.

26 November 2008

Russell Coker: Bill Joy

Some time ago Bill Joy (who is famous among other things for being a co-founder of Sun) [1] wrote an article for Wired magazine titled “Why the future doesn’t need us” [2]. He wrote many sensible things but unfortunately focussed on the negative issues and didn’t receive a good response. On reading it today I thought more highly of the article than I did in 2000 when it was printed, largely due to having done some background research on the topic. I’ve recently been reading Accelerating Future [3] which has a more positive approach. Now a talk by Bill Joy from 2006 has been published on the TED.com web site [4]. He starts by talking about “super empowered individuals” re-creating the 1918 flu. He also claims that “more technology super-empowers people more“. Such claims seem to be excessively hyped, I would be interested to see comments from someone who has a good knowledge of current bio-technology as to the merits of those claims. He talks briefly about politics and has some good points such as “the bargain that gives us civilisation is the bargain to not use power” and “we can’t give up the rule of law to fight an asymmetric threat - which is what we seem to be doing“. He mentions Moore’s law and suggests that a computer costing $1000 then (2006) might cost $10 in 2020. He seems to be forgetting the cost of the keyboard and other mechanical parts. I can imagine a high-end CPU which cost about $800 a couple of years ago being replaced by a $2 CPU in 2020, but I don’t expect a decent keyboard to be that cheap any time before we get fully automated nano-factories (which is an entirely separate issue). Even the PSU (which has a significant amount of government regulations for safety reasons) will have a floor cost that is a good portion of $10. Incidentally the keyboard on my EeePC 701 sucks badly, I guess I’m spoiled by the series of Thinkpad keyboards that I keep getting under warranty (which would cost me a moderate amount of money if I had to pay every time I wore one out). I will make a specific prediction, that by 2015 one of the better keyboards will comprise a significant portion of the entire cost of a computer system (more than a low end computer unit) - such that in some reasonable configurations the keyboard will be the most expensive part. It would be good if PCs could be designed to use external PSUs (such as the Compaq Evo models that took laptop PSUs). Then the PSU, keyboard, and monitor would be optional extras thus giving a small base price. Given that well built PSUs and keyboards tend not to wear out as fast as people want to replace computers, it seems that financial savings could be provided to most customers by allowing them to purchase the computer unit without the extra parts. People like me who type enough to regularly wear out keyboards and who keep using computers for more than 5 years because they still work are in a small minority and would of course be able to buy the same bundle of computer, PSU, and keyboard that new users would get. In 2020 a device the size of an iPaQ H39xx with USB (for keyboard and mouse) and the new DisplayPort [5] digital display interface (that is used in recent Lenovo laptops [6]) would make a great PDA/desktop. You could dock a PDA the way some people dock laptops now and carry all your data around with you. Bill cites an example of running the Mac interface on an Apple ][ (does anyone know of a reference for this) as an example of older hardware being more effective with newer software. It's a pity that often new software needs such powerful hardware. EG it's only recently that hardware developments have overtaken the developments of OpenOffice to make it deliver decent performance. He has an interesting idea of using insurance companies to replace government regulation of food and drugs. The general concept is that if you can convince an insurance company that your new drug is not a great risk to them so that they charge a premium you can afford then you could sell it without any further testing. Normally I'm in favor of government regulation of such things, but given the abject failures of the US government this idea has some merit. Of course there's nothing stopping insurance companies from just taking a chance and running up debts that they could never hope to repay (in a similar manner to so many banks). Finally I think it's interesting to note the camera work that the TED people used. My experience of being in the audience for many lectures (including one of Bill Joy's lectures in the late 90's) is that a speaker who consults their notes as often as Bill does gives a negative impression to the audience. Note that I'm not criticising Bill in this regard, often a great talk requires some significant notes - very few people can deliver a talk in a convincing manner entirely from memory. It seems to me that the choices of camera angle are designed to give a better impression than someone who was seated in the audience might receive - there's no reason why a video of a talk should be spoiled by seeing the top of the speaker's head while they consult their notes!

Next.

Previous.