The problem On a couple of Debian Etch systems we have a plone-site that is published using a backport of the nginx web server. The Zope instance is running on the standard port and serves the Plone contents under the /plone path. Initially we were publishing the site to the external world using an https site served by nginx using the following entry on the configuration:

  location /plone/  
    proxy_pass http://plone:9673;
    include    /etc/nginx/proxy.conf;
   

The proxy.conf contents are quite standard:

  # proxy.conf
  proxy_redirect                  off;
  proxy_set_header                Host $host;
  proxy_set_header                X-Real-IP $remote_addr;
  proxy_set_header                X-Forwarded-For $proxy_add_x_forwarded_for;
  client_max_body_size            0;
  client_body_buffer_size         128k;
  proxy_connect_timeout           90;
  proxy_send_timeout              90;
  proxy_read_timeout              90;
  proxy_buffer_size               4k;
  proxy_buffers                   4 32k;
  proxy_busy_buffers_size         64k;
  proxy_temp_file_write_size      64k;

With this settings we see the /plone contents using the same path that is used by the Zope instance, but after testing we have decided to change the /plone path and server the contents under the /web path. The Wrong Solution The fist option I though about was quite simple, rename the Zope's plone object to web. Seems reasonable and simple for someone without Zope experience (I don't administer the internals of the Zope/Plone site), but now I know that it is a very big mistake, because renaming objects in Zope in not cheap, as it implies that the server has to modify all the contents of the renamed object and the operation can take a very long time. With my ignorance I tried to rename the plone object using the Zope administrative interface and after a minute or so I cancelled the page loading that was running on my browser, thinking that I had cancelled the rename operation. To make a long story short I'll tell you that the operation was still running and after several hours the folder was renamed (in fact I noticed when the good solution broke, as I had already solved the problem using the next method), but something went wrong and part of the site functionality was broken... the final solution to the debacle has been to recover a backup of the Zope instance older than the rename operation and continue from that copy. The Right Solution (TM) It seems that Zope has a couple of systems to do Virtual Hosting and the best option is the use of the product called Virtual Host Monster, a weird and confusing system (IMHO, of course), that does the job once the right configuration settings are in place. The best solution to our problem was to modify the requests done by the reverse proxy without touching anything on the Plone site (the original one already had a Virtual Host Monster object installed and that was the only thing that we needed to add). The nginx configuration for the new /web path is the following:

  location /web/  
    proxy_pass http://plone:9673/plone/VirtualHostRoot/_vh_web/;
    include    /etc/nginx/proxy.conf;
   

With this change, when the user asks for anything under the /web/ path the Zope server gets the contents traversing the /plone object and adding to it the elements that appear after the VirtualHostRoot component, ignoring components that start with the _vh_ prefix (the protocol and host name of the requests are not modified, as we did not touched that). Once the object is found, the server rewrites the URLs included on the HTML files using the path components that appear after the VirtualHostRoot one, including the suffix of the components that start with the prefix _vh_. For example, when the Zope server receives a request for an URL like:

  http://plone:9673/plone/VirtualHostRoot/_vh_web/home

it publishes the content found on:

  http://plone:9673/plone/home

but the HTML files returned assume that their base URL is:

  http://plone:9673/web/home

For the past couple of days I've been working on some "easy hosting" setup for Debian. This is a continuation of my shell-script based solution, but intended to be much more dynamic. The system makes it simple for us to deploy a Debian Etch installation, and allow users to create virtualhosts easily. (And by easily I mean by creating directories, and very little else.) So, for example, to create a new website simple point the IP address of your domain example.org to the IP of your machine. Then run:

mkdir -p /srv/example.com/cgi-bin
mkdir -p /srv/example.com/htdocs

If you then want to allow FTP access to upload you may run:

echo "mysecretpass" > /srv/example.com/ftp-password

This will give you FTP access, username "example.com", password "mysecretpass". You'll be chrooted into the /srv/example.com/ directory. All of this is trivial. Via Apache's mod_vhost_alias, and a simple script to split logfiles and generate statistics via webalizer for each domain. The only thing that I really needed to do was to come up with a simple shell script & cron entry to build up an FTP password file for pure-ftpd. So here's where it gets interesting. The next job, obviously, is to handle mail for the domains. Under Debian it should be a matter of creating an appropriate /etc/exim/exim4.conf - and ignoring the rest of the setup. I'm getting some help with that, because despite knowing almost too much about SMTP these days I'm still a little hazy on Exim4 configuration. I'm watching the recent debian configuration packages system with interest, because right now I'm not touching any configuration files I'm sure that it is only a matter of time.
In other news I cut prices, and am seeing a lot of interest in my mail-scanning. Finally my .emacs file has been tweaked a lot over the previous few days. Far too much fun. (support files.)

That's it...

Student for now :-)

Damn it, I should had wrote about it a couple of days ago. Anyway, it happened last Friday (20080222), my last day at work. After 4.5 years (exactly 4 years, 6 months and a week), I quit my job to dedicate more time for my "supposed" last year studying Computer Science at the University.

In a far far away galaxy...

During the translation of packages' descriptions, when I found something that looks like a typo or a wrong reference I try to report wishlist bugs and, when possible, provide patches, I know this is somewhat noisy, it is just to register the fact, so Smith Review can also spotted it, sorry for the noise. :-)

BTW, I would still vote for a genius if we are going to elect a mascot.

Thanks: frolic

Yesterday really didn't go to plan. The plan was to clear a few work tasks and then travel to London for an early afternoon Koha-related meeting, then fix some other problems on site before travelling home this evening. Unusually, this trip involved bike, train, tube coach and bus. Everything but the car, indeed, so a pretty good example of using greener transport for TTLLP. Got about 80% of the work tasks done (not bad), biked to Worle station and caught the train, then changed to a high-speed train at Bristol. So far, so good. I spotted our Member of Parliament changing trains in Bristol too. I did a double-take, which I think he noticed - he's in the local papers every week or two, but he's not that visually distinctive and looks superficially similar to my last MP. Didn't harangue him this time - he was queuing for coffee, while I wanted to visit the buffet. Anyway, about 10 minutes before arrival in London, someone walked onto the tracks near Southall. We reversed into Slough and were eventually told to travel via Windsor into Waterloo (I think if they'd been a bit quicker, I could have saved 30 minutes). I planned to arrive in London 75 minutes before my meeting, but this happened at about the most delaying point, so I was actually 70 minutes late and missed the whole thing. Damn. Of the three on-site problems, I fixed one and made progress on one, but that fixed one means that we can continue working remotely on the others, so not a total failure. The coach back to Bristol was fine - once it stops weaving around London, it's as comfortable as an unrefurbished HST (not quite as much leg or arm room as a current one). The bus to Worle was awful, bouncy as hell and badly-driven, stopping ahead of the lines at red lights. It's also expensive (thanks to First's route monopoly?) - in future, I'd walk 15 minutes and pay 30p more for the train. The above was written on the return journey, then bluetoothed here. Now it's time to catch up with the stack of emails and back to work.

Ever since I started University, I've been a member of the Institution of Electrical and Electronic Engineers. Although I don't really do what you would call "Engineering" any more, I like to read the IEEE Spectrum magazine and some of the Computer society journals are quite relevant to what I do. However, this year the IEEE have produced what is, without a doubt, one of the most shithouse online ordering systems I have ever had the misfortune of trying to use. To start with, if you happen to go to the account login screen with cookies disabled, you get a page that is utterly blank. I've seen a lot of ways to fail to handle cookies, but I think the completely blank page is a first for me. Once you work out what's going on and log in, you're presented with a long and painful journey through a series of "checkout" pages, mostly alike. I think I got presented with my list of journals about four times. After navigating that pest, it comes up with this little gem when you start the credit card processing step: Then, every time you input practically anything into the form, it does a Javascript submit and does things to the form. For example, when you enter your credit card number, you get a form back that has blanked out all but the last four digits of the card number. OK, a little dumb, but not spectacularly so. Although, when you're a quick typist and have filled in the next three fields while the web browser was playing footsie with the slow-as-molasses web server, only to have that data wiped out when the page refreshes, is a bit of a pest. Far more annoying is when you start entering your address "for verification purposes". Here, you get a form resubmit every time, presumably to update the form based on (for example) your selected country, but not a lot changes. Even though I picked Australia, I still got a pull-down with all of the the US states in it. You could short-circuit all this with the "choose existing address" option, except that (despite me being logged in) my existing addresses aren't in a select box, but in a separate pop-up window (which didn't actually display anything after all). In short, the IEEE's online renewal system is a disgrace, and is an embarrassment to what is purportedly a technical organisation.

Today's xkcd reminds me that about 15 pounds of grapefruit into this year, I tried to calculate how many I could eat a day, if I were to do were peel and eat grapefruit, all day. I may have been trying to figure out if it is possible to live on a pure grapefruit diet. I remember arriving at a figure around 70. My actual daily consumption at the height of grapefruit season is 3 to 5. It's become my favorite winter fruit and quite puts one off nasty sweet oranges. There's a trick to both the peeling and the eating of grapefruit which the xkcd guy has apparently never learned, that avoids the healhful skin and leaves only the delicious tart flavour droplets behind. I'm available for lessons, the first 2 or 3 weeks that the Texas ruby reds are in season.

---

(Of course as wrong as Randall is about grapefruit, he's 5 million times as wrong about a proper tomato..)

Another awards show took place today but nobody is paying any attention to that. All of Hollywoods elite can be found at the King Plaza Buffet in Jersey City for the 80th annual Things Jaldhar Discovered Last Year That Every Else Knew About Already Awards. (Known in the industry as "the Golden Sombreros") Here is a list of the winners. Game: Hungry Hungry Hippos This game is awesome! But don't play with three year olds. They cheat. Book: Discworld series by Terry Prachett When The Color Of Magic came out in 1983, I dismissed it as merely "The Hichhikers Guide to Mordor" but last year I read it again and this time it grew on me. Since then I've been steadily reading the whole series since June at an average of one a week. Music: Rammstein Smell the umlauts! I have no idea what they are actually going on about but this is what Metal is supposed to sound like. Mental Condition: Oedipus Complex All of sudden last October or so, my son started screaming and crying whenever I got anywhere near my wife and he would cling to her like the proverbial limpet. Then three months later, just as suddenly, he stopped. Very strange, not to mention creepy. Linux command: grep --color=auto Everyone knows GNU ls can be colorized but did you know the same is true of GNU grep? I didn't. Why wasn't I informed of this earlier? L. Ron Hubbard Lifetime Achievement Award: Coke Zero I used to drink way more Coke than was good for me so I decided to give it up. Diet sodas don't taste very good so I was resigned to not drinking soda at all. Luckily for me Coke Zero has no calories or sodium but does have caffeine and a taste like actual Coca-Cola. Perfect!

So I have been learning about Git this weekend. It has some really nice-looking features for sure -- some things Mercurial doesn't have.

I was getting interested in switching, until I found what I consider a big problem.

Many projects that use git require you to submit things using git-format-patch instead of pushing/pulling from you. They don't want your merge history.

git-format-patch, though, doesn't preserve SHA1s, nor does it preserve merges.

Now, say we started from a common base where line 10 of file X said "hi", I locally changed it to "foo", upstream changed it to "bar", and at merge time I decide that we were both wrong and change it to "baz". I don't want to lose the fact that I once had it at "foo", in case it turns out later that really was the right decision.

When we track upstream changes, and submit with git format-patch, the canonical way to merge upstream appears to be:

git fetch; get rebase origin/master

Now, problem with that is it loses your original pre-conflict code on a case like this.

There appears to be no clean way around that whatsoever. I tried a separate "submission" branch, that rebases a local development-with-merge branch, but it requires a ton of git rebase --skip during the rebase process.

Thoughts?

A few days ago I got myself an Asus EEEPC to experiment with it being in a role of a small server and a tiny internet kiosk. I installed Debian on it, but the process was not for the feint of heart, that’s for sure. First of all the d-i font was messed up and all the menus overflowed the screen making it very hard to select anything. Additionally it seems very strange to me that there was a special d-i image made for EEEPC, but that image did not include built-in support for the computers wired or wireless network interfaces. That made my day highly problematic as I do not have an easy way to get to the Internet via a wired connection and the provided d-i image did not have enough files on it to finish the base install without networking.
This again made me think that the approach Ubuntu took is more favorable in most situations - have the install image boot a mostly functional system (it does not have to be X even) and then install from there. It actually feels more flexible than using the highly restricted d-i environment.
I will be looking to make a Debian rescue image designed for the EEEPC that you could dd onto a USB key, boot from and have a minimal Debian system with working ethernet, wifi and some basic rescue tools and a way to install a basic Debian system as well. That should make it much easier for people to get Debian onto their EEE PCs. I do hope that the Debian EEEPC project will improve as well.

I held a talk this morning about the Belgian electronic ID card in Debian (and derivates, like Ubuntu). Things were a bit tight there for a moment, because I overslept; but eventually, we did make it in time. The talk didn't go perfect. Because I overslept, I didn't have enough time to test; because I didn't test enough, I found out that the smartcard didn't work on the demo machine. Darn; it's pretty silly to hold a talk about how easy it is to use the electronic ID card on Debian, only to find out that I, myself, am unable to use it. Grmbl. Anyway, the talk worked out okay. Despite the early hour, a decent amount of people showed up—including Fabian Arrotin, who held a talk about a similar topic in the CentOS room last year. Now for breakfast. Really.

For the last few years I have been actively seeking free hardware to give to members of my local LUG. Whenever a friend or business associate mentions that they are upgrading or replacing computers I enquire what they plan to do with the old ones and request that the old gear be given to me if there are no other plans for it. There is a moderate amount of hardware that I use for my own purposes, but the free hardware that is available is often in excess of my requirements and also sometimes just unsuitable for my use (I am happy to install a second-hand IBM or HP machine for a client but I won’t install a white-box machine). One organisation that I sometimes give computers to is Computerbank [1]. The purpose of Computerbank is to take donations of old machines, fix them and install Linux, and then sell them for extremely low prices to people who can’t afford new machines. It’s been a while since I gave them any computers because for a long time the minimum specs on machines that they were willing to accept were higher than the machines that I obtained. Generally I offer my old hardware to the mailing list of Linux Users of Victoria [2]. I offer not only working systems but also broken systems and other things that might be useless to most people - but are greatly desired by the small minority who can use them. One member of that list wants PC power supplies for repairing other electronic devices, so I collect batches of machines that are broken but appear to have working PSUs and give them to him. I once received a box of free two-button mice. I offered them to LUV members expecting that many people would want one or two for test machines. No-one wanted them for use as mice but one guy wanted all of them to use their sensors in robotics projects. One thing that impresses me is the community spirit demonstrated. Often I will offer some free machines and the first response will be something like “I’d like to take that machine apart for the bits, but if someone wants a complete system please give it to them instead”. There aren’t many occasions when you see someone suggesting that they may not be the most deserving recipient for something that is free! My aims in this effort are to help random Linux users in my area, and to help the environment by reducing the amount of land-fill. My efforts aren’t going to make a significant impact on the environmental situation, but they do make a significant impact on the availability of hardware for members of the Linux community - which seems to be of particular interest to people who want cheap machines for their children or grand-children. I encourage other people to do similar things. One thing that impressed me was the organisation of used hardware gifts at LCA. Near the start of the conference hardware was given away to anyone who put their hand up. At the end of the conference more hardware was given away (I expect it was mostly by delegates who lived locally). It would be good if this idea (which worked so well) was spread to other conferences.

• [1] http://vic.computerbank.org.au/


• [2] http://www.luv.asn.au/

Share This

Yesterday, collectd 4.3.0-1 has been uploaded to Debian unstable. The most important changes since 3.x are (in chronological order):

• the much improved plugin infrastructure offers a lot more flexibility
• the much improved networking code offers a lot more flexibility
• the new bindings to Perl allow plugins to be written in the scripting language Perl
• the new unixsock and exec plugins allow external programs to communicate with the daemon and submit data
• integration with Nagios has been implemented
• support for flexible intervals has been added
• support to read values from SNMP enabled devices has been added
• basic support for monitoring has been added
• many new plugins

See the changelog for details. About collectd:
collectd is a small daemon which collects system information periodically and provides mechanisms to monitor and store the values in a variety of ways. It is written in C for performance. Since the daemon doesn't need to startup every time it wants to update the values it's very fast and easy on the system and allows very fine-grained statistics.

Jeremy Visser writes about the evilness of User-Agent sniffing I tend to disagree with him, particularly after reading Wikipedia's robots.txt file, and some of the comments in there. wget can be quite a nasty program if used incorrectly. Chances are, if you know what you're doing, you'll know how to change the User-Agent string, and therefore you're probably not going to do something dumb like try to download the entire site recursively. If you don't, you're probably Doing It Wrong.

I learned about two bash features I hadn't known about today.

From a colleague, GLOBIGNORE. A colon-separated glob of files to ignore when expanding globs. Helpful behavior when set to "*~" and used with grep.

From the Git FAQ, in a section explaining that it breaks the Git build process, CDPATH. A colon-separated search path to use when you type cd. Possibly useful to refer to subdirs of ~ or other common areas. Seems like it's prone to break a ton of scripts if exported though.

Every since I found out that the first digits of any credit card denote the issuer identifier (i.e., folks can tell who issued a credit card and what type it is just from the first digit or two) I've been annoyed almost each time I have to input credit card information on the web. Any decent credit card system knows that if a sixteen digit credit card number starts with 4, it's a Visa. And yet, each time anyone buys anything on the web, they must select "Visa" from the drop-down box. On a certain level we all know this; People in stores and restaurants never have to select the type of card before swiping. When I'm feeling generous, I imagine this is so that the credit card companies can give an extra reminder that they only accept certain credit cards -- not being able to select a card type in an "essential" input field constrained to multiple choices is a pretty strong reminder. When I'm feeling less generous, I suspect it might be so that the companies can subtly remind us that they have their own brand credit cards that we might like to acquire.

Today, in a fit of fool-Hardy-ness, I upgraded one of my computers to the most recent Ubuntu development release, codename Hardy Heron. Amongst the several other bugs I encountered (the kernel not booting was a rather noticable one…) was this adorable little bug in tracker. (Some background info: Tracker is a file alteration monitor - like Apple’s Spotlight, and …whatever the Vista knockoff is called. ImageMagick is an image processing utility, invoked with the command “convert”.) So - picture this situation: Tracker is searching along, and encounters a graphics file. Being a graphics file, Tracker realizes it wants a thumbnail. Tracker calls convert. Convert creates a temporary file. Tracker notices a file has been created. It notices that this file is an image file, and therefore wants a thumbnail. Tracker calls convert… You can guess what happens from here on. This lovely bug reminds me quite a lot of a story I read a while ago which had been uploaded by Eric Smith to his wonderful PDP-6/PDP-10 resource and folklore website, “I/O Deja Vu, A Farce in One Act“. Worth a read. Update: Within ten minutes, the very helpful Saleem Abdulrasool (”compnerd” on IRC) helped me debug it and comitted a change to upstream, so it’s fixed in the next release (expected in a few days). Filed a bug on the Ubuntu package. I love FOSS.

Russ Nelson: The Influence of Government writes:

"Larry Lessig proposes to lessen the influence of money on legislators. Unfortunately for him, he is trying to hide a symptom without curing the disease. The disease is that governments regulate businesses. The symptom is that businesses then have a profit motive in regulating governments. If you want a government which is free of corruption, you have to eliminate the motive for corrupting them."

which looks like utter bovine excrement to me. Corporations - and the overpowering profit motive of some corporations - are artificial things, created purely by regulations. If you free corporations from all regulations, you destroy them. If you only free them from taxes, you remove some of the few restraints on their bad behaviour in pursuit of profit. See Joel Bakan: The Corporation for a better approach. In summary:-

• Relegislate. Base regulations on the precautionary principle for environmental, health and safety issues;
• Set realistic enforcement agency staffing levels and punishments;
• Involve local elected representatives in more of the regulatory system;
• Protect and enhance the role of trade unions and civil society in advising regulators.

Some of those may seem surprising, but if you read the book, you'll see why those conclusions make sense.

I'm (again) writing a library that is supposed to be cross-platform, but needs sort of an event loop, which I'd rather leave in the main program and just give them a bunch of file descriptors to watch and a timeout. Problem is, file descriptors are platform dependent. There are a few approaches to this. The mighty ASIO simply annexes the entire loop, and just tells people to hook into it if they need anything -- it works, but on the downside you get threads for free when you didn't want or need them. Simply spawning a new thread for everyone isn't a solution either, as it requires me to do lots of synchronised inter-thread communication -- and on Windows, file handles are thread affine, so I cannot even call into the same library from different threads. I will probably go with ASIO for this project, but I'm fairly certain that there are better solutions out there.

I'm going to admit up front here that I'm pushing my luck, and that I anticipate the chances of success are minimal. But that aside .. There are a lot of people who read my entries, because of syndication, and I'm optimistic that somebody here in the UK will have a copy of the following three books they could send me:

• Flash Gordon vol 3: Crisis on Citiadel II
• Flash Gordon vol 5: Citadels under attack
• Flash Gordon vol 6: Citadels on Earth

(All three are cheap paperback pulp fiction novels from the 1980s written by Alex Raymond.) If you have a copy of any of those three books, and are willing to part with them, then I'd love to hear from you. Either as a comment or via email. I'm certainly expecting to pay for them up to around £5 for each volume.

Backstory: I read the first when I was 10-12, then mostly forgot about it. A while back I remembered enjoying it and bought volumes 1, 2, 3, & 4 from an online store. I got screwed and volume 3 hasn't arrived, but possibly that will be rectified soon. Here in the UK the last two volumes are either extremely rare or extremely in demand. Typically they seem to sell for £15-30 - I'm frustrated to not have the conclusion, but not desperate to spend so much money upon them, (been there, done that).

So if anybody has some or all of these books and can bear to part with them please do let me know. </luck:pushing>

I've two video-related queries, which I'd be greatful if people could help me out with:

Mass Video Uploading

Is there any tool, or service, which will allow me to upload a random movie to multiple video-download sites? Specifically I'm curious to learn whether there is a facility to transcode as necessary a given input file and then upload to youtube, google video, and other sites as a one-step operation.

Mass Video Searching

Relating to that is there a service which will allow me to search for vidoes with given titles/tags/keywords across multiple video-hosting networks?

Regarding the searching I see that YouTube has support for "OpenSearch", but Google's video hosting has neither that nor a sitemap.xml file: Irony Is ...