mod_wsgi 1.0 Yesterday the official 1.0 version of mod_wsgi was released and uploaded to unstable (the binary package you want to install is called libapache2-mod-wsgi, in case you were wondering), so I think it's a good time to blog about it. Mod_wsgi is an Apache module which can host any Python application which supports the WSGI interface (the reference is also available as package). Major applications and frameworks which are known to work well with mod_wsgi include CherryPy, Django, Karrigell, Pylons, TurboGears, web.py, MoinMoin, PyBlosxom and Trac. In fact every application which provides a WSGI compatible interface should be able to be hosted with mod_wsgi. The module is very simple to use, is very well configurable, documented, has a very active and responsive upstream author and is suitable for use in hosting high performance production web sites, as well as your average personal site. There are two common ways of operation:

• Embedded mode - working in a way similar to mod-python: The Python code is executed within the context of the Apache child process.
• Daemon mode - similar to FASTCGI/SCGI solutions, but with the important difference that a separate infrastructure is not needed when implementing the WSGI application and everything is handled automatically by mod_wsgi. Of course this mode has a smaller impact on the normal Apache child processes and also allows to run applications under different users.

If you're running python web applications which provide a WSGI interface: go and give mod_wsgi a try! ~~DISCUSSION~~

Following the path created by Kenta Cho, many other game developers, most of them Japanese, are doing really interesting and cute games. This time is HIZ the one selected. Even though his games don’t have the high speed feeling from Kenta’s, they definitely are interesting to play and really lovely visually:

• Tatan - pointing STG shooter game
• ProjectL - sword action shooting
• ES - shooter game with an interesting control scheme

As usual, I’ve developed packages for those games for Debian SID, but in this case they’ll probably take some time to get into Debian because it’s not totally clear that the music is DFSG-free, even though they’re gratis/free (as in beer) and they’re redistributable. Lets see how we solve that, if there’s no other way, we might need to replace those songs, we’ll see. They are coded in D language too, BTW.

A significant problem with the old-fashioned media is that as a general rule they don’t cite references for anything. Some of the better TV documentaries and non-fiction books cite references, but this is the exception not the norm. Often documentaries only cite references in DVD extras which are good for the people who like the documentary enough to buy it but not for people who want to rebut it (few people will pay for a resource if they doubt the truth and accuracy of it’s claims). I can understand newspapers not wanting to publish much in the way of background information in the paper version as every extra line of text in an article is a line of advertising that they can’t sell. So they have financial pressure to produce less content, and the number of people like me who want to check the facts and figures used in articles is probably a small portion of the readership. Another issue with newspapers is that they are often considered as primary authoritative sources (by themselves and by the readers). It is often the case that journalists will interview people who have first-hand knowledge of an issue and the resulting article will be authoritative and a primary source in which case all they need to do is to note that they interviewed the subject. However the majority of articles published will be sourced from elsewhere (news agencies [ http://en.wikipedia.org/wiki/News_agency ] such as Reuters are commonly used). Also articles will often be written based on press releases - it is very interesting to read press releases and see how little work is done by some media outlets to convert them to articles, through a well written press release a corporation or interest group can almost write it’s own articles for publication in the old media. One way of partially addressing the problem of citing references in old media would be to create a web site of references, then every article could have a URL that is a permanent link to the references and calculations to support the claims and numbers used. Such a URL could be produced by any blogging software, and a blog would be an ideal way of doing this. For bloggers however it’s much easier to cite references and readers have much higher expectations of links to other sites to support claims and of mathematical calculations shown to indicate how numbers are determined. But there is still room for improvement. Here are some of the most common mistakes that I see in posts by people who are trying to do the right thing:

1. Indirect links. When you refer to a site you want to refer to it directly. In email (which is generally considered a transient medium) a service such as TinyURL [ www.TinyURL.com ] can be used to create short URLs to refer to pages that have long URLs. This is really good for email as there are occasions when people will want to write the address down and type it in to another computer. For blogging you should assume that your reader has access to browse the web (which is the case most of the time). Another possibility is to have the textual description of a link include a reference to the TinyURL service but to have the HREF refer to the real address. Any service on the net may potentially go away at some future time. Any service on the net may have transient outages, and any reader of your blog may have routing problems that make parts of the net unavailable to them. If accessing a reference requires using TinyURL (or a similar service) as well as the target site then there are two potential things that might break and prevent your readers from accessing it.
   One situation where indirect links are acceptable is for the printed version. So you could have a link in the HTML code for readers to click on to get to the reference page directly and a TinuURL link for people who have a printed version and need to type it in.
   Also when linking to a blog it’s worth considering the fact that a track-back won’t work via TinyURL and track-backs may help you get more readers…


2. Links that expire. For example never say “there’s a good article on the front page of X” (where X is a blog or news site). Instead say “here’s a link to a good article which happens to be on the front page now” so that someone who reads your post in a couple of years time can see the article that you reference.
   Another problem is links to transient data. For example if you want to comment on the features of a 2007 model car you should try to avoid linking to the car manufacturer page, next year they will release a new car and delete the old data from their site.
   A potential problem related to this is the Google cache pages which translate PDF to HTML and high-light relevant terms and can make it much easier to extract certain information from web pages. It can provide value to readers to use such links but AFAIK there is no guarantee that they will remain forever. I suggest that if you use them you should also provide the authoritative link so that if the Google link breaks at some future time then the reader will still be able to access the data.


3. Not giving the URLs of links in human readable form. Print-outs of blog pages will lose links and blog reading by email will also generally lose links (although it would be possible to preserve them). This counts for a small part of your readership but there’s no reason not to support their needs by also including links as text (either in the body or at the end of the post). I suggest including the URL in brackets, the most important thing is that no non-URL text touch the ends of the URL (don’t have it in quotes and have the brackets spaced from it). Email clients can generally launch a web browser if the URL is clear. Note that prior to writing this post I have done badly in this regard, while thinking about the best advice for others I realised that my own blogging needed some improvement.
   I am not certain that the practice I am testing in this post of citing URLs inline will work. Let me know what you think via comments, I may change to numbering the citations and providing a list of links in the footer.


4. Non-specific links. For example saying “Russell Coker wrote a good post about the SE Linux” and referring to my main blog URL is not very helpful to your readers as I have written many posts on that topic and plan to write many more (and there is a chance that some of my future posts on that topic may not meet your criteria of being “good”). Saying “here is a link to a good post by Russell Coker, his main blog URL is here” is more useful, it gives both the specific link (indicating which post you were referring to) and the general information (for people who aren’t able to find it themselves, for the case of deleted/renamed posts, and for Google). The ideal form would be “<A HREF=”http://etbe.coker.com.au/whatever”>here is a link to a good post by Russell Coker [ http://etbe.coker.com.au/whatever ]</A>, his main blog URL is <A HREF=”http://etbe.coker.com.au/”> [ http://etbe.coker.com.au ]</A>” (note that this is an example of HTML code as a guide for people who are writing their own HTML, people who use so-called WYSIWYG editors will need to do something different).


5. Links that are likely to expire. As a rule of thumb if a link is not human readable then the chance of it remaining long-term is low. Companies with content management systems are notorious for breaking links.


6. Referencing data that you can’t find. If you use data sourced from a web site and the site owner takes it down then you may be left with no evidence to support your assertions. If data is likely to be removed then you should keep a private copy off-line (online might be an infringement of copyright) for future reference. It won’t let you publish the original data but will at least let you discuss it with readers.


7. Referencing non-public data. The Open Access movement [ http://en.wikipedia.org/wiki/Open_access ] aims to make scholarly material free for unrestricted access. If you cite papers that are not open access then you deny your readers the ability to verify your claims and also encourage the companies that deny access to research papers.
   An insidious problem is with web sites such as the New York Times [ www.nytimes.com ] which need a login and store cookies. As I have logged in to their site at some time in the past I get immediate access to all their articles. But if I reference them in a blog post many readers will be forced to register (some readers will object to this). With the NYT this isn’t such a problem as it’s free to register so anyone who is really interested can do so (with a fake name if they wish). But I still have to keep thinking about the readers for such sites.
   I should probably preview my blog posts from a different account without such cookies.


8. Failing to provide calculations. My current procedure is to include the maths in my post, for example if you have a 32bit data type used to store a number of milliseconds then it can store 2^32/1000 seconds which is 2^32/1000/60/60/24 = 49.7 days, in this example you can determine with little guessing what each of the numbers represent. For more complex calculations an appendix could be used. A common feature of blogs is the ability to have a partial post sent to the RSS feed and the user has the ability to determine where the post gets cut. So you could cut the post before the calculations, the people who want to see them will find it’s only one click away, and the people who are happy to trust you will have a shorter post.


9. Linking with little reason. Having a random word appear highlighted with an underline in a blog post is often not very helpful for a reader. It sometimes works for Wikipedia links where you expect that most readers will know what the word means but you want to link to a reference for the few who don’t (my link for the word Wikipedia is an example). In the case where most readers are expected to know what you are referring to then citing the link fully (with a description of the link and a human-readable form for an email client) is overkill and reduces the readability of the text.
   The blogging style of “see here and here for examples” does not work via email and does not explain why a reader should visit the sites. If you want to include random links in a post then having a section at the footer of related links would probably be best.


10. Linking to a URL as received. Many bloggers paste URLs from Google, email, and RSS feeds into their blog posts. This is a bad idea because it might miss redirection to a different site. If a Google search or an email gives you a URL that is about to go away then it might redirect to a different site. In that case citing the new URL instead of the old one is a service to your readers and will decrease the number of dead-links in your blog over the long-term. Also using services such as www.feedburner.com may cause redirects that you want to avoid when citing a blog post, see my previous post about Feedburner [ http://etbe.coker.com.au/2007/08/20/feedburner-item-link-clicks/ ].

Here are some less common problems in citing posts:

1. Inappropriately citing yourself. Obviously if there is a topic that you frequently blog about then there will be benefit to linking to old posts instead of covering all the background material, and as long as you don’t go overboard there should not be any problems (links to your own blog are assumed to have the same author so there is no need for a disclaimer). If you write authoritative content on a topic that is published elsewhere then you will probably want to blog about it (and your readers will be interested). But you must mention your involvement to avoid giving the impression that you are trying to mislead anyone. This is particularly important if you are part of a group that prepares a document, your name may not end up on the list of authors but you have a duty to your readers to declare this.
   Any document that you helped prepare can not be used by itself as a support of claims that you make in a blog post. You can certainly say “I have previously demonstrated how to solve this problem, see the following reference”. But links with comments such as “here is an example of why X is true” are generally interpreted to be partly to demonstrate the popular support for an idea.


2. Citing secret data. The argument “if you knew what I know then you would agree with me” usually won’t be accepted well. There are of course various levels of secrecy that are appropriate. For example offering career advice without providing details of how much money you have earned (evidence of one aspect of career success) is acceptable as the readers understand the desire for some degree of financial secrecy (and of course in any game a coach doesn’t need to be a good player). Arguing the case for a war based on secret data (as many bloggers did) is not acceptable (IMHO), neither is arguing the case for the use of a technology without explaining the science or maths behind it.


3. Not reading the context of a source. For example I was reading the blog of a well regarded expert in an area of computer science, and he linked to another blog to support one of his claims. I read the blog in question (more than just the post he cited) and found some content that could be considered to be racially offensive and much of the material that I read contained claims that were not adequately supported by facts or logic. I find it difficult to believe that the expert in question (for whom I have a great deal of respect) even casually inspected the site in question. In future I will pay less attention to his posts because of this. I expect a blogger to pay more attention to the quality of their links than I do as a reader of their blog.

While writing this post I realised that my own blogging can be improved in this regard. Many of my older posts don’t adequately cite references. If you believe that any of my future posts fail in this regard then please let me know.

Share This

I was just thinking about this. 1995, in the mac lab. The mbone window using a tons of bandwidth to show video of someone's paper crane mobile. Mostly hidden in the background, but ocasionally watched as an antidote to the stess-making car-crash sounds from 30 other macs as we all learned how to make them crash programming them in assembly. Anyone else remember those paper cranes on the mbone?

Got back to the appartment last night to find police on High School Way, interviewing somebody innocuous-looking. When I went to the door to the appartment complex, it turned out that there were upwards of a dozen police, with six patrol cars, on Castro St. Most were standing around doing nothing but one of them appeared to have stopped a car and was interviewing the driver. I approached a chap at the edge of a crowd of onlookers and asked what was going on, and he declined to comment. At about that moment, the helicopter flew past for the second time. Just as Pete and I were about to go, somebody cycled along the and stopped to cross Casto, halting in the middle of a gathering of police officers. One told him, "It’s illegal to cycle on the sidewalk, sir." His reply was a work of genius: "Oh, really?" It was all very bizarre, though not at all menacing or confrontational. No guns were drawn and nobody was incapacitated by tazer or baton. Just another data point. Then we went up to the appartment proper and watched Hot Fuzz. Excellent film, especially given that the room was still being illuminated by the occasional strobe of blue lights.

Great essay from Paul Graham about why college prestige doesn’t matter. (I wish I had both the time and ability to churn out his volume of high-quality writing.) My favorite parts were the following two footnotes:

…No one ever measures recruiters by the later performance of people they turn down.[2]…

[2] Actually, someone did, once. Mitch Kapor’s wife Freada was in charge of HR at Lotus in the early years. (As he is at pains to point out, they did not become romantically involved till afterward.) At one point they worried Lotus was losing its startup edge and turning into a big company. So as an experiment she sent their recruiters the resumes of the first 40 employees, with identifying details changed. These were the people who had made Lotus into the star it was. Not one got an interview.

…Obviously you can’t prove this in the case of a single individual, but you can tell from aggregate evidence: you can’t, without asking them, distinguish people who went to one school from those who went to another three times as far down the US News list. [3]… [3] The US News list? Surely no one trusts that. Even if the statistics they consider are useful, how do they decide on the relative weights? The reason the US News list is meaningful is precisely because they are so intellectually dishonest in that respect. There is no external source they can use to calibrate the weighting of the statistics they use; if there were, we could just use that instead. What they must do is adjust the weights till the top schools are the usual suspects in about the right order. So in effect what the US News list tells us is what the editors think the top schools are, which is probably not far from the conventional wisdom on the matter. The amusing thing is, because some schools work hard to game the system, the editors will have to keep tweaking their algorithm to get the rankings they want.

As an alum of a fairly prestigious institution, I think I endorse Graham’s conclusions.

Share This

I've been using source control management (SCM) systems going back to the the very limited RCS back in the early 1990's. Over the years, I've migrated my SCM to take advantage of improved functionality. Recently, I migrated from subversion to git. While I keep my home directory in SCM (initially CVS, then SVN, now GIT) and that directory is replicated across Linux, Windows, and Mac OS X systems, a big reason for changing from svn to git is to give people more access to my open-source projects. Periodically, I'd have my CVS and SVN repositories available on a public web server. But, that typically required some effort of mine to mirror a public copy from my private repository. Using a distributed SCM like git, there is no additional effort needed to make a public copy of my respository. But, a bigger win over just making a public respository available is allow users to have their own local respository. While I appreciate the patches (bug fixes and improvments) users email to me, there are some cases where I feel the patch has limited value and don't incorporate it into my upstream respository. With a distributed SCM, users can maintain their own local patches while being able to merge changes that I make to the project's canonical repository. Though, I do hope users to continue to send patches that would be useful to a project's user base. As an added benefit, git has much improvement mechanisms for users to do so. For my private use, the distributed repository system is not a big win. However, git has much improved branching over subversion which will help when I want to make a local branch for experimental changes. To convert from my single private respository to a useful public git repository, I've separated my open-source projects into discrete git repositories. The histories of the git repositories on some projects go back to 2002. Unfortunately, I was not able to migrate my tags from subversion to git because git-svnimport got confused about the odd tagging/commits from the cvs2svn tool I used when I migrated from CVS to SVN. If checking out a particular old version is important, one can refer to the dates in the project's debian/changelog file to find the commit matching that release. At some point, I may write an automatic tool to retag the version numbers on the projects. The public git repositories can be browsed at http://git.b9.com/gitweb. I'm modifing my Lisp web site creation tool so it will automatically add the git repository to each project's website. For now, the typical URI to clone a git repository would in the form of git://git.b9.com/<project>.git. I expect users of my projects will find the new, much deeper access to my open-source projects empowering. The set of commands to clone the current set of open-source repositories are:
git clone git://git.b9.com/cl-base64.git
git clone git://git.b9.com/cl-modlisp.git
git clone git://git.b9.com/cl-photo.git
git clone git://git.b9.com/cl-readline.git
git clone git://git.b9.com/cl-rss.git
git clone git://git.b9.com/clsql.git
git clone git://git.b9.com/cluck.git
git clone git://git.b9.com/ctsim.git
git clone git://git.b9.com/getopt.git
git clone git://git.b9.com/hyperobject.git
git clone git://git.b9.com/irc-logger.git
git clone git://git.b9.com/kmrcl.git
git clone git://git.b9.com/lml.git
git clone git://git.b9.com/lml2.git
git clone git://git.b9.com/md5.git
git clone git://git.b9.com/pipes.git
git clone git://git.b9.com/postoffice.git
git clone git://git.b9.com/ptester.git
git clone git://git.b9.com/pubmed.git
git clone git://git.b9.com/puri.git
git clone git://git.b9.com/reversi.git
git clone git://git.b9.com/rlc.git
git clone git://git.b9.com/rt.git
git clone git://git.b9.com/uffi.git
git clone git://git.b9.com/umlisp.git
git clone git://git.b9.com/umlisp-orf.git
git clone git://git.b9.com/vcs-tree.git
git clone git://git.b9.com/wdq2wav.git
git clone git://git.b9.com/wol.git
git clone git://git.b9.com/xlunit.git
git clone git://git.b9.com/xmlutils.git
git clone git://git.b9.com/xptest.git

I've been using source control management (SCM) systems going back to the the very limited RCS back in the early 1990's. Over the years, I've migrated my SCM to take advantage of improved functionality. Recently, I migrated from subversion to git. While I keep my home directory in SCM (initially CVS, then SVN, now GIT) and that directory is replicated across Linux, Windows, and Mac OS X systems, a big reason for changing from svn to git is to give people more access to my open-source projects. Periodically, I'd have my CVS and SVN repositories available on a public web server. But, that typically required some effort of mine to mirror a public copy from my private repository. Using a distributed SCM like git, there is no additional effort needed to make a public copy of my respository. But, a bigger win over just making a public respository available is allow users to have their own local respository. While I appreciate the patches (bug fixes and improvments) users email to me, there are some cases where I feel the patch has limited value and don't incorporate it into my upstream respository. With a distributed SCM, users can maintain their own local patches while being able to merge changes that I make to the project's canonical repository. Though, I do hope users to continue to send patches that would be useful to a project's user base. As an added benefit, git has much improvement mechanisms for users to do so. For my private use, the distributed repository system is not a big win. However, git has much improved branching over subversion which will help when I want to make a local branch for experimental changes. To convert from my single private respository to a useful public git repository, I've separated my open-source projects into discrete git repositories. The histories of the git repositories on some projects go back to 2002. Unfortunately, I was not able to migrate my tags from subversion to git because git-svnimport got confused about the odd tagging/commits from the cvs2svn tool I used when I migrated from CVS to SVN. If checking out a particular old version is important, one can refer to the dates in the project's debian/changelog file to find the commit matching that release. At some point, I may write an automatic tool to retag the version numbers on the projects. The public git repositories can be browsed at http://git.b9.com/gitweb. I'm modifing my Lisp web site creation tool so it will automatically add the git repository to each project's website. For now, the typical URI to clone a git repository would in the form of git://git.b9.com/<project>.git. I expect users of my projects will find the new, much deeper access to my open-source projects empowering. The set of commands to clone the current set of open-source repositories are:
git clone git://git.b9.com/cl-base64.git
git clone git://git.b9.com/cl-modlisp.git
git clone git://git.b9.com/cl-photo.git
git clone git://git.b9.com/cl-readline.git
git clone git://git.b9.com/cl-rss.git
git clone git://git.b9.com/clsql.git
git clone git://git.b9.com/cluck.git
git clone git://git.b9.com/ctsim.git
git clone git://git.b9.com/getopt.git
git clone git://git.b9.com/hyperobject.git
git clone git://git.b9.com/irc-logger.git
git clone git://git.b9.com/kmrcl.git
git clone git://git.b9.com/lml.git
git clone git://git.b9.com/lml2.git
git clone git://git.b9.com/md5.git
git clone git://git.b9.com/pipes.git
git clone git://git.b9.com/postoffice.git
git clone git://git.b9.com/ptester.git
git clone git://git.b9.com/pubmed.git
git clone git://git.b9.com/puri.git
git clone git://git.b9.com/reversi.git
git clone git://git.b9.com/rlc.git
git clone git://git.b9.com/rt.git
git clone git://git.b9.com/uffi.git
git clone git://git.b9.com/umlisp.git
git clone git://git.b9.com/umlisp-orf.git
git clone git://git.b9.com/vcs-tree.git
git clone git://git.b9.com/wdq2wav.git
git clone git://git.b9.com/wol.git
git clone git://git.b9.com/xlunit.git
git clone git://git.b9.com/xmlutils.git
git clone git://git.b9.com/xptest.git

I’ve found a reasonably good free spider for checking for dead links, www.dead-links.com. When I told it to spider my main site www.coker.com.au it worked well and informed me of several dead links that I fixed. When I told it to spider my blog it reported an error 404 and didn’t give any useful output. Strangely it was able to display a thumbnail picture of my main blog page that correctly had all data. I was wondering if it didn’t like Wordpress so I tested it out on Chris Samuel’s blog [ www.csamuel.org ]. It seemed to have no problems with his blog, so I suspected that it was the request time. I reduced the number of blog posts shown on the front page of my blog to 1 to speed things up (which resulted in my page loading faster than his) but still the dead-links.com spider didn’t like me. I now suspect that it’s something related to me using Wordpress 2.2. I would be interested in feedback from other people who try checking their blogs for dead links, whether they are using Wordpress 2.2 or something else. In any case it’s a useful service and I recommend using it.

Share This

Anyone who has done an operating systems 101 course is familiar with the standard hierarchical page-table structure. This is essentially a radix-tree where bits of the virtual address are taken as indexes into an array of pointers, each pointing to the next level down or, at the leaves, to a physical entry. One potential issue with this style of page-table is with a very large, very sparse address space (such as in a single address-space operating system) there will be many intermediate steps with possibly only a single translation at the end (i.e. lots of single pages scattered all around your address-space). Thus we can add a bit-field guard to each of the entries in the page table (Liedtke, 1994) to help indicate only those paths through the tree which are valid (similar to a Patricia tree). Translation first proceeds as usual; the top bits are used as an index into the top level table. However, each entry now contains extra information about further valid addresses. The guard is masked against the next bits in the virtual address; if it matches (e.g. in our example is 0x2) the next step can proceed directly to the leaf node, bypassing the middle look-up. The entry can be marked as either final (i.e. pointing to a leaf node) or intermediate, in which case the process continues with the next level (which also has its own guard, and so forth). Any entry that does not match the guard is known to be false, and therefore a fault can be raised. To examine this further, consider that we know that each page in our 24-bit address space above is 4KiB, thanks to the 2¹² = 4KiB offset. Therefore each of the 16 possible values of the top four bits selects 1MiB of the 2²⁴ = 16MiB address space; e.g. 0x0 selects the first megabyte, 0x1 the second and so forth. The second 4 bits selects a 64KiB region of this 1MiB, and the last 4 bits selects a 4KiB region of this 64KiB region. This scheme could work quite well if we were using only 16 pages each separated by 1MiB. Each of these pages would only require an entry in the top level with an appropriate guard which could then be marked as final and point directly to a physical frame. Once things become less sparse however, we need to start filling in middle levels. If you have 2 pages within a 16MiB region, you need at least one extra bit of address to tell them apart. If you have 3 or 4 pages, you need at least 2 bits to tell them apart, and so forth. It would therefore be beneficial to have variable level sizes to dynamically adapt and create the most efficient lookup possible given the address space layout of a process. For example, if you knew your process was only going map pages 0x123 (100100011) and 0x124 (100100100) a good option would be to make your top-level only two entries (i.e. check the top bit) with a guard on the entry at index 0x1 of 00100. You then need a second level with 8 pointers to differentiate the remaining 3 bits (each of these would be final and hence point directly to a physical page). The more pages you start mapping, the less bits you can discard with the guard. If you follow this through to a guard of 0 bits you end up with a linear page-table. It has been shown (Elphinstone, 1999) that fixed level sizes with 16 entries tends to work well. For a 32-bit address-space with 4KiB pages (e.g. 12 bits offset) this leaves 20 bits to be mapped by the page-tables; with each level mapping 2⁴ bits this means a 5 level tree. The guard values provide enough path-compression that the table is not constantly walked. The disadvantage is that it may "over-allocate" meaning more space is dedicated to page-tables than strictly required. Deciding where and when to split levels, what size indexes to put at each level and updating all the guard values dynamically to make the smallest, most efficient page-table would create a variable-radix page-table (Szmajda and Heiser, 2003). In summary, a guarded page-table is more useful the more sparse your address space is. A variable-radix guarded page-table is complex, but could offer advantages for implementing variable page-size support and thus having positive effects on TLB coverage.

Today, I ordered a dedicated server for some friends of mine and me. We will split it up into 5 Xen instance, quite similar to Steve s xen-hosting. Thanks for the idea! Our project is name serverama, since our hosts will be named after Futurama characters.I might blog about the setup here, if I think it s interesting for the general public. I can t wait to get my hands wet in a chaos of RAID, LVM, Xen and virtual network bridges...

Time to revisit this topic. I asked debian perl for advice on packaging a CGI with modules and my current solution is to use ExtUtils::MakeMaker - I'm just writing the PL files which will generate the configuration files. I also took a look at Module::Build and PAR::Tutorial but I'm not convinced that they're both as portable and as smooth as if I can do it all from MakeMaker. There are occasional posts about Module::Build problems on lists like this one on ledgersmb-users which worry me. After koha-1.2's bespoke installer, I really want to make it as easy and simple as possible, for both development and use. There are also periodic debates about M::B at perlmonks too. Maybe we can use PAR later, like this perlmonks post suggests and apparently KrangCMS does. Now, what have I missed? (2007-09-07: 3 comments)

If you're going to build a computer case that looks like a media player ...

and if you're going to include a big volume control and an LCD player on the front ...

then aren't some buttons like "play", "next", and "pause" also a good idea?

I ask because I was browsing computer cases at the store yesterday, and apparently no-one working for the companies that design these things has thought of including some control buttons on front. To me, this seems like it would rank somewhere in the top 10 most obvious ideas ever, somewhere below "eat when you get hungry" and above "don't pick a fight with a grizzly bear", but clearly I'm missing something.

Here's my latest silly trick with the udev rules discussed previously. Add a udev rule of ACTION=="add", KERNEL=="sd*[!0-9]", ENV ID_SERIAL =="10d6_USB_2.0(HS)_Flash_Disk_A00000000001", RUN+="/usr/local/bin/dapsync" where dapsync is something like this script:-

/sbin/mount /mnt/USB-DAP \
&& cd /opt/Podcasts \
&&   /bin/tar --create --remove-files * \
       /bin/tar --extract --directory /Mount/USB-DAP  
/sbin/umount /Mount/USB-DAP \
&& /bin/ogg123 -q /home/mjr/House/Audio/three-note.ogg

and fstab has a line for /Mount/USB-DAP pointing to the right device/location thing in /dev/disk/by-id Then, build up podcasts under /opt/Podcasts, plug the audio player 10d6_USB_2.0(HS)_Flash_Disk_A00000000001 into the right port and wait for the three notes to play...

Some of my blogging is going to take place on other sites from now on. Sometimes for money reasons, sometimes for audience reasons. Eagle-eyed readers may have spotted that I've already switched on a feed of my WsM Forum writing and my new cycle-racing feed should go live in the middle of next week. I'll still be archiving those posts, just in case the other sites go off-line, and they still appear in my "all posts" feeds. I may move my satellite TV stuff off-site soon too, which would leave programming, webmastering and my cooperative business posts on my personal site. This means a site which "fits together" more neatly than my current mix, I think. After that, I may edit the categories again, merging koha back into hacks. Are there other changes which I should consider?

I have just given my parents a new computer, and part of the upgrade process lost their email passwords (which were stored in KDE preferences - this seems to happen every time KDE is upgraded). The only password that is not under my control is the password for their Optus account. So I had to get the password changed, I got my mother to phone Optus but they decided that my father had to make the request (not that I would have had any difficulty impersonating him, any of the details that they asked for which I might not have known could have been provided by my mother). So my father requested that my mother be listed as someone who is authorised to make changes to the account. It turned out that changing a password for a mailbox is a difficult operation and I needed to talk to the phone-support guy. He needed to get permission from my father for this (he didn’t seem to realise that he had just granted my mother full access so that she could authorise such things on future calls). After my father had given permission for the second time I got the password changed, the new password was “changeme“. The call-centre guy advised me to change it as “it’s a very common password“! An ISP with any clue in their call centre would get the password changed to a semi-random string. Then there was the process of changing the password. The web site didn’t work at all with Konqueror (the call-centre guy told me that only IE is supported). I used Iceweasel (Firefox) and it allowed me to change the password. The Optus web site is one of the worst I’ve ever seen, and redirecting to the MS site live.com as part of the process was only part of it. Another bad thing that Optus does is punish customers who do things that they don’t like. My original Optus contract specified unlimited uploads, the first time they cut me off for actually uploading a moderate amount of data (running Bittorrent) they disconnected me and claimed that it was a DOS attack. On a later occasion they disconnected me on a Wednesday because I had uploaded some files on the weekend. They alleged that the upload was continuing on the Sunday (although I had a very clear recollection of ending it on the Saturday), but we all agreed that the upload had ceased before the Monday. But apparently I needed to be disconnected on the Wednesday to teach me a lesson. The lesson of course is that Optus sucks really badly. They do have reasonable prices and at the time their cable network offered download speeds significantly higher than any ADSL plans could match, which is why I have continued using them (and subscribed again after buying a new house). Recently ADSL2+ has started to become common and prices have been falling. For my use and for most people who I know ADSL is now either cheaper than cable or close enough that it’s worth paying the extra to escape Optus. My parents are on an Optus plan that gives them 100MB per month of data transfer (which counts both download and upload) before they are limited to modem speed. This is enough for them, but I expect that when their Optus contract runs out there will be an ADSL plan that’s not much more expensive and they can cease using Optus too. I just wish there was a real choice of providers for the base phone service. Currently there is only Telstra and Optus, both of which are expensive and suck. Fortunately Telstra has an insane CEO who is determined to make sure that everyone in Australia learns that Telstra is a nasty monopoly that needs to be broken up or severely constrained.

Share This

Like Russell Coker I had to deal with an apache OOM crashing my main Xen instance yesterday. In my case, it is due to the Czech IP 88.102.6.114, which decided it would be a good idea to download the entire output of viewvc, recursively, and apparently also in parallel. After 2 hours and some 40 thousand hits, this eventually started enough instances of viewvc and svn that it used up 512 mb of ram and OOMed the system. I don't have a satesfactory fix yet. Viewvc shouldn't let itself be run dozens of times simulantaneuosly, surely. Apache doesn't seem to provide a way to limit the total number of CGIs it can launch at once, except by limiting the total number of connections. Apache's rlimitnproc option sadly only applies the limits after starting a CGI, rather that applying them to apache itself. Something that could control the number of CGIs started on a per-directory and per-vhost basis would be ideal. For now, I used ulimit -u when starting apache, which prevents many viewvc processes from running at once, but allows anyone who DOSes viewvc to also DOS all other CGIs on the system.

Today, I spent the entire morning working on the iso-codes package. This package features, among others, the giant ISO-3166-2 list, with all subdivisions of all countries of the world, thus about 4000names. Alastair McKinstry built this list long time ago, from various sources, as the ISO-3166 maintenance agency, does not provide the official list freely. Therefore, the work we (Alastair, Tobias Toedter, Li Daobing and myself) are doing on iso-codes is of some importance as this is the only freely available list of ISO-639 (language names), ISO-3166 (country names and subdivisions) and ISO-4217 (currency names) available in a machine parseable format, along with translations in a lot of languages. We are indeed upstream for that list which is used in many Linux distributions. So, today, I went on the ISO-3166 maintenance agency newsletters (*these* are freely available) and used them to update the subdivision list for as many countries as possible. That involved *copy and pasting* from PDF files to text files so that names are preserved along with the correct various funky characters used in many languages. You can help this work. Just install the "iso-codes" package, then check /usr/share/xml/iso-codes/iso_3166_2.xml for your country and report anythink you consider being a problem as a bug against the iso-codes package. Please note that the ISO-3166 MA only uses official references from the national standard bodies of all countries, so please use official sources as your reference. Of course, patches are very welcomed because dealing with all these names without messing with the "funky characters" is not easy. Please also note that the ISO-3166-2 standard uses "romanized" names, so only Latin characters should be used (all variants of such characters are accepted, though). My conclusion of the day: the world is bloody complicated. There are even subdivisions (named "castles") for....San Marino, one of the smallest states in the world. Luckily, the Holy See (the reall smallest state in the world), doesn't seem to have subdivisions...so far. And the country with the highest number of official subdivisions is....Slovenia (184!).

We're about 40 miles outside of Bakersfield, CA now, leaving us 289 miles to get home. We'll be packing up shortly to start the trip while Leif is still asleep. This puts us in San Francisco a day earlier than projected. It means another night in a hotel in the target city, but at least it'll mean that the driving is behind us.

Lessee, I think we were in Albuquerque last time I posted. Since then, we saw the petrified forest a second time, this time when the sun was up so we could enjoy it. The temperatures were so warm during the day, we had to idle with the AC on to keep Leif from waking up screaming. We continued on to Flagstaff after that where we found a nice healthfood deli for dinner and collecting more snacks. My back reminded me that twisting to pick up a diaper bag while holding Leif isn't a particularly good idea, so we spent the night in a hotel and a bed instead of in the camper, and trekked off to the grand canyon.

We met Karina and Chris up there. Sometime in this stretch, we also misplaced the camera. Hopefully it's not sitting at the hotel in Flagstaff. So, we only have a couple pictures of us at the GC, sitting on Karina and Chris' camera. The two of them flew up from Tucson, and Chris offered to take Angie and I up for a sightseeing tour. The last trip I had been on overtop had been an adventure, but this flight was smooth, and nice to be able to say "Hey, I missed that", and then spin around to see it again". Karina managed to keep Leif calm and happy for the hour that we were up, which was also a nice break for us. Angie and I both started to worry about him at around the same time - something like 10 minutes before the end of the flight. =)

The rest of the day was spent wandering along the rim, ending in a nice dinner and sleep at the campsite. The next morning, we went to hear a ranger talk to people passing by about his topic-du-jour. In this case, it seemed to be about the wildfires going on the south rim. He had a telescope pointed at the river and would answer any questions, though. After that we packed up and headed out.

Our intended stop yesterday had been Needles, CA. It turns out that there's nothing there. No name-brand hotels, no Mojave information centre (despite there being signs for it), and a lot of dwellings with the sides falling off. The stretch over to Barstow seemed like the most boring stretch of the trip, probably because we had intended to stop at Needles, but also probably because we know that from here we're just going home. We carried on until Leif's patience totally expired and stopped in Tehachapi for the night.

We think Leif might be starting to push out a second tooth. He also might've gotten a touch too much sun at the Grand Canyon. It was hard to find shady spots to nurse him, and his hat seems to have gotten buried somewhere in the van. He's in good spirits, but his cheeks had a red tinge to them last night. =( He's almost six months, now, so we'll be able to put sunscreen on soon. In the meantime, we'll try to find a place to buy another hat today.

Update: We got access to the place today. Yay! Straight home for us!

In the recent online article heise online - IFA special - Force-Feedback-Weste mit Kitzelattacke, Heise News (a german IT news site) had a really nice caption under one image showing a new force-feedback vest by Philips. The image looks like this:

The german caption is: “Philips’ Force-Feedback-Weste erm glicht t dliche Kitzelattacken in PC-Shootern.”
Translation to english is: “Philips Force-Feeback-Vest allows deathly tickling-attacks in PC-Shooters.” (Means First-Person-Shooters). Nice. That’s at least a pleasant way of dying: Being tickled to death. Of course, the article itself clears up the misunderstanding: The vest is more tickling than “punching” the player, even if his game-ego is killing most ferociously.