I had written a little script to create a local config, so one can as user run everything (short of actually installing packages) as user. (Which is quite useful to download all packages needed to update an offline system or to install something on that. Of course one needs that system status file for that).

When updating that script to the apt now checking signatures I had to realize, that the file with the keys to look for in Release.gpg files seems to be no file. At least it&aposs location is not stored in apt&aposs Dir section, where it would be nicely adapted to changes of the other directory, but is stored as a simple value elsewhere, so it needs an additional overwriting.</rant>

Anyway. The updated script can be downloaded here, just in case it might be of interest to anyone else.

* Sometime back, I came to know this little cool tool for previewing and comparing fonts. Did RFP–>ITP and then I came to know that it has one file gnome-specimen-about.svg under cc-by-sa license (Thanks to Torsten Werner for pointing out!). What to do?? I searched over packages and found that atleast one package is under similar license. So, I went ahead and submitted package (Was it brave step?). It took long time in NEW (due to license issue), but finally it got place. I contacted author (Wouter Bolsterlee) and told about this. A nice discussion followed. Tried Inkscape, no luck. And finally, 0.3.1 is with-out that cc-by-sa license!! You can get upstream tarball here. /me (Debian package will hit soon!)

A fairly common request is to be able to duplicate a Xen instance. For example you might have a DomU for the purpose of running Wordpress and want another DomU to run MediaWiki. The difference in configuration between two DomU’s for running web based services that are written in PHP and talking to a MySQL back-end is quite small, so copying the configuration is easier than a clean install. It is a commonly held opinion that a clean install should be done every time and that Kickstart on Red Hat, FAI on Debian and comparable technologies on other distributions can be used for a quick automatic install. I have not yet got FAI working correctly or got Kickstart working on Xen (it’s on my todo list - I’ll blog about it when it’s done). Regardless of whether it’s a good idea to copy a Xen DomU, there are often situations where clients demand it or when it’s impractically difficult to do a fresh install. I believe that the most sensible way to store block devices with Xen is to use LVM. It is a requirement for a Xen system that you can easily create new block devices while the machine is running and that the size of block devices can be changed with minimal effort. This rules out using Linux partitions and makes it unreasonably difficult to use LUNs on a fiber-channel SAN or partitions on a hardware RAID. LVM allows creating new block devices and changing the size of block devices with minimal effort. Another option would be to use files on a regular filesystem to store the filesystem data for Xen DomU’s, if choosing this option I recommend using XFS [1] filesystem (which delivers good performance with large filesystems and large files). If you use XFS to store the block devices for the DomU that you want to copy then you will need to halt the DomU for the duration of the copy as there is no other way of getting an atomic copy of the filesystem while it’s in use. The way of doing this would be to run the command “xm console foo ; cp /mnt/whatever/foo-root /mnt/whatever/bar-root ; xm create -c foo” where “foo” is the name of the DomU and “/mnt/whatever/foo-root” is the file that is used to store the root device for the DomU (note that multiple cp commands would be needed if there are multiple block devices). The reason for having the two xm commands on the one line is that you initially login to the DomU from the console and type halt and then the xm command will terminate when the DomU is destroyed. This means that there is no delay from the time the domain is destroyed to the time that the copy starts. If you use LVM to store the block device then things are a little easier (and you get no down-time). You simply run the command “lvcreate -s -L 300m -n foo-snap /dev/V0/foo-root” to create a snapshot with the device name /dev/V0/foo-snap which contains a snapshot the of the LV (Logical Volume) /dev/V0/foo-root. The “-L 300m” option means to use 300Meg of storage space for the snapshot - if the writes to /dev/V0/foo-root exceed 300Meg of data then your snapshot breaks. There is no harm in setting the allocated space for the snapshot to be the same as the size of the volume that you are going to copy - it merely means that more disk space is reserved and unavailable for other LVM operations. Note that V0 needs to be replaced by the name of the LVM VG (Volume Group) Once you have created the snapshot you can create a new LV with the command “lvcreate -n new-root -L X /dev/V0” where X is the size of the device (must be at least as big as the device you are copying) and then copy the data across with a command similar to “dd if=/dev/V0/foo-snap of=/dev/V0/new-root bs=1024k“. After the copy is finished you must remove the snapshot with the command “lvremove /dev/V0/foo-snap” (please be very careful when running this command - you really don’t want to remove an LV that has important data). Note that in normal operation lvremove will always give a prompt “Do you really want to remove active logical volume“. If you made the new device bigger then you must perform the operations that are appropriate for your filesystem to extend it’s size to use the new space. There is no need to copy a swap device, it’s easier to just create a new device and run mkswap on it. After copying the data you will need to create the new Xen config (by copying /etc/xen/foo to the new name). Make sure that you edit the Xen config file to use the correct block devices and if you are specifying the MAC address [2] by a “vif” line in the config file make sure that you change them to unique addresses for your LAN segment (reference [2] has information on how to select addresses). Now you must mount the filesystem temporarily to change the IP address (you really don’t want two DomU’s with the same IP address). If your Dom0 has untrusted users or services that are accessed by untrusted users (IE any Internet facing service) then you want to mount the filesystem in question with the options nosuid and nodev so that if the DomU has been cracked it won’t allow cracking of the Dom0. After changing the configuration files to change the IP address(es) of the DomU you can then umount the filesystem and start it with the xm create command. If instead of creating the clone DomU on the same Dom0 you want to put it on a different system you can copy the block devices to files on a regular filesystem on removable media (EG an IDE disk with USB attachment). When copying the block devices you also need to copy the Xen configuration and edit it to reflect the new paths to block devices for the data once it’s copied to the new server, but you won’t necessarily need to change the MAC address if you are copying it to a different LAN segment. References:

1. http://en.wikipedia.org/wiki/XFS


2. http://en.wikipedia.org/wiki/MAC_address

Share This

Some people at DebConf 7 were (like Mario) wondering how many users Debian has. They realised that while we have a distributed mirror network we do have all installed systems use security.debian.org by default and we do have control over that. Since apt will only download files when they have been changed we can make a minimal estimate by looking at the number of downloads of the release files while they remain unchanged. Obviously, this will only be a minimum - for example, some users won’t have network access, some users will use caching proxies or local mirrors - but it’s a start. Unfortunately I can’t actually remember the numbers they came up with but perhaps someone could enlighten us.

A significant problem with the old-fashioned media is that as a general rule they don’t cite references for anything. Some of the better TV documentaries and non-fiction books cite references, but this is the exception not the norm. Often documentaries only cite references in DVD extras which are good for the people who like the documentary enough to buy it but not for people who want to rebut it (few people will pay for a resource if they doubt the truth and accuracy of it’s claims). I can understand newspapers not wanting to publish much in the way of background information in the paper version as every extra line of text in an article is a line of advertising that they can’t sell. So they have financial pressure to produce less content, and the number of people like me who want to check the facts and figures used in articles is probably a small portion of the readership. Another issue with newspapers is that they are often considered as primary authoritative sources (by themselves and by the readers). It is often the case that journalists will interview people who have first-hand knowledge of an issue and the resulting article will be authoritative and a primary source in which case all they need to do is to note that they interviewed the subject. However the majority of articles published will be sourced from elsewhere (news agencies [ http://en.wikipedia.org/wiki/News_agency ] such as Reuters are commonly used). Also articles will often be written based on press releases - it is very interesting to read press releases and see how little work is done by some media outlets to convert them to articles, through a well written press release a corporation or interest group can almost write it’s own articles for publication in the old media. One way of partially addressing the problem of citing references in old media would be to create a web site of references, then every article could have a URL that is a permanent link to the references and calculations to support the claims and numbers used. Such a URL could be produced by any blogging software, and a blog would be an ideal way of doing this. For bloggers however it’s much easier to cite references and readers have much higher expectations of links to other sites to support claims and of mathematical calculations shown to indicate how numbers are determined. But there is still room for improvement. Here are some of the most common mistakes that I see in posts by people who are trying to do the right thing:

1. Indirect links. When you refer to a site you want to refer to it directly. In email (which is generally considered a transient medium) a service such as TinyURL [ www.TinyURL.com ] can be used to create short URLs to refer to pages that have long URLs. This is really good for email as there are occasions when people will want to write the address down and type it in to another computer. For blogging you should assume that your reader has access to browse the web (which is the case most of the time). Another possibility is to have the textual description of a link include a reference to the TinyURL service but to have the HREF refer to the real address. Any service on the net may potentially go away at some future time. Any service on the net may have transient outages, and any reader of your blog may have routing problems that make parts of the net unavailable to them. If accessing a reference requires using TinyURL (or a similar service) as well as the target site then there are two potential things that might break and prevent your readers from accessing it.
   One situation where indirect links are acceptable is for the printed version. So you could have a link in the HTML code for readers to click on to get to the reference page directly and a TinuURL link for people who have a printed version and need to type it in.
   Also when linking to a blog it’s worth considering the fact that a track-back won’t work via TinyURL and track-backs may help you get more readers…


2. Links that expire. For example never say “there’s a good article on the front page of X” (where X is a blog or news site). Instead say “here’s a link to a good article which happens to be on the front page now” so that someone who reads your post in a couple of years time can see the article that you reference.
   Another problem is links to transient data. For example if you want to comment on the features of a 2007 model car you should try to avoid linking to the car manufacturer page, next year they will release a new car and delete the old data from their site.
   A potential problem related to this is the Google cache pages which translate PDF to HTML and high-light relevant terms and can make it much easier to extract certain information from web pages. It can provide value to readers to use such links but AFAIK there is no guarantee that they will remain forever. I suggest that if you use them you should also provide the authoritative link so that if the Google link breaks at some future time then the reader will still be able to access the data.


3. Not giving the URLs of links in human readable form. Print-outs of blog pages will lose links and blog reading by email will also generally lose links (although it would be possible to preserve them). This counts for a small part of your readership but there’s no reason not to support their needs by also including links as text (either in the body or at the end of the post). I suggest including the URL in brackets, the most important thing is that no non-URL text touch the ends of the URL (don’t have it in quotes and have the brackets spaced from it). Email clients can generally launch a web browser if the URL is clear. Note that prior to writing this post I have done badly in this regard, while thinking about the best advice for others I realised that my own blogging needed some improvement.
   I am not certain that the practice I am testing in this post of citing URLs inline will work. Let me know what you think via comments, I may change to numbering the citations and providing a list of links in the footer.


4. Non-specific links. For example saying “Russell Coker wrote a good post about the SE Linux” and referring to my main blog URL is not very helpful to your readers as I have written many posts on that topic and plan to write many more (and there is a chance that some of my future posts on that topic may not meet your criteria of being “good”). Saying “here is a link to a good post by Russell Coker, his main blog URL is here” is more useful, it gives both the specific link (indicating which post you were referring to) and the general information (for people who aren’t able to find it themselves, for the case of deleted/renamed posts, and for Google). The ideal form would be “<A HREF=”http://etbe.coker.com.au/whatever”>here is a link to a good post by Russell Coker [ http://etbe.coker.com.au/whatever ]</A>, his main blog URL is <A HREF=”http://etbe.coker.com.au/”> [ http://etbe.coker.com.au ]</A>” (note that this is an example of HTML code as a guide for people who are writing their own HTML, people who use so-called WYSIWYG editors will need to do something different).


5. Links that are likely to expire. As a rule of thumb if a link is not human readable then the chance of it remaining long-term is low. Companies with content management systems are notorious for breaking links.


6. Referencing data that you can’t find. If you use data sourced from a web site and the site owner takes it down then you may be left with no evidence to support your assertions. If data is likely to be removed then you should keep a private copy off-line (online might be an infringement of copyright) for future reference. It won’t let you publish the original data but will at least let you discuss it with readers.


7. Referencing non-public data. The Open Access movement [ http://en.wikipedia.org/wiki/Open_access ] aims to make scholarly material free for unrestricted access. If you cite papers that are not open access then you deny your readers the ability to verify your claims and also encourage the companies that deny access to research papers.
   An insidious problem is with web sites such as the New York Times [ www.nytimes.com ] which need a login and store cookies. As I have logged in to their site at some time in the past I get immediate access to all their articles. But if I reference them in a blog post many readers will be forced to register (some readers will object to this). With the NYT this isn’t such a problem as it’s free to register so anyone who is really interested can do so (with a fake name if they wish). But I still have to keep thinking about the readers for such sites.
   I should probably preview my blog posts from a different account without such cookies.


8. Failing to provide calculations. My current procedure is to include the maths in my post, for example if you have a 32bit data type used to store a number of milliseconds then it can store 2^32/1000 seconds which is 2^32/1000/60/60/24 = 49.7 days, in this example you can determine with little guessing what each of the numbers represent. For more complex calculations an appendix could be used. A common feature of blogs is the ability to have a partial post sent to the RSS feed and the user has the ability to determine where the post gets cut. So you could cut the post before the calculations, the people who want to see them will find it’s only one click away, and the people who are happy to trust you will have a shorter post.


9. Linking with little reason. Having a random word appear highlighted with an underline in a blog post is often not very helpful for a reader. It sometimes works for Wikipedia links where you expect that most readers will know what the word means but you want to link to a reference for the few who don’t (my link for the word Wikipedia is an example). In the case where most readers are expected to know what you are referring to then citing the link fully (with a description of the link and a human-readable form for an email client) is overkill and reduces the readability of the text.
   The blogging style of “see here and here for examples” does not work via email and does not explain why a reader should visit the sites. If you want to include random links in a post then having a section at the footer of related links would probably be best.


10. Linking to a URL as received. Many bloggers paste URLs from Google, email, and RSS feeds into their blog posts. This is a bad idea because it might miss redirection to a different site. If a Google search or an email gives you a URL that is about to go away then it might redirect to a different site. In that case citing the new URL instead of the old one is a service to your readers and will decrease the number of dead-links in your blog over the long-term. Also using services such as www.feedburner.com may cause redirects that you want to avoid when citing a blog post, see my previous post about Feedburner [ http://etbe.coker.com.au/2007/08/20/feedburner-item-link-clicks/ ].

Here are some less common problems in citing posts:

1. Inappropriately citing yourself. Obviously if there is a topic that you frequently blog about then there will be benefit to linking to old posts instead of covering all the background material, and as long as you don’t go overboard there should not be any problems (links to your own blog are assumed to have the same author so there is no need for a disclaimer). If you write authoritative content on a topic that is published elsewhere then you will probably want to blog about it (and your readers will be interested). But you must mention your involvement to avoid giving the impression that you are trying to mislead anyone. This is particularly important if you are part of a group that prepares a document, your name may not end up on the list of authors but you have a duty to your readers to declare this.
   Any document that you helped prepare can not be used by itself as a support of claims that you make in a blog post. You can certainly say “I have previously demonstrated how to solve this problem, see the following reference”. But links with comments such as “here is an example of why X is true” are generally interpreted to be partly to demonstrate the popular support for an idea.


2. Citing secret data. The argument “if you knew what I know then you would agree with me” usually won’t be accepted well. There are of course various levels of secrecy that are appropriate. For example offering career advice without providing details of how much money you have earned (evidence of one aspect of career success) is acceptable as the readers understand the desire for some degree of financial secrecy (and of course in any game a coach doesn’t need to be a good player). Arguing the case for a war based on secret data (as many bloggers did) is not acceptable (IMHO), neither is arguing the case for the use of a technology without explaining the science or maths behind it.


3. Not reading the context of a source. For example I was reading the blog of a well regarded expert in an area of computer science, and he linked to another blog to support one of his claims. I read the blog in question (more than just the post he cited) and found some content that could be considered to be racially offensive and much of the material that I read contained claims that were not adequately supported by facts or logic. I find it difficult to believe that the expert in question (for whom I have a great deal of respect) even casually inspected the site in question. In future I will pay less attention to his posts because of this. I expect a blogger to pay more attention to the quality of their links than I do as a reader of their blog.

While writing this post I realised that my own blogging can be improved in this regard. Many of my older posts don’t adequately cite references. If you believe that any of my future posts fail in this regard then please let me know.

Share This

The short story: we lost the baby. "Cord accident". The long story: Sarah woke me up at about 2:30am on Tuesday morning in a panic because she'd woken up herself at about 2am, and realised that she hadn't felt the baby move since about 8am the previous day. I had a quick feel, but couldn't feel anything like his usual vigorous kicking, and eager to put her mind at rest, called Labor an Delivery at Lucile Packard Children's Hospital to see if we could come in. They said we could, so we jumped in the car and got there at about 3:25am. They hooked up Sarah to a fetal heart monitor, but after anxious minutes, they couldn't pick up anything, so they got out the ultrasound machine. It was pretty obvious from that that there was no movement. We'd lost him. So from about 4am onwards, we, well, tried to come to grips with the fact that we'd lost our baby. That we'd been robbed, 13 days from the finish line, and that we weren't going to be bringing a baby home to all the stuff we had ready for him there. The doctors briefly mulled over whether to have a C-section to deliver Joshua, as was the plan, or induce. Apparently it's psychologically better for the mother if she delivers naturally, since she doesn't have the constant reminder for 6+ weeks while she recovers from the C-section. However, the doctors decided that as nothing had really changed from a delivery point of view, and they wanted to do a C-section for reduced strain on Sarah's heart, they opted for a C-section. They managed to get a free operating room slot in the Stanford ORs, so they transferred Sarah over there, and they started performing the surgery at about 10am. Joshua was delivered sometime around 10:15am. I got to see him and hold him while we were still in the OR. He was perfect. 4 pounds 12 ounces. He didn't look that small. We were wondering if he was going to make 5 pounds or not. Maybe he would have by the 17th. I brought him around to show Sarah while she was still on the table, but it was hard to get him up to a good position that she could see him from, and she was a bit zonked out from all of the drugs. They finished up at about 10:45am, and took her to the recovery room, where they didn't really want me, so one of the nurses from Labor and Delivery took me back to Lucile Packard, to spend some time with Joshua. There she let me hold him for a while, and look at him, and take some photographs. Then she weighed him and measured him, and took some more photographs. The nurse was really compassionate about the whole thing, and it felt like we were there for a good couple of hours, which was good, because apparently Lucile Packard and Stanford don't really inter-operate very well, and whilst physically transferring a patient just involves wheeling them down a corridor, getting all the paperwork sorted out is a completely different story. So while Sarah was only supposed to spend an hour in recovery, she spent three hours there, because they were waiting on paperwork to release her, then waiting on a room to be available, then the right people weren't talking to each other, so they didn't know they didn't know that a room was available from midday. I forget what time we actually got out of recovery. The doctors, being the overcautious bunch of people that they are, wanted to keep Sarah in the Cardiac Surveillance Unit for 24-48 hours so they could check on her vitals post-delivery. They let us out of there late today. We're currently in a general ward. I thought they said it was for gynaecology patients (they didn't think it'd be terribly considerate to stick us in with all the other recovering mothers and their babies), but there's some male patients in here too, so I'm not sure what this ward is. In terms of coming to grips with things, they brought Joshua up to our room for pretty much as long as we wanted yesterday, and Sarah was more with it by then, so she could hold him, and we could take more photographs. It's been really hard. I was awake from when Sarah woke me up in the morning until about 10pm that night. Fortunately I was able to get a cot in the room with Sarah, so I could stay the night there. I'm staying here tonight as well. The obstetrician that delivered Joshua said that his umbilical cord was wrapped around his neck twice and once around his body, and was pretty tight around his neck. They can't say for sure that that's what happened, but it's a good enough explanation for us, and we're not going to have the little guy cut up for an autopsy, which may not tell us anything anyway. So we're taking some small comfort from the fact that we're able to make a healthy looking baby, and that Sarah's heart performed fine throughout the pregnancy and delivery, and that this was just one of those so very unlucky accidents. Apparently babies get their umbilical cords wrapped around their necks all the time, and most of the time it's not a problem. So we know that he didn't have any birth defects, and it wasn't because of something Sarah ate, or because Sarah got sick or anything. It's just plain bad, rotten luck. We'll most probably try again, but the next pregnancy is going to be hell. It'll be like walking on egg shells the entire time. Clearly, your baby isn't out of the water until he's screaming in the delivery room. There's a lot more I want to write, but I also want to get this finished and get some sleep, so here's a brain dump of what's been going through my head for the last 48 hours:

• Dammit!
• Dammit! Dammit! Dammit!
• We only had 13 days to go
• We just had the baby shower on Saturday!
• We just bought the last few things at Babies 'r' Us on Monday. He was probably dead by then.
• We were so ready. We'd gotten everything. We bought a cool jogging stroller! We bought a super-awesome safety table of a high chair.
• I never thought my Tuesday would end up like this when I went to bed on Monday night
• He looked so perfect. We've been robbed. He's been robbed.
• The little bugger was too active for his own good
• We were supposed to be having our Caesarian class the next day
• Should we try again? Can I handle seeing Sarah go through all the pain and suffering of another C-section?
• It's just not fair!
• I wouldn't wish this on anyone
• This sucks
• This just isn't fair
• We'll get through this

We've been very grateful for the near endless stream of visitors, phone calls and SMSes, and flowers. It's helped us not dwell on things, especially given all of our family is so far away. Sarah's Mum, who was going to arrive in 5 weeks, is now arriving on Friday, so that'll be good for Sarah. Sarah will hopefully be discharged on Friday or Saturday. We're planning on cremating Joshua, as it just doesn't seem right to bury him over here and then someday move back to Australia and leave him all alone. Current thinking is we won't have a funeral service. Here's what I emailed to the people we were going to send the birth announcement to:

Hi everyone, It is with heavy hearts, that we have to tell you that rather than announcing Joshua's birth on the 17th of September, we have to say that yesterday, we learned that we lost him, 13 days before he was to be delivered. Sarah woke me yesterday morning at about 2:30am, because she'd woken up at about 2am, and realised that she hadn't felt Joshua moving since about 8am the previous day. She tried a few things to get him moving, but they didn't work, so we hurried into Labor & Delivery at the Lucille Packard Children's hospital. There, they confirmed the worst, no heartbeat. Yesterday morning at about 10am, Sarah had a c-section so the doctors could deliver little Joshua. He was absolutely perfect, weighing in at 4 pounds 12 ounces (2.15 kilograms) and 26.7 centimetres (10.5 inches). Yesterday was a very long day. Sarah is currently recovering in the Cardiac Surveillance Unit (CSU) of Stanford Hospital (her aorta has been absolutely fine, the doctors are just being their usual cautious selves). It's a little bit up in the air as to whether she'll spend the entirety of her recovery in the CSU, or if she'll be transferred back to the Post-Partum area of Lucille Packard after 24 hours. (We hope that she does get transferred back, as the CSU isn't really used to dealing with post-c-section patients, particularly ones that have lost the baby). The obstetrician that delivered Joshua said that his umbilical cord was wrapped around his neck twice, fairly tightly, and once around his body. So whilst we'll never know for sure, it looks like it was a cord accident. That's a good enough explanation for us, so we're not going to have him autopsied. That's about all I can report at the moment. If you'd like to speak to Sarah, you can either call home and I'll try to keep Sarah's extension forwarding to her hospital room, or you can call my cell phone. I've got it on vibrate all of the time, and I'll answer if we're not in the middle of something. If you're calling from Australia, my extension at home will let you connect to my cell phone. If you're local, and would like to visit, we'd love to see you, just call ahead to check where we are. It's going to be hard, but we'll get through this.

I've been using source control management (SCM) systems going back to the the very limited RCS back in the early 1990's. Over the years, I've migrated my SCM to take advantage of improved functionality. Recently, I migrated from subversion to git. While I keep my home directory in SCM (initially CVS, then SVN, now GIT) and that directory is replicated across Linux, Windows, and Mac OS X systems, a big reason for changing from svn to git is to give people more access to my open-source projects. Periodically, I'd have my CVS and SVN repositories available on a public web server. But, that typically required some effort of mine to mirror a public copy from my private repository. Using a distributed SCM like git, there is no additional effort needed to make a public copy of my respository. But, a bigger win over just making a public respository available is allow users to have their own local respository. While I appreciate the patches (bug fixes and improvments) users email to me, there are some cases where I feel the patch has limited value and don't incorporate it into my upstream respository. With a distributed SCM, users can maintain their own local patches while being able to merge changes that I make to the project's canonical repository. Though, I do hope users to continue to send patches that would be useful to a project's user base. As an added benefit, git has much improvement mechanisms for users to do so. For my private use, the distributed repository system is not a big win. However, git has much improved branching over subversion which will help when I want to make a local branch for experimental changes. To convert from my single private respository to a useful public git repository, I've separated my open-source projects into discrete git repositories. The histories of the git repositories on some projects go back to 2002. Unfortunately, I was not able to migrate my tags from subversion to git because git-svnimport got confused about the odd tagging/commits from the cvs2svn tool I used when I migrated from CVS to SVN. If checking out a particular old version is important, one can refer to the dates in the project's debian/changelog file to find the commit matching that release. At some point, I may write an automatic tool to retag the version numbers on the projects. The public git repositories can be browsed at http://git.b9.com/gitweb. I'm modifing my Lisp web site creation tool so it will automatically add the git repository to each project's website. For now, the typical URI to clone a git repository would in the form of git://git.b9.com/<project>.git. I expect users of my projects will find the new, much deeper access to my open-source projects empowering. The set of commands to clone the current set of open-source repositories are:
git clone git://git.b9.com/cl-base64.git
git clone git://git.b9.com/cl-modlisp.git
git clone git://git.b9.com/cl-photo.git
git clone git://git.b9.com/cl-readline.git
git clone git://git.b9.com/cl-rss.git
git clone git://git.b9.com/clsql.git
git clone git://git.b9.com/cluck.git
git clone git://git.b9.com/ctsim.git
git clone git://git.b9.com/getopt.git
git clone git://git.b9.com/hyperobject.git
git clone git://git.b9.com/irc-logger.git
git clone git://git.b9.com/kmrcl.git
git clone git://git.b9.com/lml.git
git clone git://git.b9.com/lml2.git
git clone git://git.b9.com/md5.git
git clone git://git.b9.com/pipes.git
git clone git://git.b9.com/postoffice.git
git clone git://git.b9.com/ptester.git
git clone git://git.b9.com/pubmed.git
git clone git://git.b9.com/puri.git
git clone git://git.b9.com/reversi.git
git clone git://git.b9.com/rlc.git
git clone git://git.b9.com/rt.git
git clone git://git.b9.com/uffi.git
git clone git://git.b9.com/umlisp.git
git clone git://git.b9.com/umlisp-orf.git
git clone git://git.b9.com/vcs-tree.git
git clone git://git.b9.com/wdq2wav.git
git clone git://git.b9.com/wol.git
git clone git://git.b9.com/xlunit.git
git clone git://git.b9.com/xmlutils.git
git clone git://git.b9.com/xptest.git

I've been using source control management (SCM) systems going back to the the very limited RCS back in the early 1990's. Over the years, I've migrated my SCM to take advantage of improved functionality. Recently, I migrated from subversion to git. While I keep my home directory in SCM (initially CVS, then SVN, now GIT) and that directory is replicated across Linux, Windows, and Mac OS X systems, a big reason for changing from svn to git is to give people more access to my open-source projects. Periodically, I'd have my CVS and SVN repositories available on a public web server. But, that typically required some effort of mine to mirror a public copy from my private repository. Using a distributed SCM like git, there is no additional effort needed to make a public copy of my respository. But, a bigger win over just making a public respository available is allow users to have their own local respository. While I appreciate the patches (bug fixes and improvments) users email to me, there are some cases where I feel the patch has limited value and don't incorporate it into my upstream respository. With a distributed SCM, users can maintain their own local patches while being able to merge changes that I make to the project's canonical repository. Though, I do hope users to continue to send patches that would be useful to a project's user base. As an added benefit, git has much improvement mechanisms for users to do so. For my private use, the distributed repository system is not a big win. However, git has much improved branching over subversion which will help when I want to make a local branch for experimental changes. To convert from my single private respository to a useful public git repository, I've separated my open-source projects into discrete git repositories. The histories of the git repositories on some projects go back to 2002. Unfortunately, I was not able to migrate my tags from subversion to git because git-svnimport got confused about the odd tagging/commits from the cvs2svn tool I used when I migrated from CVS to SVN. If checking out a particular old version is important, one can refer to the dates in the project's debian/changelog file to find the commit matching that release. At some point, I may write an automatic tool to retag the version numbers on the projects. The public git repositories can be browsed at http://git.b9.com/gitweb. I'm modifing my Lisp web site creation tool so it will automatically add the git repository to each project's website. For now, the typical URI to clone a git repository would in the form of git://git.b9.com/<project>.git. I expect users of my projects will find the new, much deeper access to my open-source projects empowering. The set of commands to clone the current set of open-source repositories are:
git clone git://git.b9.com/cl-base64.git
git clone git://git.b9.com/cl-modlisp.git
git clone git://git.b9.com/cl-photo.git
git clone git://git.b9.com/cl-readline.git
git clone git://git.b9.com/cl-rss.git
git clone git://git.b9.com/clsql.git
git clone git://git.b9.com/cluck.git
git clone git://git.b9.com/ctsim.git
git clone git://git.b9.com/getopt.git
git clone git://git.b9.com/hyperobject.git
git clone git://git.b9.com/irc-logger.git
git clone git://git.b9.com/kmrcl.git
git clone git://git.b9.com/lml.git
git clone git://git.b9.com/lml2.git
git clone git://git.b9.com/md5.git
git clone git://git.b9.com/pipes.git
git clone git://git.b9.com/postoffice.git
git clone git://git.b9.com/ptester.git
git clone git://git.b9.com/pubmed.git
git clone git://git.b9.com/puri.git
git clone git://git.b9.com/reversi.git
git clone git://git.b9.com/rlc.git
git clone git://git.b9.com/rt.git
git clone git://git.b9.com/uffi.git
git clone git://git.b9.com/umlisp.git
git clone git://git.b9.com/umlisp-orf.git
git clone git://git.b9.com/vcs-tree.git
git clone git://git.b9.com/wdq2wav.git
git clone git://git.b9.com/wol.git
git clone git://git.b9.com/xlunit.git
git clone git://git.b9.com/xmlutils.git
git clone git://git.b9.com/xptest.git

I want a way to take small, infrequent, one-off card payments on my web sites. Some of my customers much prefer to pay by card and I'd be happy to do that, if there's a decent card payment service. Is there one? I've not found it yet. What do I want? The main thing is not to have a significant monthly fee. I'd rather not be paying if I'm not using the service, but I'd probably stomach up to 5% transaction fee for a good credit card handler. The 8% fee listed by Moneybookers seems a little steep, but I might use it as a last resort. I'd prefer no automatic "inactivity" account closure if I don't use it for a couple of months. Sometimes I take no card payments for a while, then need to handle several at once. I'd also prefer a cooperative or mutual, but that seems unlikely in such a small field. So far, I've only found two suitable UK Card Payment Services - Paypal and NoChex - and I've used both, but I've now got problems with both! Paypal won't let me take my money out to my business account. To do that, I need to upgrade my account, but Paypal do not list LLP as a valid type of business and won't fix my bug reports by web, email or telephone. Last time I telephoned, after patiently explaining to them what an LLP is, I was told to put a different business type on the form. That scared me, so I asked for email confirmation of the instruction, which they said they'd send, but it never arrived. NoChex froze my account without warning, due to inactivity. I've sent in their reactivation forms a few months ago and had no reply. I've also filed a web ticket today, but I've done that before. They don't even list a telephone number any more. That's a worrying sign in a bank. So, both of these providers suck for the above reasons and I still need a way to take this month's card payments. All the UK banks charge monthly fees, don't they? Are there European card services which I should consider? International? (2007-09-07: 5 comments)

Anyone who has done an operating systems 101 course is familiar with the standard hierarchical page-table structure. This is essentially a radix-tree where bits of the virtual address are taken as indexes into an array of pointers, each pointing to the next level down or, at the leaves, to a physical entry. One potential issue with this style of page-table is with a very large, very sparse address space (such as in a single address-space operating system) there will be many intermediate steps with possibly only a single translation at the end (i.e. lots of single pages scattered all around your address-space). Thus we can add a bit-field guard to each of the entries in the page table (Liedtke, 1994) to help indicate only those paths through the tree which are valid (similar to a Patricia tree). Translation first proceeds as usual; the top bits are used as an index into the top level table. However, each entry now contains extra information about further valid addresses. The guard is masked against the next bits in the virtual address; if it matches (e.g. in our example is 0x2) the next step can proceed directly to the leaf node, bypassing the middle look-up. The entry can be marked as either final (i.e. pointing to a leaf node) or intermediate, in which case the process continues with the next level (which also has its own guard, and so forth). Any entry that does not match the guard is known to be false, and therefore a fault can be raised. To examine this further, consider that we know that each page in our 24-bit address space above is 4KiB, thanks to the 2¹² = 4KiB offset. Therefore each of the 16 possible values of the top four bits selects 1MiB of the 2²⁴ = 16MiB address space; e.g. 0x0 selects the first megabyte, 0x1 the second and so forth. The second 4 bits selects a 64KiB region of this 1MiB, and the last 4 bits selects a 4KiB region of this 64KiB region. This scheme could work quite well if we were using only 16 pages each separated by 1MiB. Each of these pages would only require an entry in the top level with an appropriate guard which could then be marked as final and point directly to a physical frame. Once things become less sparse however, we need to start filling in middle levels. If you have 2 pages within a 16MiB region, you need at least one extra bit of address to tell them apart. If you have 3 or 4 pages, you need at least 2 bits to tell them apart, and so forth. It would therefore be beneficial to have variable level sizes to dynamically adapt and create the most efficient lookup possible given the address space layout of a process. For example, if you knew your process was only going map pages 0x123 (100100011) and 0x124 (100100100) a good option would be to make your top-level only two entries (i.e. check the top bit) with a guard on the entry at index 0x1 of 00100. You then need a second level with 8 pointers to differentiate the remaining 3 bits (each of these would be final and hence point directly to a physical page). The more pages you start mapping, the less bits you can discard with the guard. If you follow this through to a guard of 0 bits you end up with a linear page-table. It has been shown (Elphinstone, 1999) that fixed level sizes with 16 entries tends to work well. For a 32-bit address-space with 4KiB pages (e.g. 12 bits offset) this leaves 20 bits to be mapped by the page-tables; with each level mapping 2⁴ bits this means a 5 level tree. The guard values provide enough path-compression that the table is not constantly walked. The disadvantage is that it may "over-allocate" meaning more space is dedicated to page-tables than strictly required. Deciding where and when to split levels, what size indexes to put at each level and updating all the guard values dynamically to make the smallest, most efficient page-table would create a variable-radix page-table (Szmajda and Heiser, 2003). In summary, a guarded page-table is more useful the more sparse your address space is. A variable-radix guarded page-table is complex, but could offer advantages for implementing variable page-size support and thus having positive effects on TLB coverage.

I've found some time for quicklist, finally, to move the port on to a second pre-release. New additions include a Recent Files submenu and GtkHTML report engine but the Sort and Filter options are not complete and adding a new report is problematic.
quicklist is now ready for some translations (the first one identified a problem in pre1 that prevented certain strings from showing up as translated). Also added a prototype manual using docbook that I've made available as HTML for dwww and making the docbook file itself available to yelp.
Currently, I'm undecided just where QuickList will fit between pure-Gtk2 and pure-Gnome2. pre2 uses libgnomeprint and libgnomevfs but yelp would bring in the main gnome libraries so yelp is just a Recommends and only as an alternative to dwww:

Recommends: www-browser, dwww yelp

I hope to find some time to fix the remaining bugs but if anyone fancies helping out, see http://quicklist.sourceforge.net/ and take a look at the current CVS prior to the pre2 upload. CVS provides a method to build the updated code docs for developers (rather than the Manual which is aimed at users) using doxygen - the version on the website is out of date. This includes a list of bugs and ToDo items - most of which are centred on removing the last vestiges of the original QuickList code and refactoring to make full use of libglib2.0-0 functions like GList and GHashTable instead of fixed length multi-dimensional arrays.

This is the result of Swiss voting on ISO/IEC DIS 29500, the fast-tracking of the Microsoft Office Open XML file format.

4 screen AG	approval
Accenture AG	approval
ADVIS AG	approval
ALTRAN AG	approval
Baggenstos Wallisellen	approval
Bechtle IT-Systemhaus Thalwil	approval
CIS-Consulting	approval
Comsoft Direct AG	approval
Coris SA	approval
Dr. Pascal Sieber & Partners AG	approval
dynawell ag	approval
Ecma International	approval
ELCA Informatik AG	approval
EPFL Lausanne	disapproval
FSFE Free Software Foundation Europe	disapproval
GARAIO AG	approval
Gysel Ulrich Emanuel	disapproval
H.R. Thomann Consulting	approval
Hewlett-Packard (Schweiz) GmbH	approval
HSW Luzern, Institut IWI	approval
IAMCP Switzerland	approval
IBM (Schweiz)	disapproval
Informatikstrategieorgan Bund ISB	approval
isolutions gmbh	approval
itsystems AG	approval
Kull AG	approval
leanux.ch AG	approval
Leuchter Informatik AG	approval
MESO Products	approval
Microsoft Schweiz GmbH	approval
MondayCoffee AG	approval
Namics AG	approval
NEXPLORE AG	approval
Novell (Schweiz) AG	approval
Online Consulting AG	approval
Open Text	approval
PageUp Bern	approval
PC-WARE Systems (Schweiz) AG	approval
Puzzle ITC GmbH	disapproval
SBS Solutions AG	approval
Secunet SwissIT AG	disapproval
SIUG Swiss Internet User Group	disapproval
SKSF	approval
Skybow AG	approval
SoftwareONE	approval
SyGroup GmbH	disapproval
Sylog Consulting SA	approval
Syndrega	disapproval
TheAlternative	disapproval
Trivadis AG	approval
Unic Internet Solutions	approval
usedSoft AG	approval
Verein /ch/open	disapproval
WAGNER AG Kirchberg	approval
Wilhelm Tux (Verein)	disapproval
Würgler Consulting	disapproval
Zürcher Hochschule der Künste	disapproval
Total of voting (75% majority)	43 approval (75.4%); 14 disapproal (24.6%)

A majority with 75% has been reached with one vote. Why do Hewlett-Packard and Novell vote IN FAVOUR for OOXML!?

Time to revisit this topic. I asked debian perl for advice on packaging a CGI with modules and my current solution is to use ExtUtils::MakeMaker - I'm just writing the PL files which will generate the configuration files. I also took a look at Module::Build and PAR::Tutorial but I'm not convinced that they're both as portable and as smooth as if I can do it all from MakeMaker. There are occasional posts about Module::Build problems on lists like this one on ledgersmb-users which worry me. After koha-1.2's bespoke installer, I really want to make it as easy and simple as possible, for both development and use. There are also periodic debates about M::B at perlmonks too. Maybe we can use PAR later, like this perlmonks post suggests and apparently KrangCMS does. Now, what have I missed? (2007-09-07: 3 comments)

In the last week, I have been witness to and part of many conversations and probably one of my favorite ones was about coffee. My friend reasoned that coffee is as wonderful as it is because of its dependability (unlike, for example, your relatives). You know that for a moderate sum of money, you can drink a drink that makes you happy, alert, and, for some of us, allows us to face the rest of the day on an even keel. It is pure comfort that derives from a form of almost ritualistic dependability. This morning, as I was sipping my coffee, I came across a short blog post by Stanley Fish who certainly does not make me as happy as my morning cup of joe, but I do admire him for his dependability and consistency when it comes to reporting on matters of liberalism. For over 20 years he has dependably written on the quandaries and limit of liberal political ideology and his most recent installment, which focuses primarily on a new book by Paul Starr, is no different. Well, his conclusion strikes just a little differently than the tone of some of his previous works. In the past (or perhaps in some of his longer academic works), Fish’s solution to the problem of competing ideologies is that there are no solutions, just incommensurable ideologies and you gotta sort of duke it out, and the strong man/woman/group wins (see Terry Eagleton for this characterization of Fish’s work. But the ending to this piece is subtly different, a tone and stance I rather prefer: So again, what to do? Lilla s answer is pragmatic rather than philosophical (and all the better for that). All we can do, he says, is cope ; that is, employ a succession of ad hoc, provisional strategies that take advantage of, and try to extend, moments of perceived mutual self-interest and practical accommodation. We need to recognize that coping is the order of the day, not defending high principles. Now there s a principle we can live with, maybe. What I like about his ending is that it acknowledges there are times when compromise is possible, where a common meeting ground can be forged, however provisional these may be. As someone interested in the politics of consensus and accommodation, I think it is important to recognize that human beings are not simply molded by one set of values but are are often dwelling within various systems (of sometimes contradictory) values. And it is because of this multiplicity that forms of accommodation and consensus emerge and can emerge, signaling a more hopeful politics that derive not from abstract adherence to precept such as tolerance, but from the far messier realm of actual life experience.

To Blog, or Not To Blog...

Amazon: in the hands of a few

Thinking about writing something for a while, about my Debian work or about my daily stuff, sometimes you lack the time or the motivation to do so, I should kept this more up-to-date so people can follow a little bit what's going on with me... specially considering that I try to keep this diary neutral about non-technical subjects, but hey! it is somewhat hard to be neutral in every single situation. Anyway, I found that I should help to spread the word from a Brazilian colleague, Sulamita, she posted about democracy (or the lack of it) today. Two videos, about the Amazon Forest area and some of the crazy things we find in Brazil. You can watch the Brazilian Portuguese version or the English one.

A lot of people are not aware of some of the political problems that Brazil faces in the last 10 years, it is not like it doesn't exist before, but it is getting worst and worst and, for a while, I've been wondering...

A lot of friends want to leave Brazil as soon as they can, I love Brazil but I can understand their motivation.


• Where do you find the desire to stay when you look around and you can't find fairness and equality? And if you look to the houses or places that should protect people, you realize that there is no protection?
• How do you keep the passion for a country with huge levels of corruption in public government, in police forces, in your law makers? And day after day, corruption schemas, robbery and law abuse are noticed on the television?
• What to do when you realize that the perception of the disloyal advantage is shared among your colleagues and that the feeling of failure dominates a great part of the population that could actually do something to change?
• Can anybody, or group of individuals, feeling alone, work to change that? How? Specially when you look around and although you know there are people willing to help they need to take care of their lives (money, food, family)?

Or should I ask: Where's the love? (Black Eyed Peas Lyrics).

And before somebody thinks that the entire Brazil is lost, that we are under a Civil War and that monkeys and other animals are walking on our Jungle streets in the middle of the cities, no it is not... yes, we need help; yes people need to join forces; yes, we need to find ourselves and fix it; but it is still a great country to live, and yes, I love it!

Thanks: beraldo and sysdebug. And I'm happy to discover that advogato upgraded my status, I'm now a Journeyer.

I spend a few hours today trying to get Scalix to work right...I managed to be able to loginto the admin console at the end, yay!Still: it seems that it's pretty much unusable except by Outlook (maybe, I havn't tried that).The reason is simple: they don't let me log into the web mail. With none of my browsers. Because they havn't understood what 'Gecko' is.For tech people (apparently the Scalix people don't fall into this category), Gecko is the 'rendering engine' inside Firefox, Mozilla and a few others.With 'others' including the trademark-unencumbered Iceweasel fork of Firefox, or the Gnome webbrowsers "Epiphany" and "Galeon". Which are not "supported" by Scalix (do you happen to actually know what you are supporting? Did you develop for the Gecko rendering engine or for the Firefox UI?).Other people in the "I'm a web 2.0 company, but I don't know my stuff" hall-of-shame include the huge german email provider GMX. Their new webmail 'preview' is also only enabled if you claim to be using Firefox or Internet Exploder. They neither have understood why all Mozilla browsers include the Gecko id...

Heads up: emacs-snapshot 20070902-1 has the multi-tty code, as it was merged in the CVS trunk last week. It's probably broken in many ways, so don't upgrade (unless you want to).

This upload also removes the Lucid version, as upstream chose GTK+ as the default toolkit. This probably causes even more breakage. You've been warned.

I have just given my parents a new computer, and part of the upgrade process lost their email passwords (which were stored in KDE preferences - this seems to happen every time KDE is upgraded). The only password that is not under my control is the password for their Optus account. So I had to get the password changed, I got my mother to phone Optus but they decided that my father had to make the request (not that I would have had any difficulty impersonating him, any of the details that they asked for which I might not have known could have been provided by my mother). So my father requested that my mother be listed as someone who is authorised to make changes to the account. It turned out that changing a password for a mailbox is a difficult operation and I needed to talk to the phone-support guy. He needed to get permission from my father for this (he didn’t seem to realise that he had just granted my mother full access so that she could authorise such things on future calls). After my father had given permission for the second time I got the password changed, the new password was “changeme“. The call-centre guy advised me to change it as “it’s a very common password“! An ISP with any clue in their call centre would get the password changed to a semi-random string. Then there was the process of changing the password. The web site didn’t work at all with Konqueror (the call-centre guy told me that only IE is supported). I used Iceweasel (Firefox) and it allowed me to change the password. The Optus web site is one of the worst I’ve ever seen, and redirecting to the MS site live.com as part of the process was only part of it. Another bad thing that Optus does is punish customers who do things that they don’t like. My original Optus contract specified unlimited uploads, the first time they cut me off for actually uploading a moderate amount of data (running Bittorrent) they disconnected me and claimed that it was a DOS attack. On a later occasion they disconnected me on a Wednesday because I had uploaded some files on the weekend. They alleged that the upload was continuing on the Sunday (although I had a very clear recollection of ending it on the Saturday), but we all agreed that the upload had ceased before the Monday. But apparently I needed to be disconnected on the Wednesday to teach me a lesson. The lesson of course is that Optus sucks really badly. They do have reasonable prices and at the time their cable network offered download speeds significantly higher than any ADSL plans could match, which is why I have continued using them (and subscribed again after buying a new house). Recently ADSL2+ has started to become common and prices have been falling. For my use and for most people who I know ADSL is now either cheaper than cable or close enough that it’s worth paying the extra to escape Optus. My parents are on an Optus plan that gives them 100MB per month of data transfer (which counts both download and upload) before they are limited to modem speed. This is enough for them, but I expect that when their Optus contract runs out there will be an ADSL plan that’s not much more expensive and they can cease using Optus too. I just wish there was a real choice of providers for the base phone service. Currently there is only Telstra and Optus, both of which are expensive and suck. Fortunately Telstra has an insane CEO who is determined to make sure that everyone in Australia learns that Telstra is a nasty monopoly that needs to be broken up or severely constrained.

Share This

One item on my todo list is to set up a bunch of email addresses on sub-domains of domains that I am responsible for (with the consent of all people involved of course) and perform various actions to get the addresses noticed by spammers and measure how effective the various anti-spam measures are. As part of such tests I would click on every URL in every message sent to some accounts and see what difference it makes. My plan is to run a set of Xen virtual machines with different configurations of some common anti-spam measures used in MTAs and see how they fare with sets of accounts with similar publicity. I am not aware of any work having been done in this area (a quick Google search turned up nothing). There are many honeypots for tracking spam sources, matching email address harvesting to spamming, etc. But I’m not aware of any research into the effectiveness of various methods of combatting spam by setting up multiple honeypots. Please inform me via comments if I have missed something! The most common advice about spam is to NEVER click on the URL that supposedly removes you from a list. By clicking on such a URL the spammer can recognise that you actually read the email and therefore know that it’s a live address and a good target for more spam. I am not aware of any good studies proving this, which is why it’s one of the things I’d like to investigate. A counter theory (for which there is also a lack of evidence AFAIK) is that spammers used to measure delivery etc but now that bot-nets are large and cheap it’s easier to just send mail to all possible addresses. Even though I am not aware of any great evidence to support the idea I avoid clicking on URLs in spam messages. Refraining from hitting the spam web-sites can’t do any harm (it’s not as if the meager contribution to their system load caused by my web browser will cause them a problem). But today I was tricked. A spammer subscribed me to a mailman mailing list, as I am subscribed to many lists (about half of which use mailman) the fact that I didn’t recognise the list name didn’t necessarily mean that I hadn’t signed up to it. After signing in I saw the list archives which had only one post concerning spam. I unsubscribed (there was no other reasonable option open to me) and sent the mailman message to SpamCop. This technique will probably be effective for a while. People will think that they subscribes to a list and forgot about it and that it’s just another list that doesn’t have strong anti-spam measures. That should greatly increase the amount of time taken to black-list the spam server. So from now on if I receive a spam via a mailing list that I am not familiar with then I’ll send it to SpamCop immediately. Also this is yet another good reason for not subscribing people to mailing lists without their consent (a practice that is far too common - it’s really not difficult to send someone an email asking whether they would like to join the list). If you subscribe me to a list without prior discussion and the first post I receive on the list is a spam then it will be sent to SpamCop and this might result in you being black-listed.

Share This

Some good news after all After removing all remaining license issues regarding missing copyright statements or license statements in some files virtualbox is finally in debian unstable. Since I needed the statements in these files and cause of some bugfixes debian ships now an svn version. So this big and important piece of software is now finally official available. To integrate virtualbox in debian I had to remove some sourcecode and therefore virtualbox doesn't contain the Guest Additions (they contain non-free code from e.g Microsoft). The good news here are that in the next version of virtualbox these Guest Additions will be available from the website and can be downloaded automatically with virtualbox. Enjoy it! Thanks to feivel, who sponsored me