A significant problem with the old-fashioned media is that as a general rule they don’t cite references for anything. Some of the better TV documentaries and non-fiction books cite references, but this is the exception not the norm. Often documentaries only cite references in DVD extras which are good for the people who like the documentary enough to buy it but not for people who want to rebut it (few people will pay for a resource if they doubt the truth and accuracy of it’s claims). I can understand newspapers not wanting to publish much in the way of background information in the paper version as every extra line of text in an article is a line of advertising that they can’t sell. So they have financial pressure to produce less content, and the number of people like me who want to check the facts and figures used in articles is probably a small portion of the readership. Another issue with newspapers is that they are often considered as primary authoritative sources (by themselves and by the readers). It is often the case that journalists will interview people who have first-hand knowledge of an issue and the resulting article will be authoritative and a primary source in which case all they need to do is to note that they interviewed the subject. However the majority of articles published will be sourced from elsewhere (news agencies [ http://en.wikipedia.org/wiki/News_agency ] such as Reuters are commonly used). Also articles will often be written based on press releases - it is very interesting to read press releases and see how little work is done by some media outlets to convert them to articles, through a well written press release a corporation or interest group can almost write it’s own articles for publication in the old media. One way of partially addressing the problem of citing references in old media would be to create a web site of references, then every article could have a URL that is a permanent link to the references and calculations to support the claims and numbers used. Such a URL could be produced by any blogging software, and a blog would be an ideal way of doing this. For bloggers however it’s much easier to cite references and readers have much higher expectations of links to other sites to support claims and of mathematical calculations shown to indicate how numbers are determined. But there is still room for improvement. Here are some of the most common mistakes that I see in posts by people who are trying to do the right thing:

1. Indirect links. When you refer to a site you want to refer to it directly. In email (which is generally considered a transient medium) a service such as TinyURL [ www.TinyURL.com ] can be used to create short URLs to refer to pages that have long URLs. This is really good for email as there are occasions when people will want to write the address down and type it in to another computer. For blogging you should assume that your reader has access to browse the web (which is the case most of the time). Another possibility is to have the textual description of a link include a reference to the TinyURL service but to have the HREF refer to the real address. Any service on the net may potentially go away at some future time. Any service on the net may have transient outages, and any reader of your blog may have routing problems that make parts of the net unavailable to them. If accessing a reference requires using TinyURL (or a similar service) as well as the target site then there are two potential things that might break and prevent your readers from accessing it.
   One situation where indirect links are acceptable is for the printed version. So you could have a link in the HTML code for readers to click on to get to the reference page directly and a TinuURL link for people who have a printed version and need to type it in.
   Also when linking to a blog it’s worth considering the fact that a track-back won’t work via TinyURL and track-backs may help you get more readers…


2. Links that expire. For example never say “there’s a good article on the front page of X” (where X is a blog or news site). Instead say “here’s a link to a good article which happens to be on the front page now” so that someone who reads your post in a couple of years time can see the article that you reference.
   Another problem is links to transient data. For example if you want to comment on the features of a 2007 model car you should try to avoid linking to the car manufacturer page, next year they will release a new car and delete the old data from their site.
   A potential problem related to this is the Google cache pages which translate PDF to HTML and high-light relevant terms and can make it much easier to extract certain information from web pages. It can provide value to readers to use such links but AFAIK there is no guarantee that they will remain forever. I suggest that if you use them you should also provide the authoritative link so that if the Google link breaks at some future time then the reader will still be able to access the data.


3. Not giving the URLs of links in human readable form. Print-outs of blog pages will lose links and blog reading by email will also generally lose links (although it would be possible to preserve them). This counts for a small part of your readership but there’s no reason not to support their needs by also including links as text (either in the body or at the end of the post). I suggest including the URL in brackets, the most important thing is that no non-URL text touch the ends of the URL (don’t have it in quotes and have the brackets spaced from it). Email clients can generally launch a web browser if the URL is clear. Note that prior to writing this post I have done badly in this regard, while thinking about the best advice for others I realised that my own blogging needed some improvement.
   I am not certain that the practice I am testing in this post of citing URLs inline will work. Let me know what you think via comments, I may change to numbering the citations and providing a list of links in the footer.


4. Non-specific links. For example saying “Russell Coker wrote a good post about the SE Linux” and referring to my main blog URL is not very helpful to your readers as I have written many posts on that topic and plan to write many more (and there is a chance that some of my future posts on that topic may not meet your criteria of being “good”). Saying “here is a link to a good post by Russell Coker, his main blog URL is here” is more useful, it gives both the specific link (indicating which post you were referring to) and the general information (for people who aren’t able to find it themselves, for the case of deleted/renamed posts, and for Google). The ideal form would be “<A HREF=”http://etbe.coker.com.au/whatever”>here is a link to a good post by Russell Coker [ http://etbe.coker.com.au/whatever ]</A>, his main blog URL is <A HREF=”http://etbe.coker.com.au/”> [ http://etbe.coker.com.au ]</A>” (note that this is an example of HTML code as a guide for people who are writing their own HTML, people who use so-called WYSIWYG editors will need to do something different).


5. Links that are likely to expire. As a rule of thumb if a link is not human readable then the chance of it remaining long-term is low. Companies with content management systems are notorious for breaking links.


6. Referencing data that you can’t find. If you use data sourced from a web site and the site owner takes it down then you may be left with no evidence to support your assertions. If data is likely to be removed then you should keep a private copy off-line (online might be an infringement of copyright) for future reference. It won’t let you publish the original data but will at least let you discuss it with readers.


7. Referencing non-public data. The Open Access movement [ http://en.wikipedia.org/wiki/Open_access ] aims to make scholarly material free for unrestricted access. If you cite papers that are not open access then you deny your readers the ability to verify your claims and also encourage the companies that deny access to research papers.
   An insidious problem is with web sites such as the New York Times [ www.nytimes.com ] which need a login and store cookies. As I have logged in to their site at some time in the past I get immediate access to all their articles. But if I reference them in a blog post many readers will be forced to register (some readers will object to this). With the NYT this isn’t such a problem as it’s free to register so anyone who is really interested can do so (with a fake name if they wish). But I still have to keep thinking about the readers for such sites.
   I should probably preview my blog posts from a different account without such cookies.


8. Failing to provide calculations. My current procedure is to include the maths in my post, for example if you have a 32bit data type used to store a number of milliseconds then it can store 2^32/1000 seconds which is 2^32/1000/60/60/24 = 49.7 days, in this example you can determine with little guessing what each of the numbers represent. For more complex calculations an appendix could be used. A common feature of blogs is the ability to have a partial post sent to the RSS feed and the user has the ability to determine where the post gets cut. So you could cut the post before the calculations, the people who want to see them will find it’s only one click away, and the people who are happy to trust you will have a shorter post.


9. Linking with little reason. Having a random word appear highlighted with an underline in a blog post is often not very helpful for a reader. It sometimes works for Wikipedia links where you expect that most readers will know what the word means but you want to link to a reference for the few who don’t (my link for the word Wikipedia is an example). In the case where most readers are expected to know what you are referring to then citing the link fully (with a description of the link and a human-readable form for an email client) is overkill and reduces the readability of the text.
   The blogging style of “see here and here for examples” does not work via email and does not explain why a reader should visit the sites. If you want to include random links in a post then having a section at the footer of related links would probably be best.


10. Linking to a URL as received. Many bloggers paste URLs from Google, email, and RSS feeds into their blog posts. This is a bad idea because it might miss redirection to a different site. If a Google search or an email gives you a URL that is about to go away then it might redirect to a different site. In that case citing the new URL instead of the old one is a service to your readers and will decrease the number of dead-links in your blog over the long-term. Also using services such as www.feedburner.com may cause redirects that you want to avoid when citing a blog post, see my previous post about Feedburner [ http://etbe.coker.com.au/2007/08/20/feedburner-item-link-clicks/ ].

Here are some less common problems in citing posts:

1. Inappropriately citing yourself. Obviously if there is a topic that you frequently blog about then there will be benefit to linking to old posts instead of covering all the background material, and as long as you don’t go overboard there should not be any problems (links to your own blog are assumed to have the same author so there is no need for a disclaimer). If you write authoritative content on a topic that is published elsewhere then you will probably want to blog about it (and your readers will be interested). But you must mention your involvement to avoid giving the impression that you are trying to mislead anyone. This is particularly important if you are part of a group that prepares a document, your name may not end up on the list of authors but you have a duty to your readers to declare this.
   Any document that you helped prepare can not be used by itself as a support of claims that you make in a blog post. You can certainly say “I have previously demonstrated how to solve this problem, see the following reference”. But links with comments such as “here is an example of why X is true” are generally interpreted to be partly to demonstrate the popular support for an idea.


2. Citing secret data. The argument “if you knew what I know then you would agree with me” usually won’t be accepted well. There are of course various levels of secrecy that are appropriate. For example offering career advice without providing details of how much money you have earned (evidence of one aspect of career success) is acceptable as the readers understand the desire for some degree of financial secrecy (and of course in any game a coach doesn’t need to be a good player). Arguing the case for a war based on secret data (as many bloggers did) is not acceptable (IMHO), neither is arguing the case for the use of a technology without explaining the science or maths behind it.


3. Not reading the context of a source. For example I was reading the blog of a well regarded expert in an area of computer science, and he linked to another blog to support one of his claims. I read the blog in question (more than just the post he cited) and found some content that could be considered to be racially offensive and much of the material that I read contained claims that were not adequately supported by facts or logic. I find it difficult to believe that the expert in question (for whom I have a great deal of respect) even casually inspected the site in question. In future I will pay less attention to his posts because of this. I expect a blogger to pay more attention to the quality of their links than I do as a reader of their blog.

While writing this post I realised that my own blogging can be improved in this regard. Many of my older posts don’t adequately cite references. If you believe that any of my future posts fail in this regard then please let me know.

Share This

Got back to the appartment last night to find police on High School Way, interviewing somebody innocuous-looking. When I went to the door to the appartment complex, it turned out that there were upwards of a dozen police, with six patrol cars, on Castro St. Most were standing around doing nothing but one of them appeared to have stopped a car and was interviewing the driver. I approached a chap at the edge of a crowd of onlookers and asked what was going on, and he declined to comment. At about that moment, the helicopter flew past for the second time. Just as Pete and I were about to go, somebody cycled along the and stopped to cross Casto, halting in the middle of a gathering of police officers. One told him, "It’s illegal to cycle on the sidewalk, sir." His reply was a work of genius: "Oh, really?" It was all very bizarre, though not at all menacing or confrontational. No guns were drawn and nobody was incapacitated by tazer or baton. Just another data point. Then we went up to the appartment proper and watched Hot Fuzz. Excellent film, especially given that the room was still being illuminated by the occasional strobe of blue lights.

I came across John Wang's posting titled Database Abstraction - code vs infrastructure. In it, he talks about the problems many people face when migrating applications tied to a specific RDBMS to another one. He recommends:

One solution of modern programming is to move database abstraction from the code to the infrastructure using a ORM (Object-Relational Mapper) or Data Mapper. A ORM and Data Mapper abstracts the database for you so you no longer have to do tie db abstraction to each app. Not only does it let you code once for multiple databases it lets your users migrate their data from one database to another. This blog runs Typo which is based on Ruby on Rails and ActiveRecord. I've been contemplating migrating Typo from MySQL to PostgreSQL and I've been told that it would be as simple as exporting the data with YAML, updating the database.yml file and importing the data.

Umh... Thing is, you often want to use your DB as way more than a data store - At least, when using Postgres, I do. There are many things that can be done inside the DB (such as guarding the programmer against his own mistakes by keeping sanity checks which often go beyond just referential integrity. Constraints and triggers. Oh, and of course, user-defined functions and views. Of course, most times you can go by without using several of such tools - but when they would come in handy, you can sorely miss them.
On the other hand, John mentions (right after the previous paragraph):

ActiveRecord is a data mapper and isn't as flexible as a full blown ORM but it gets the job done for the most part. For a full-blown ORM, I think of Perl's DBIx::Class which provides a full OO interface to the RDBMS allowing you to code just once for multiple DBs without limiting you when you want to use some esoteric database-specific SQL

Currently, I'm among the many that have jumped on the Rails bandwagon - And so far, I really like it. But yes, ActiveRecord -although great for some tasks- is way under what I'd like for many actions I've got used to making.
I might be seen as a backwards guy for this, but until before I started with Rails, I prefered not to use full ORMs (not that I tried too hard anyway - Probably starting a project with DBIx::Class would be all it takes for me to become a convert, from what I've read), but to do the SQL myself - Only, of course, keeping well in mind that I should separate front-end from back-end (or using other jargon to say the same, separating cleanly what should be in a Controller and what should be in a Model - Of course, no one in their right mind would put SQL on a template or a View!). It's not so hard as many people seem to think, although mixing syntaxes for two different languages (and even more if you, as I do, like to keep your code under 80 columns) is sometimes dirt-ugly. Still, on to my last bit of rant:

There are PHP frameworks out there like Symfony and Cake but do any of them have stand-alone ORMs? If so, could Drupal move to something like that and solve their maintainership problems once and for all? Drupal is part of the Go PHP5 effort so there should be no issue using PHP 5 OO. Something to think about for the Drupal folks if a PHP ORM is available

Umh... Even if there are some great ORMs in PHP... Choosing an ORM is something you do when you start a project. I doubt the Drupal people can now just decide to move over to using a ORM, as they potentially have hundreds or thousands of points where they have DB interaction. And although for most of them moving over to an ORM should even be automatizable, every ORM has some very special features which make it damn hard to craft several kinds of query.

Providing a lot of information in a bug report is useful. It helps the developer work out what's wrong. However, you should be careful not to reveal too much information as one submitter did to Gnome in their bug report. Have a look (if you're not in work) at the .xsession-errors at the end of their report to see what movies they were playing.

Update: mind you, at least they were vaguely covert, compared to the brutal honesty of this report. Thanks to Florent Bayle for that link.

I’ve found a reasonably good free spider for checking for dead links, www.dead-links.com. When I told it to spider my main site www.coker.com.au it worked well and informed me of several dead links that I fixed. When I told it to spider my blog it reported an error 404 and didn’t give any useful output. Strangely it was able to display a thumbnail picture of my main blog page that correctly had all data. I was wondering if it didn’t like Wordpress so I tested it out on Chris Samuel’s blog [ www.csamuel.org ]. It seemed to have no problems with his blog, so I suspected that it was the request time. I reduced the number of blog posts shown on the front page of my blog to 1 to speed things up (which resulted in my page loading faster than his) but still the dead-links.com spider didn’t like me. I now suspect that it’s something related to me using Wordpress 2.2. I would be interested in feedback from other people who try checking their blogs for dead links, whether they are using Wordpress 2.2 or something else. In any case it’s a useful service and I recommend using it.

Share This

This week has consisted of fighting registrars and doing a bit of hacking on xen-shell, xen-tools, and the chronicle blog compiler. CJ has done some good work trying to get the code modularised, and I expect between the pair of us we can make things neater and better generally. I've also fixed a couple of bugs relating to the hard-wiring of device names (/dev/sda, /dev/tty1, etc). These devices are replaced in newer versions of Xen which wants to use /dev/xvc0 and /dev/xvd[a-z] instead. There's nothing else happening at the moment; I'm just having a lot of fun laughing at our new kitten sliding around on our polished wooden floor! Chronicle seems to be getting pretty popular which is ironic because it was a quick hack to allow me to post blog entries on a couple of hosted sites - which I've not yet done. Oops. In other news I'm loving the Nintendo DS at the moment, Megan brought me one back from America on her recent trip and I think a day hasn't passed where one or both of us has played less than 30 minutes each. I'm annoyed that Sim City DS only allows us to play with one city - right now it is her turn, and I'll have to wait until she's finished with her creation before I can have a go - because otherwise I'll wipe her city out.. :(

The latest version of d2x-xl works quite well, so I've started packaging it. Since it needs the data from the commercial game to work, it will probably end up in contrib. The interesting thing is that there is another project, Into Cerberon, working on a Doom 3 mod with free game data -- it would be interesting to see whether both projects could be combined in order to get a completely free game. Difficulty: Into Cerberon is planning for a triangle based engine while d2x-xl thinks in cubes. Update: Into Cerberon use the Doom 3, not the Quake 3 engine. Thanks "zulu9" for noticing. Also, Josh alerts me of some interesting (cough) licensing issues around this code. As I'm still busy wading through the uclibc source compiling a proper list of attributions and licenses (ftp-master thinks a blanket statement "If you link against this, the LGPL applies to you, if you look at the files, please see each individual file" is not enough -- I can somehow see their point, but it takes ages to do that) that will have somewhat lower priority. I know why I rewrite things from scratch rather than fix them these days...

Peter De Wachter and I have been packaging most of Kenta Cho’s games in the latest weeks. As it might take a while for them to reach Debian repositories, mostly because they’ll have to wait for libbulletml to go through NEW queue, and afterwards they’ll have to go through the NEW queue themselves, I thought it might make sense to have them available online. Some of them will need libbulletml, you might as well download it too if you’re interested. So far, the following games are available:

• Titanion
• Mu-cade
• Gunroar
• Torus Trooper
• TUMIKI Fighters
• PARSEC47
• A7Xpg
• rRootage
• Wok

I hope to finish Val & Rick soon too. We probably won’t package Suzuri and Masashikun Hi!, as they’d probably be quite difficult to port to Linux. Noiz2sa is already packaged in Debian (well, so is rRootage, but I will upload the newer version soon, I just don’t want to harass my sponsors too much, so please be patient). The games can be temporarily downloaded here, and libbulletml here. libbulletml has been coded in C++ by shinichiro.h, while BulletML design is Kenta Cho’s. You also have packages available for 360 degree barrages (bulletss) and White barrage (sdmkun) by shinichiro.h. The former is a small demo for libbulletml, while the latter lets you see and experiment the power of the language.

If you're going to build a computer case that looks like a media player ...

and if you're going to include a big volume control and an LCD player on the front ...

then aren't some buttons like "play", "next", and "pause" also a good idea?

I ask because I was browsing computer cases at the store yesterday, and apparently no-one working for the companies that design these things has thought of including some control buttons on front. To me, this seems like it would rank somewhere in the top 10 most obvious ideas ever, somewhere below "eat when you get hungry" and above "don't pick a fight with a grizzly bear", but clearly I'm missing something.

I previously posted about Interesting Ideas from George Monbiot, one of which was to establish individual emissions trading. Gyros Geier disagrees with this and cites the current emission trading schemes as evidence. There are several fundamental differences between George’s idea and the current implementations of emission trading. The biggest flaw in current emission trading schemes is that the emission credits are assigned to the worst polluters. George is proposing that an equal amount be assigned to all citizens. Assigning credits to the worst polluters is another form of by the polluting industries. The way to solve these problems through emission trading is to start by fairly assigning the credits (and what better way than to equally distribute them among all citizens) and to then reduce the amounts assigned over time. Gyros claims that emission trading which allows people who use little emissions to get large credits will cause people to have resources used in their name which they would not otherwise use. The solution to this is to assign to each citizen in a country a set of credits that is equal to the use by someone on the median income. Note specifically that setting credits equal to average use is not the right thing to do, the vast majority of the population produce significantly less emissions than average. The result of such a policy would be that people who produce median emissions (and most of whom would be close to the median income) would reduce their emissions as much as possible so that they could sell the credits, they would even have an incentive to spend money to reduce their emissions (for example by installing better insulation in their home) as it would be an investment. Then people who produce more emissions than the median would be forced to buy credits to support their extravagant lifestyle. This would give a significant reduction in emissions (the median income is about half the average income and I presume that the emissions produced are in line with income). Gyros also makes the startling claim that emissions trading increases emissions. I can’t imagine that being possible, in fact I can’t imagine how the coal industry could do more damage to the environment if they tried. Finally, taking a positive approach to blogging is a really good idea. I welcome discussion with people who want to claim that my ideas (and the ideas that I quote) are bad, but if you are going to do this please describe something that you consider to be better.

Share This

In the last week, I have been witness to and part of many conversations and probably one of my favorite ones was about coffee. My friend reasoned that coffee is as wonderful as it is because of its dependability (unlike, for example, your relatives). You know that for a moderate sum of money, you can drink a drink that makes you happy, alert, and, for some of us, allows us to face the rest of the day on an even keel. It is pure comfort that derives from a form of almost ritualistic dependability. This morning, as I was sipping my coffee, I came across a short blog post by Stanley Fish who certainly does not make me as happy as my morning cup of joe, but I do admire him for his dependability and consistency when it comes to reporting on matters of liberalism. For over 20 years he has dependably written on the quandaries and limit of liberal political ideology and his most recent installment, which focuses primarily on a new book by Paul Starr, is no different. Well, his conclusion strikes just a little differently than the tone of some of his previous works. In the past (or perhaps in some of his longer academic works), Fish’s solution to the problem of competing ideologies is that there are no solutions, just incommensurable ideologies and you gotta sort of duke it out, and the strong man/woman/group wins (see Terry Eagleton for this characterization of Fish’s work. But the ending to this piece is subtly different, a tone and stance I rather prefer: So again, what to do? Lilla s answer is pragmatic rather than philosophical (and all the better for that). All we can do, he says, is cope ; that is, employ a succession of ad hoc, provisional strategies that take advantage of, and try to extend, moments of perceived mutual self-interest and practical accommodation. We need to recognize that coping is the order of the day, not defending high principles. Now there s a principle we can live with, maybe. What I like about his ending is that it acknowledges there are times when compromise is possible, where a common meeting ground can be forged, however provisional these may be. As someone interested in the politics of consensus and accommodation, I think it is important to recognize that human beings are not simply molded by one set of values but are are often dwelling within various systems (of sometimes contradictory) values. And it is because of this multiplicity that forms of accommodation and consensus emerge and can emerge, signaling a more hopeful politics that derive not from abstract adherence to precept such as tolerance, but from the far messier realm of actual life experience.

Well, checking ISO-3166-2 data for my country, I found out that France officially listed the following subdivisions in ISO-3166-2:

• 22 metropolitan regions
• 4 overseas regions/departments
• 96 metropolitan departments
• 2 territorial collectivities
• 4 overseas territories

Thankfully, they didn't list the hundreds of "cantons" and "arrondissements", nor the 36,000 municipalities. Ah, and France is also one of the very few countries that have *more than one* ISO-3166 code as, besides the logical FR, we have GP for Guadeloupe, GF for Guyane, MQ for Martinique, RE for R union, YT for Mayotte, PM for Saint-Pierre-et-Miquelon, NC for New Caledonia, PF for French Polynesia, TF for the Southern French Territories and WF for Wallis et Futuna. Dammit..:-) Even our British friends didn't steal codes for Scotland or Wales...

Xen support finally landed in upstream Linux. Okay, it is rather limited yet, but usable. It supports the following:

• Unprivileged domain (domU).
• x86_32 (i386 without PAE) and x86_32p (i386 with PAE).
• Console.
• netfront and blkfront.

Changes to the old Xen patch:

• Block devices does not support takeover of hdXY and sdXY. Use xvcX (xvca, xvcb, ...) as device names.
• Console is hvc0. You must supply console=hvc0 at the command line. (Use the extra definition in the Xen domain config file.)

Yet missing things:

• Suspend, resume (also makes migration impossible).
• Ballooning.
• x86_64. Xen upstream currently waits for an unified x86 tree.
• Privileged domain (dom0).
• netback, blkback, pciback and pcifront.

I’ve been having problems with one of my Xen virtual servers crashing with kernel error messages regarding OOM conditions. One thing I had been meaning to do is to determine how to make a core dump of a Xen domain and then get data such as the process list from it. But tonight I ended up catching the machine after the problem occurred but before the kernel gave OOM messages so I could log in to fix things. I discovered the following issues:

1. 10+ instances of spf-policy.pl (a Postfix program to check the Sender Policy Framework data for a message that is being received), most in D state.


2. All my Planet news feeds being updated simultaneously (four of them taking 20M each takes a bite out of 256M for a virtual machine).


3. Over 100 Apache processes running in D state.

I think that there is a bug with having so many instances spf-policy.pl, I’ve also been seeing warning messages from Postfix about timeouts when running it. For the Planet feeds I changed my cron jobs to space them out. Now unless one job takes 40 minutes to run there will be no chance of having them all run at the same time. For Apache I changed the maximum number of processes from 150 to 40 and changed the maximum number of requests that a client may satisfy to 100 (it used to be a lot higher). If more than 40 requests come in at the same time then the excess ones will wait in the TCP connection backlog (of 511 entries) until a worker process is ready to service the request. While keeping the connections waiting is not always ideal, it’s better than killing the entire machine! Finally I installed my memlockd program so that next time I have paging problems the process of logging in and fixing them will be a little faster. Memlockd locks a specified set of files into RAM so that they won’t be paged out when memory runs low. This can make a dramatic difference to the time taken to login to a system that is paging heavily. It also can run ldd on executables to discover the shared objects that they need so it can lock them too.

Share This

debconf7 flashback Somehow I didn't find the time to blog during the last weeks, so there's a lot of stuff on my blog todo list. But let's start with a little flashback , back to Debconf7 in Edinburgh. Yesterday I got a film back from the lab with photos I've taken during the daytrip-day. As a few of you probably realized I was not on the train/boat to the island in the west, mainly because Andreas (afit), Julian (codemonk) and me decided to spend the day in Ediburgh, not only for some geocaching fun but also to see a bit more of this beautiful town and area. We've also been up on Arthur's Seat, Edinburgh, where I took this panorama: It will be published on placesofinterest.info, which is currently in public beta state, but definitely worth a look (just don't use IE, but I doubt you use non-free software, right?). I'll blog more about placesofinterest.info as soon as the page is finished. As usual, the pano was created with the help of The Gimp, hugin and enblend. The only commercial software involved was the software for the Imacon scanner at university. Original resolution: 24044×2551 pixel, stitched from 14 slides. If anybody is interested - large format prints for Debconf participants are available for the cost of paper and ink.

One item on my todo list is to set up a bunch of email addresses on sub-domains of domains that I am responsible for (with the consent of all people involved of course) and perform various actions to get the addresses noticed by spammers and measure how effective the various anti-spam measures are. As part of such tests I would click on every URL in every message sent to some accounts and see what difference it makes. My plan is to run a set of Xen virtual machines with different configurations of some common anti-spam measures used in MTAs and see how they fare with sets of accounts with similar publicity. I am not aware of any work having been done in this area (a quick Google search turned up nothing). There are many honeypots for tracking spam sources, matching email address harvesting to spamming, etc. But I’m not aware of any research into the effectiveness of various methods of combatting spam by setting up multiple honeypots. Please inform me via comments if I have missed something! The most common advice about spam is to NEVER click on the URL that supposedly removes you from a list. By clicking on such a URL the spammer can recognise that you actually read the email and therefore know that it’s a live address and a good target for more spam. I am not aware of any good studies proving this, which is why it’s one of the things I’d like to investigate. A counter theory (for which there is also a lack of evidence AFAIK) is that spammers used to measure delivery etc but now that bot-nets are large and cheap it’s easier to just send mail to all possible addresses. Even though I am not aware of any great evidence to support the idea I avoid clicking on URLs in spam messages. Refraining from hitting the spam web-sites can’t do any harm (it’s not as if the meager contribution to their system load caused by my web browser will cause them a problem). But today I was tricked. A spammer subscribed me to a mailman mailing list, as I am subscribed to many lists (about half of which use mailman) the fact that I didn’t recognise the list name didn’t necessarily mean that I hadn’t signed up to it. After signing in I saw the list archives which had only one post concerning spam. I unsubscribed (there was no other reasonable option open to me) and sent the mailman message to SpamCop. This technique will probably be effective for a while. People will think that they subscribes to a list and forgot about it and that it’s just another list that doesn’t have strong anti-spam measures. That should greatly increase the amount of time taken to black-list the spam server. So from now on if I receive a spam via a mailing list that I am not familiar with then I’ll send it to SpamCop immediately. Also this is yet another good reason for not subscribing people to mailing lists without their consent (a practice that is far too common - it’s really not difficult to send someone an email asking whether they would like to join the list). If you subscribe me to a list without prior discussion and the first post I receive on the list is a spam then it will be sent to SpamCop and this might result in you being black-listed.

Share This

Many people will have seen their postfix reporting new errors these days, like if is overloaded:

postfix/master[7287]: warning: service "smtp" (25) has reached its process limit "100": new clients may experience noticeable delays

This is not really because your system is overloaded. To me, it looks more like a (deliberate?) malfunction of some spammers software. When you rejected it's spam mail a few times, it starts opening up lots of connections to your system, but never sends any data. They just sit there and time out.The solution is simple: limit the number of connections per client. Make sure the 'anvil' service is enabled (default at least on Debian) and then add

smtpd_client_connection_count_limit = 10

to your postfix configuration.You should now begin to see messages such as

postfix/smtpd[21680]: warning: Connection concurrency limit exceeded: 11 from unknown[210.210.247.52] for service smtp

A corean cable IP. Yeah, I don't think this would have been legitimate mail traffic anyway.Feel free to pick the limit value even lower than 10 on really busy systems (or if these 'attacks' become stronger) - most people should never need more than 2 parallel connections (note: you can send multiple emails over one connection, there isn't much to be gained by using more than one connection).

Eurosport are broadcasting the Vuelta a Espa a at 19 east from today with commentary in English (analogue only) and German. Scheduled times:-

Sat 01.09.2007

16:30-17:30 CET

Sun 02.09.2007

16:00-17:30 CET

Mon 03.09.2007 to Thu 06.09.2007

16:30-17:30 CET

Fri 07.09.2007 to Mon 10.09.2007

16:00-17:30 CET

Tue 11.09.2007

no show (MTB at 08:45)

Wed 12.09.2007

16:00-17:30 CET

Thu 13.09.2007

01:00-01:30 and 15:45-17:30 CET

Fri 14.09.2007 and Sat 15.09.2007

16:00-17:30 CET

TVEi are broadcasting at 13 and 19 east. Scheduled times:-

Sat 01.09.2007

16:40-18:00 CET

Sun 02.09.2007

16:40-18:30 CET

Mon 03.09.2007

16:35-18:00 CET

Tue 04.09.2007 to Sun 09.09.2007

16:40-18:00 CET

I'll post more times next week. I'd bet on a strong showing from Caisse d'Epargne after a relatively quiet TdF. Who would you tip? If you've not got your satellite set yet, some Lidl specials are still available and most Maplin branches have sets. Finding the broadcasts at 19 east is pretty easy in the UK. See my asteFAQ for hints and links. I've still not fixed my dish up properly, yet it gets most 19 east transmissions.

Russell thinks that trading emission "rights" individually would decrease emissions. The global emission market has shown quite the opposite happening, countries that have no industry and did not contribute to the problem now sell their rights to industrial countries, with the net effect being as if all these countries suddenly became industrialized over night. I don't think anything different would happen at an individual level. The fundamental problem is this: emission rights trading happens only when buying rights is cheaper than actually reducing your emissions. When this is relatively cheap, you remove any incentive to develop better processes in individual companies; if it is more expensive, most industries will remain at the status quo (no emission trading) while those where it still makes sense fiscally now have an alternative, which diminishes the market for technology companies specializing in filtering technology and process optimization (i.e. it becomes even more expensive to behave environmentally conscious). At the same time, the cost of buying these rights from individuals will be passed on to the consumer, the only way to get this money back will be to sell off their emission rights. From a macroeconomic perspective, no money ever changes hands, but you have effectively transferred the individual emission rights over to the industry. Looking at individuals, you are penalizing those that behave environmentally conscious and neither use their "rights" nor sell them. So, emission rights trading is a really bad idea.