Search Results: "spa"

10 September 2007

Steve Kemp: I should be so lucky

Recently the topic of spam on the Debian lists was revisited. I laugh at somebody who recieves 200 spam messages a day. Here's my stats for yesterday:
                                          Total Mails    : 6399
                                          Total SPAM     : 6077
                                          Total Accepted : 322
                                          Spam Percentage: 94.97%
That's 6077 mails rejected at SMTP time via my filters, and only 322 mails accepted. The breakdown of the spam rejected looks like this:
                                  Plugin      Count
--------------------------------------------------------------
                                   dnsbl       3755
                             hosts_allow        724
                             greylisting        661
                       check_earlytalker        303
                          check_spamhelo        238
             require_resolvable_fromhost        219
                           virus::clamav         79
                         check_badrcptto         75
                       check_badmailfrom         23
--------------------------------------------------------------

Dirk Eddelbuettel: MythTV data service switch

One of several things I never got around to blog about is my experience with MythTV (which, in case you don't know, turns your pc into a tivo, given a suitable tv decoder card and the mythtv software). To make up for the lack of a more detailed post, let's just recap: I had set it up a little over two years ago, on what was then a 'possibly temporary' Kubuntu box. Well the box is still there and still running Kubuntu, four releases later, and MythTV only got easier as Ubuntu now includes the decoder drivers. So far no worries. TV schedule information had always been provided via the 'free as in beer if you answer a question or two' zap2it subsidiary of the for-profit Tribune publishing company (i.e. the folks who own the Chicago Tribune, Cubs baseball team, WGN TV station around here and what have you, and are in the process of being sold to Sam Zeil). As had been noted in other places, that 'free' service is no more, and a seemingly cooperatively-run 'small fee' service has been set up by a few volunteers at schedulesdirect. So I just switched, and for the benefit of anybody sitting on the fence about this, the switch is trivial:
  1. If need be, add 'feisty-proposed' to apt's sources.list file to get the new MythTV 0.20.2 release directly from Ubuntu.
  2. Install MythTV 0.20.2 which takes as little as wajig update; wajig dist-upgrade
  3. Follow the instructions in this email, in particular point 4) about re-creating lineup info
  4. sit back and enjoy
Including the signup at schedulesdirect, the whole thing took maybe ten minutes. Not bad. Now I only have to find some spare time so that the backlog of unwatched episodes of the Daily Show does not grow much beyond six months... Oh, and last but not least, I'll have to pony up the fifteen bucks for the subscription once my trial is up next Sunday.

Bernd Zeimetz: winpdb - a platform independent Python debugger

winpdb - a platform independent Python debugger A few minutes ago a mail arrived which told me, that my winpdb package migrated to testing, which was just another reminder to blog about it. winpdb is not just yet another Python debugger, it is actually the first really well working one I've used so far. One of my favorite features is the embedded debugging, which allows you to debug scripts which are embedded somewhere. I still have to try it with a Zope product, though - I'm really wondering if I could embed the remote debugger in there. That just brought us to another very useful feature - winpdb is a remote debugger. It connects to the real debugger (called rpdb2) using an encrypted and password protected tcp connection, which allows you to run rpdb2 under a totally different user, probably on some embedded box somewhere on the other side of the world. If you don't want to use a graphical frontend, rpdb2 is able to run the debug sessions in a 'screen' window. Although winpdb doesn't ship much documentation, it is very well documented. And in case you don't like winpdb or wx - pida ships with a debugger plugin which is in fact a gtk frontend for rpdb2. The following quote is part of a mail from Nir Aides, the author of winpdb. Although it may be biased, I think it'll probably give you some ideas in case you want to compare winpdb with some other common (commercial) debuggers. He kindly allowed me to quote it here:
I have just spent a couple of hours (instead of doing some real work…) with the commercial Python debuggers of Komodo IDE 4.1 and Wing IDE 3 beta, and (without declaring objectiveness) Winpdb seems currently better than both in several important areas:

* It is the only one of the three to support os.fork() (wingide debugger crashes)

* It handles thread debugging best in many aspects (e.g allows inspection of threads that are still running, stays with the correct thread when stepping, general usability, etc…)

* Its name-space browser is the most consistent and stable (in particular with describing class inheritance, very large objects, sets!)

* It is the only one that prevents breakpoints in invalid lines (Komodo does not care at all and Wing allows setting breakpoints inside remarks and thus possibly in other invalid lines). This is important since in Python it is sometimes surprisingly hard to tell (by looking at the source code) which lines are valid for breakpoints.
Not to forget that winpdb supports debugging of curses based applications, namespace modification and the coming version is supposed to have full unicode support. Happy debugging!

9 September 2007

David Welton: Wiki Spammer

http://www.google.com/search?q=211.46.221.34 Yuck... look at all the damage one idiot with some scripts can do. I wish there were a way to make people like this pay.

8 September 2007

Wouter Verhelst: EMIRATES HERITAGE LOTTERY

Some spam was sent to a Debian list with that subject (sorry, no link, that would increase the pagerank). I guess spammers are now trying to combine two strategies in one: I have 1.000.000.000.000.000.000 US$ that I want to get off, but I can't possibly do so without getting into trouble. Therefore, I declare that you have won the LOTTERY!!!1! Now please claim your prize, and help me out of my problems. Thank you. Whether it'll work is another matter, of course.

Romain Francoise: Mario's secret power plant

.flickr-photo border: solid 2px #000000; .flickr-yourcomment .flickr-frame text-align: left; padding: 3px; .flickr-caption font-size: 0.8em; margin-top: 0px;

Mushrooms, originally uploaded by orebokech.

7 September 2007

MJ Ray: Look who's squatting on Jono Bacon

So, for some reason, I was using advogato and, for some reason, I was looking at jono and, for some reason, I clicked on the link to the JonoCam. What I saw was unpleasant and not what I was expecting. Someone's squatting on the JonoCam address, hiding behind eNom's privacy service. In fact, it looks like all of kdedevelopers.net is spam now, including jono.kdedevelopers.net. Not only that, but http://www.jonobacon.co.uk/ is squatted too, by futuregate aka Mindcom Internet Limited of 20 Meridian Way, Meridian Business Park, Norwich NR7 0TA. If you're reading this from near Norwich, would you post some pictures of the home of these serial domain-squatters, please? So I hopped onto the lugradio IRC and asked jono. Apparently he lost it ages ago(!) nominet says the domain was registered in 2004 and last updated April 2007, so I didn't know if that means it's been a while, or just a few months. Does the registration date change if an owner loses control? astinus told me that it does. Anyway, I wonder how many advogato users have disused person pages with spam/squatted links. Is there a nice way we could spot them all and tag them somehow? Is there some common "about me" format which advogato (and social networking sites?) could use to keep up-to-date on our movements? List three locations in case we lose our primary domain? Also, I had another email yesterday from someone apologising for losing control of their domain, asking me to change my links and I've removed a couple more from my friends bookmarks category recently. Are people still losing personal domains to squatters? If so, is there a new reason for this, or is it the same old problems of registrar-lock-in and it being cheaper to register a new domain than to go to dispute resolution? (2007-09-09: 4 comments)

Kartik Mistry: gnome-specimen

gnome-specimen * Sometime back, I came to know this little cool tool for previewing and comparing fonts. Did RFP–>ITP and then I came to know that it has one file gnome-specimen-about.svg under cc-by-sa license (Thanks to Torsten Werner for pointing out!). What to do?? I searched over packages and found that atleast one package is under similar license. So, I went ahead and submitted package (Was it brave step?). It took long time in NEW (due to license issue), but finally it got place. I contacted author (Wouter Bolsterlee) and told about this. A nice discussion followed. Tried Inkscape, no luck. And finally, 0.3.1 is with-out that cc-by-sa license!! You can get upstream tarball here. /me :P (Debian package will hit soon!)

Russell Coker: Duplicating a Xen DomU

A fairly common request is to be able to duplicate a Xen instance. For example you might have a DomU for the purpose of running Wordpress and want another DomU to run MediaWiki. The difference in configuration between two DomU’s for running web based services that are written in PHP and talking to a MySQL back-end is quite small, so copying the configuration is easier than a clean install. It is a commonly held opinion that a clean install should be done every time and that Kickstart on Red Hat, FAI on Debian and comparable technologies on other distributions can be used for a quick automatic install. I have not yet got FAI working correctly or got Kickstart working on Xen (it’s on my todo list - I’ll blog about it when it’s done). Regardless of whether it’s a good idea to copy a Xen DomU, there are often situations where clients demand it or when it’s impractically difficult to do a fresh install. I believe that the most sensible way to store block devices with Xen is to use LVM. It is a requirement for a Xen system that you can easily create new block devices while the machine is running and that the size of block devices can be changed with minimal effort. This rules out using Linux partitions and makes it unreasonably difficult to use LUNs on a fiber-channel SAN or partitions on a hardware RAID. LVM allows creating new block devices and changing the size of block devices with minimal effort. Another option would be to use files on a regular filesystem to store the filesystem data for Xen DomU’s, if choosing this option I recommend using XFS [1] filesystem (which delivers good performance with large filesystems and large files). If you use XFS to store the block devices for the DomU that you want to copy then you will need to halt the DomU for the duration of the copy as there is no other way of getting an atomic copy of the filesystem while it’s in use. The way of doing this would be to run the command “xm console foo ; cp /mnt/whatever/foo-root /mnt/whatever/bar-root ; xm create -c foo” where “foo” is the name of the DomU and “/mnt/whatever/foo-root” is the file that is used to store the root device for the DomU (note that multiple cp commands would be needed if there are multiple block devices). The reason for having the two xm commands on the one line is that you initially login to the DomU from the console and type halt and then the xm command will terminate when the DomU is destroyed. This means that there is no delay from the time the domain is destroyed to the time that the copy starts. If you use LVM to store the block device then things are a little easier (and you get no down-time). You simply run the command “lvcreate -s -L 300m -n foo-snap /dev/V0/foo-root” to create a snapshot with the device name /dev/V0/foo-snap which contains a snapshot the of the LV (Logical Volume) /dev/V0/foo-root. The “-L 300m” option means to use 300Meg of storage space for the snapshot - if the writes to /dev/V0/foo-root exceed 300Meg of data then your snapshot breaks. There is no harm in setting the allocated space for the snapshot to be the same as the size of the volume that you are going to copy - it merely means that more disk space is reserved and unavailable for other LVM operations. Note that V0 needs to be replaced by the name of the LVM VG (Volume Group) Once you have created the snapshot you can create a new LV with the command “lvcreate -n new-root -L X /dev/V0” where X is the size of the device (must be at least as big as the device you are copying) and then copy the data across with a command similar to “dd if=/dev/V0/foo-snap of=/dev/V0/new-root bs=1024k“. After the copy is finished you must remove the snapshot with the command “lvremove /dev/V0/foo-snap” (please be very careful when running this command - you really don’t want to remove an LV that has important data). Note that in normal operation lvremove will always give a prompt “Do you really want to remove active logical volume“. If you made the new device bigger then you must perform the operations that are appropriate for your filesystem to extend it’s size to use the new space. There is no need to copy a swap device, it’s easier to just create a new device and run mkswap on it. After copying the data you will need to create the new Xen config (by copying /etc/xen/foo to the new name). Make sure that you edit the Xen config file to use the correct block devices and if you are specifying the MAC address [2] by a “vif” line in the config file make sure that you change them to unique addresses for your LAN segment (reference [2] has information on how to select addresses). Now you must mount the filesystem temporarily to change the IP address (you really don’t want two DomU’s with the same IP address). If your Dom0 has untrusted users or services that are accessed by untrusted users (IE any Internet facing service) then you want to mount the filesystem in question with the options nosuid and nodev so that if the DomU has been cracked it won’t allow cracking of the Dom0. After changing the configuration files to change the IP address(es) of the DomU you can then umount the filesystem and start it with the xm create command. If instead of creating the clone DomU on the same Dom0 you want to put it on a different system you can copy the block devices to files on a regular filesystem on removable media (EG an IDE disk with USB attachment). When copying the block devices you also need to copy the Xen configuration and edit it to reflect the new paths to block devices for the data once it’s copied to the new server, but you won’t necessarily need to change the MAC address if you are copying it to a different LAN segment. References:
  1. http://en.wikipedia.org/wiki/XFS

  2. http://en.wikipedia.org/wiki/MAC_address

Steve Kemp: Ed il mio bacio sciogliera il silenzio che ti fa mia!

Tonight I mostly installed Centos 5 upon dedicated servers in America, for a guy based in Germany. Sometimes people do pay me for work which is nice - this job paid for a small stirling engine :) Yesterday I mostly implemented a web interface for controlling spam via a proxy server. Several people seemed interested, so I'll explain what I've done: The only part that is missing is the final plugin integration, and some glue to propogate some of the GUI settings to the mail server - since adding a new domain is currently 50% GUI and 50% manual. Each of the plugins supports whitelisting of addresses or domains, and there is a fair amount of logging detail - some is missing which I need to resolve. Once that is done and the code is running upon my live server I'm happy to let people use it if they have a spare/non-critical domain they can expirement with... The next thing to do would be to hook in spam assassin, spambayes, etc, on a per-domain basis with "training" addresses..

6 September 2007

Bernd Zeimetz: mod_wsgi 1.0

mod_wsgi 1.0 Yesterday the official 1.0 version of mod_wsgi was released and uploaded to unstable (the binary package you want to install is called libapache2-mod-wsgi, in case you were wondering), so I think it's a good time to blog about it. Mod_wsgi is an Apache module which can host any Python application which supports the WSGI interface (the reference is also available as package). Major applications and frameworks which are known to work well with mod_wsgi include CherryPy, Django, Karrigell, Pylons, TurboGears, web.py, MoinMoin, PyBlosxom and Trac. In fact every application which provides a WSGI compatible interface should be able to be hosted with mod_wsgi. The module is very simple to use, is very well configurable, documented, has a very active and responsive upstream author and is suitable for use in hosting high performance production web sites, as well as your average personal site. There are two common ways of operation: If you're running python web applications which provide a WSGI interface: go and give mod_wsgi a try! ~~DISCUSSION~~

Russell Coker: Citing References in Blog Posts

A significant problem with the old-fashioned media is that as a general rule they don’t cite references for anything. Some of the better TV documentaries and non-fiction books cite references, but this is the exception not the norm. Often documentaries only cite references in DVD extras which are good for the people who like the documentary enough to buy it but not for people who want to rebut it (few people will pay for a resource if they doubt the truth and accuracy of it’s claims). I can understand newspapers not wanting to publish much in the way of background information in the paper version as every extra line of text in an article is a line of advertising that they can’t sell. So they have financial pressure to produce less content, and the number of people like me who want to check the facts and figures used in articles is probably a small portion of the readership. Another issue with newspapers is that they are often considered as primary authoritative sources (by themselves and by the readers). It is often the case that journalists will interview people who have first-hand knowledge of an issue and the resulting article will be authoritative and a primary source in which case all they need to do is to note that they interviewed the subject. However the majority of articles published will be sourced from elsewhere (news agencies [ http://en.wikipedia.org/wiki/News_agency ] such as Reuters are commonly used). Also articles will often be written based on press releases - it is very interesting to read press releases and see how little work is done by some media outlets to convert them to articles, through a well written press release a corporation or interest group can almost write it’s own articles for publication in the old media. One way of partially addressing the problem of citing references in old media would be to create a web site of references, then every article could have a URL that is a permanent link to the references and calculations to support the claims and numbers used. Such a URL could be produced by any blogging software, and a blog would be an ideal way of doing this. For bloggers however it’s much easier to cite references and readers have much higher expectations of links to other sites to support claims and of mathematical calculations shown to indicate how numbers are determined. But there is still room for improvement. Here are some of the most common mistakes that I see in posts by people who are trying to do the right thing:
  1. Indirect links. When you refer to a site you want to refer to it directly. In email (which is generally considered a transient medium) a service such as TinyURL [ www.TinyURL.com ] can be used to create short URLs to refer to pages that have long URLs. This is really good for email as there are occasions when people will want to write the address down and type it in to another computer. For blogging you should assume that your reader has access to browse the web (which is the case most of the time). Another possibility is to have the textual description of a link include a reference to the TinyURL service but to have the HREF refer to the real address. Any service on the net may potentially go away at some future time. Any service on the net may have transient outages, and any reader of your blog may have routing problems that make parts of the net unavailable to them. If accessing a reference requires using TinyURL (or a similar service) as well as the target site then there are two potential things that might break and prevent your readers from accessing it.
    One situation where indirect links are acceptable is for the printed version. So you could have a link in the HTML code for readers to click on to get to the reference page directly and a TinuURL link for people who have a printed version and need to type it in.
    Also when linking to a blog it’s worth considering the fact that a track-back won’t work via TinyURL and track-backs may help you get more readers…

  2. Links that expire. For example never say “there’s a good article on the front page of X” (where X is a blog or news site). Instead say “here’s a link to a good article which happens to be on the front page now” so that someone who reads your post in a couple of years time can see the article that you reference.
    Another problem is links to transient data. For example if you want to comment on the features of a 2007 model car you should try to avoid linking to the car manufacturer page, next year they will release a new car and delete the old data from their site.
    A potential problem related to this is the Google cache pages which translate PDF to HTML and high-light relevant terms and can make it much easier to extract certain information from web pages. It can provide value to readers to use such links but AFAIK there is no guarantee that they will remain forever. I suggest that if you use them you should also provide the authoritative link so that if the Google link breaks at some future time then the reader will still be able to access the data.

  3. Not giving the URLs of links in human readable form. Print-outs of blog pages will lose links and blog reading by email will also generally lose links (although it would be possible to preserve them). This counts for a small part of your readership but there’s no reason not to support their needs by also including links as text (either in the body or at the end of the post). I suggest including the URL in brackets, the most important thing is that no non-URL text touch the ends of the URL (don’t have it in quotes and have the brackets spaced from it). Email clients can generally launch a web browser if the URL is clear. Note that prior to writing this post I have done badly in this regard, while thinking about the best advice for others I realised that my own blogging needed some improvement.
    I am not certain that the practice I am testing in this post of citing URLs inline will work. Let me know what you think via comments, I may change to numbering the citations and providing a list of links in the footer.

  4. Non-specific links. For example saying “Russell Coker wrote a good post about the SE Linux” and referring to my main blog URL is not very helpful to your readers as I have written many posts on that topic and plan to write many more (and there is a chance that some of my future posts on that topic may not meet your criteria of being “good”). Saying “here is a link to a good post by Russell Coker, his main blog URL is here” is more useful, it gives both the specific link (indicating which post you were referring to) and the general information (for people who aren’t able to find it themselves, for the case of deleted/renamed posts, and for Google). The ideal form would be “<A HREF=”http://etbe.coker.com.au/whatever”>here is a link to a good post by Russell Coker [ http://etbe.coker.com.au/whatever ]</A>, his main blog URL is <A HREF=”http://etbe.coker.com.au/”> [ http://etbe.coker.com.au ]</A>” (note that this is an example of HTML code as a guide for people who are writing their own HTML, people who use so-called WYSIWYG editors will need to do something different).

  5. Links that are likely to expire. As a rule of thumb if a link is not human readable then the chance of it remaining long-term is low. Companies with content management systems are notorious for breaking links.

  6. Referencing data that you can’t find. If you use data sourced from a web site and the site owner takes it down then you may be left with no evidence to support your assertions. If data is likely to be removed then you should keep a private copy off-line (online might be an infringement of copyright) for future reference. It won’t let you publish the original data but will at least let you discuss it with readers.

  7. Referencing non-public data. The Open Access movement [ http://en.wikipedia.org/wiki/Open_access ] aims to make scholarly material free for unrestricted access. If you cite papers that are not open access then you deny your readers the ability to verify your claims and also encourage the companies that deny access to research papers.
    An insidious problem is with web sites such as the New York Times [ www.nytimes.com ] which need a login and store cookies. As I have logged in to their site at some time in the past I get immediate access to all their articles. But if I reference them in a blog post many readers will be forced to register (some readers will object to this). With the NYT this isn’t such a problem as it’s free to register so anyone who is really interested can do so (with a fake name if they wish). But I still have to keep thinking about the readers for such sites.
    I should probably preview my blog posts from a different account without such cookies.

  8. Failing to provide calculations. My current procedure is to include the maths in my post, for example if you have a 32bit data type used to store a number of milliseconds then it can store 2^32/1000 seconds which is 2^32/1000/60/60/24 = 49.7 days, in this example you can determine with little guessing what each of the numbers represent. For more complex calculations an appendix could be used. A common feature of blogs is the ability to have a partial post sent to the RSS feed and the user has the ability to determine where the post gets cut. So you could cut the post before the calculations, the people who want to see them will find it’s only one click away, and the people who are happy to trust you will have a shorter post.

  9. Linking with little reason. Having a random word appear highlighted with an underline in a blog post is often not very helpful for a reader. It sometimes works for Wikipedia links where you expect that most readers will know what the word means but you want to link to a reference for the few who don’t (my link for the word Wikipedia is an example). In the case where most readers are expected to know what you are referring to then citing the link fully (with a description of the link and a human-readable form for an email client) is overkill and reduces the readability of the text.
    The blogging style of “see here and here for examples” does not work via email and does not explain why a reader should visit the sites. If you want to include random links in a post then having a section at the footer of related links would probably be best.

  10. Linking to a URL as received. Many bloggers paste URLs from Google, email, and RSS feeds into their blog posts. This is a bad idea because it might miss redirection to a different site. If a Google search or an email gives you a URL that is about to go away then it might redirect to a different site. In that case citing the new URL instead of the old one is a service to your readers and will decrease the number of dead-links in your blog over the long-term. Also using services such as www.feedburner.com may cause redirects that you want to avoid when citing a blog post, see my previous post about Feedburner [ http://etbe.coker.com.au/2007/08/20/feedburner-item-link-clicks/ ].

Here are some less common problems in citing posts:
  1. Inappropriately citing yourself. Obviously if there is a topic that you frequently blog about then there will be benefit to linking to old posts instead of covering all the background material, and as long as you don’t go overboard there should not be any problems (links to your own blog are assumed to have the same author so there is no need for a disclaimer). If you write authoritative content on a topic that is published elsewhere then you will probably want to blog about it (and your readers will be interested). But you must mention your involvement to avoid giving the impression that you are trying to mislead anyone. This is particularly important if you are part of a group that prepares a document, your name may not end up on the list of authors but you have a duty to your readers to declare this.
    Any document that you helped prepare can not be used by itself as a support of claims that you make in a blog post. You can certainly say “I have previously demonstrated how to solve this problem, see the following reference”. But links with comments such as “here is an example of why X is true” are generally interpreted to be partly to demonstrate the popular support for an idea.

  2. Citing secret data. The argument “if you knew what I know then you would agree with me” usually won’t be accepted well. There are of course various levels of secrecy that are appropriate. For example offering career advice without providing details of how much money you have earned (evidence of one aspect of career success) is acceptable as the readers understand the desire for some degree of financial secrecy (and of course in any game a coach doesn’t need to be a good player). Arguing the case for a war based on secret data (as many bloggers did) is not acceptable (IMHO), neither is arguing the case for the use of a technology without explaining the science or maths behind it.

  3. Not reading the context of a source. For example I was reading the blog of a well regarded expert in an area of computer science, and he linked to another blog to support one of his claims. I read the blog in question (more than just the post he cited) and found some content that could be considered to be racially offensive and much of the material that I read contained claims that were not adequately supported by facts or logic. I find it difficult to believe that the expert in question (for whom I have a great deal of respect) even casually inspected the site in question. In future I will pay less attention to his posts because of this. I expect a blogger to pay more attention to the quality of their links than I do as a reader of their blog.

While writing this post I realised that my own blogging can be improved in this regard. Many of my older posts don’t adequately cite references. If you believe that any of my future posts fail in this regard then please let me know.

5 September 2007

Gustavo Franco: Google!

I've just accepted an job offer from Google and I'm now moving from Rio de Janeiro to Belo Horizonte!

I would like to say thank you very much for those who helped me in the Debian project until now, you know who you are.

Adeodato Sim : Presenting Minirok

I have written a new music player to fulfill my needs, and I’ve dubbed it Minirok. It is written in Python, uses KDE libraries, and GStreamer as a backend. It is also heavily modelled after Amarok, with the look and feel being pretty much identical, but with a reduced set of features. The main deal is that there is no collection built from tags: only a tree view of the filesystem, which I sorely missed in Amarok. Because of this, most probably only folk whose collection is already structured as a tree at the filesystem level will find it enjoyable. Version 0.6 was released yesterday, and is already in Debian. Check out the screenshots, and feel free to provide feedback. P.S.: I am aware that minirok means miniskirt in Dutch.

Ian Wienand: Short review of guarded page-tables

Anyone who has done an operating systems 101 course is familiar with the standard hierarchical page-table structure. This is essentially a radix-tree where bits of the virtual address are taken as indexes into an array of pointers, each pointing to the next level down or, at the leaves, to a physical entry. Hierarchical page-table One potential issue with this style of page-table is with a very large, very sparse address space (such as in a single address-space operating system) there will be many intermediate steps with possibly only a single translation at the end (i.e. lots of single pages scattered all around your address-space). Thus we can add a bit-field guard to each of the entries in the page table (Liedtke, 1994) to help indicate only those paths through the tree which are valid (similar to a Patricia tree). Translation first proceeds as usual; the top bits are used as an index into the top level table. However, each entry now contains extra information about further valid addresses. The guard is masked against the next bits in the virtual address; if it matches (e.g. in our example is 0x2) the next step can proceed directly to the leaf node, bypassing the middle look-up. The entry can be marked as either final (i.e. pointing to a leaf node) or intermediate, in which case the process continues with the next level (which also has its own guard, and so forth). Any entry that does not match the guard is known to be false, and therefore a fault can be raised. Hierarchical page-table with guard To examine this further, consider that we know that each page in our 24-bit address space above is 4KiB, thanks to the 212 = 4KiB offset. Therefore each of the 16 possible values of the top four bits selects 1MiB of the 224 = 16MiB address space; e.g. 0x0 selects the first megabyte, 0x1 the second and so forth. The second 4 bits selects a 64KiB region of this 1MiB, and the last 4 bits selects a 4KiB region of this 64KiB region. This scheme could work quite well if we were using only 16 pages each separated by 1MiB. Each of these pages would only require an entry in the top level with an appropriate guard which could then be marked as final and point directly to a physical frame. Once things become less sparse however, we need to start filling in middle levels. If you have 2 pages within a 16MiB region, you need at least one extra bit of address to tell them apart. If you have 3 or 4 pages, you need at least 2 bits to tell them apart, and so forth. It would therefore be beneficial to have variable level sizes to dynamically adapt and create the most efficient lookup possible given the address space layout of a process. For example, if you knew your process was only going map pages 0x123 (100100011) and 0x124 (100100100) a good option would be to make your top-level only two entries (i.e. check the top bit) with a guard on the entry at index 0x1 of 00100. You then need a second level with 8 pointers to differentiate the remaining 3 bits (each of these would be final and hence point directly to a physical page). The more pages you start mapping, the less bits you can discard with the guard. If you follow this through to a guard of 0 bits you end up with a linear page-table. It has been shown (Elphinstone, 1999) that fixed level sizes with 16 entries tends to work well. For a 32-bit address-space with 4KiB pages (e.g. 12 bits offset) this leaves 20 bits to be mapped by the page-tables; with each level mapping 24 bits this means a 5 level tree. The guard values provide enough path-compression that the table is not constantly walked. The disadvantage is that it may "over-allocate" meaning more space is dedicated to page-tables than strictly required. Deciding where and when to split levels, what size indexes to put at each level and updating all the guard values dynamically to make the smallest, most efficient page-table would create a variable-radix page-table (Szmajda and Heiser, 2003). In summary, a guarded page-table is more useful the more sparse your address space is. A variable-radix guarded page-table is complex, but could offer advantages for implementing variable page-size support and thus having positive effects on TLB coverage.

Adam Rosi-Kessel: The biggest thing in their life is word of mouth

Spot-on profile of Rick Rubin and the future of music in this Sunday’s New York Times Magazine (probably already much blogged about):
The kids all said that a) no one listens to the radio anymore, b) they mostly steal music, but they don’t consider it stealing, and c) they get most of their music from iTunes on their iPod. They told us that MySpace is over, it’s just not cool anymore; Facebook is still cool, but that might not last much longer; and the biggest thing in their life is word of mouth. That’s how they hear about music, bands, everything.
That last point is key. If any new business model will thrive, it will be one that is built around the social graph. Technorati Tags: , , , , , ,

3 September 2007

Russell Coker: Photography and Censorship at APEC

World News Australia reports that Police forced three tourists to delete photos of a fence. Apparently the officers in question believed that such photos would be a threat to security. It’s interesting to note that the first sentence of the World News Australia report is “Officials say police who forced three tourists to delete photos of a huge security fence erected in Sydney for the APEC summit were not over-reacting” while later in the article it says “The police action against the tourists may have been “over the top” but was necessary, said New South Wales state Transport Minister John Watkins“. So which was it? Was it “over the top” or was it “not over-reacting“. Strangely the World News Australia web site shows a picture of the fence. If this fence is so secret then why are the pictures being published so that millions of people will see it? For that matter why even try censoring the pictures. Censoring the pictures effectively issues a challenge to everyone who has a digital camera and plenty of spare time (which means most university students among others) to get the best photos of the fence (feel free to leave comments with the URLs for your best pics). While this is happening protesters from Real Action On Climate Change have been protesting at the Loy Yang power station, according to their blog posts it seems that they were partially inspired by the APEC meeting. It seems that APEC leaders are keen on nuclear power. If they really believe that nuclear power is safe then maybe they should have their meeting at Maralinga. There would be little effort or expense required to secure Maralinga and it wouldn’t disrupt a major city. :-# But seriously if they wanted a secure location for a meeting with no protesters then an aircraft carrier would make an ideal location. The people who need to attend the meeting could get flown to a carrier that’s off-shore in the territorial waters of one of the countries concerned with a full battle group to deter any other ships from entering the area. It is reported that Sydney whores are expecting to do a lot of business during the APEC meeting. Maybe this is the reason why they wanted to have their meetings Sydney (which is rumoured to have the best brothels in Australia). I wonder if they are planning for a future APEC meeting in Bangkok…

Holger Levsen: dancing and singing

Today I'm dancing and singing. :-)

In part because I've just uploaded a new version of tuxmath which IMO has a great improvement: tuxmath is a math game for children, where the player has to solve little math computations, otherwise the cities on the bottom of the screen get destroyed like in space invaders. Which is not exactly suited very well for little children. So the idea came up, to instead use igloos, which melt, so that in the end a penguin has to escape from the igloo when it gets destroyed. Very cute, but still the same gameplay. And the best thing is, those igloos were painted by a third grade daughter of one the developers.

2 September 2007

MJ Ray: Current Issues / Re: Recycling

How are you lot getting on with North Somerset Recycling?

Badly.  I thought West Norfolk recycling was pretty awful (glass not collected at doorstep) but North Somerset is worse. Do this with one thing, that with another.  Why the blue blazes can't we have national consistency like Spain and Germany seem to?

Let me see if I've got this right:-

Paper, tins and glass can just go in the green boxes. Cloth has to be bagged and put in the green boxes.

Garden waste and corrugated cardboard can go in the green bags, but you have to remove all tape (even bio-degradable) from cardboard, put your address in the little pouch and put out a stone to weigh the bag down, else they'll not empty it.

Non-corrugated cardboard, plastics types 1 and 2 and electrical equipment have to be taken to the tip (which is awkward without using a car - try it some time).

Drinks cartons have to be put into a special wheeliebin next to Tesco's car park.

Finally, batteries have to posted to Norfolk in an envelope you can get from Sainsburys.

Figuring that out has only taken me a year... Did I miss anything? :-/

Antti-Juhani Kaijanaho: A tiny program

Written to count votes with the d’Hondt method, released as a single C99 source file. I know I know, why did I write it in C? Well, because I felt like it. Now you don’t have to ask that question in my comment space in a rude manner :)

Next.

Previous.