Search Results: "pm"

13 June 2008

David Welton: Cache it all

I recently redid my personal web site, at welton.it. Wanting to be quick about it, and make the look and feel a bit more uniform than it has been in the past, I hacked together some pages in Rails. Despite this being sort of a "killing a fly with a bazooka" situation, I've been doing lots with Rails, so it was quick to use. Here's the thing, though: Rails is definitely overkill, as the site is basically static. I don't need to calculate anything or fetch stuff from a database - I just wanted a reasonably good template system, and I am quite comfortable with Rails these days. But the idea of leaving Rails running for a static site was of course no good: I basically need to cache the entire thing, so that Rails is simply not involved. How to do this as quick as possible (in between diaper changing and other baby duties!) ? Ideally, it would be possible to introspect Rails in order to know exactly which pages are present, then cache those, and avoid Rails on the server entirely (just generate them locally and put them in subversion), but that proved to be fairly hacky, so I settled for this code, which simply caches all pages which comes across, when caches_pages is placed in application.rb:
class CacheFileName
  include ActionController::Caching::Pages
  include ActionController::Caching::Pages::ClassMethods
  def cachedname(path)
    page_cache_file(path)
  end
end
def caches_pages()
  return unless perform_caching
  after_filter do  c 
    res = c.cache_page
    cfn = CacheFileName.new
    cf = Cachedfile.new :filename => cfn.cachedname(c.request.path)
    cf.save!
  end
end
It simply caches everything. To be able to easily clear out the cache if there are any changes to the site, we record the changes in the Cachedfile model, which is defined like this:
create_table "cachedfiles", :force => true do  t 
  t.string   "filename"
  t.datetime "created_at"
  t.datetime "updated_at"
end
with this model:
class Cachedfile < ActiveRecord::Base
  def Cachedfile.clean_cache
    Cachedfile.find(:all).each do  cf 
      begin
        fn = ActionController::Base.page_cache_directory + cf.filename
        File.delete fn
      rescue => e
        logger.error "Error deleting # fn : # e.inspect "
      ensure
        cf.destroy
      end
    end
  end
end
which has a class method to go through and clean out all the cached files. I call it manually from ./script/clean_cache:
#!/usr/bin/env ruby
ENV['RAILS_ENV'] = ARGV.first   ENV['RAILS_ENV']   'development'
require File.dirname(__FILE__) + '/../config/boot'
require "# RAILS_ROOT /config/environment"
require 'console_app'
Cachedfile.clean_cache
It's not a beautiful system, but it gets the job done.

Russell Coker: I m Skeptical about Robotic Nanotech

There has been a lot of fear-mongering about nanotech. The idea is that little robots will eat people (or maybe eat things that we depend on such as essential food crops). It’s unfortunate that fear-mongering has replaced thought and there seems to have been little serious discussion about the issues. If (as some people believe) nanotech has the potential to be more destructive than nuclear weapons then it’s an issue that needs to be discussed in debates before elections and government actions to alleviate the threat need to be reported on the news - as suggested in the Accelerating Future blog [0]. I predict that there will be three things which could be called nanotech in the future:

  1. Artifical life forms as described by Craig Venter in his talk for ted.com [1]. I believe that these should be considered along with nanotech because the boundary between creatures and machines can get fuzzy when you talk about self-replicating things devised by humans which are based on biological processes.
    I believe that artificial life forms and tweaked versions of current life forms have significant potential for harm. The BBC has an interesting article on health risks of GM food which suggests that such foods should be given the same level of testing as pharmaceuticals [2]. But that’s only the tip of the iceberg, the potential use of Terminator Gene technology [3] in biological warfare seems obvious.
But generally this form of nanotech has the same potential as bio-warfare (which currently has significantly under-performed when compared to other WMDs) and needs to be handled in the same way.
  • The more commonly discussed robotic nanotech, self-replicating and which can run around to do things (EG work inside a human body). I doubt that tiny robots can ever be as effective at adapting to their environment as animals, I also doubt that they can self-replicate in the wild. Currently we create CPU cores (the most intricate devices created by humans) from very pure materials in “clean rooms”. Making tiny machines in clean-rooms is not easy, making them in dirty environments is going to be almost impossible. Robots as we know them are based around environments that are artificially clean not natural environments. Robots that can self-replicate in a clean-room when provided with pure supplies of the necessary raw materials is a solvable problem. I predict that this will remain in science-fiction.

  • Tiny robots manufactured in factories to work as parts of larger machines. This is something that we are getting to today. It’s not going to cause any harm as long as the nano-bots can’t be manufactured on their own and can’t survive in the wild.

  • In summary, I think that the main area that we should be concerned about in regard to nano-bot technology is as a new development on the biological warfare theme. This seems to be a serious threat which deserves the attention of major governments.

    12 June 2008

    Russell Coker: ECC RAM is more useful than RAID

    A common myth in the computer industry seems to be that ECC (Error Correcting Code - a Hamming Code [0]) RAM is only a server feature. The difference between a server and a desktop machine (in terms of utility) is that a server performs tasks for many people while a desktop machine only performs tasks for one person. Therefore when purchasing a desktop machine you can decide how much you are willing to spend for the safety and continuity of your work. For a server it’s more difficult as everyone has a different idea of how reliable a server should be in terms of uptime and in terms of data security. When running a server for a business there is the additional issue of customer confidence. If a server goes down occasionally customers start wondering what else might be wrong and considering whether they should trust their credit card details to the online ordering system. So it is obviously apparent that servers need a different degree of reliability - and it’s easy to justify spending the money. Desktop machines also need reliability, more so than most people expect. In a business when a desktop machine crashes it wastes employee time. If a crash wastes an hour (which is not unlikely given that previously saved work may need to be re-checked) then it can easily cost the business $100 (the value of the other work that the employee might have done). Two such crashes per week could cost the business as much as $8000. The price difference between a typical desktop machine and a low-end workstation (or deskside server) is considerably less than that (when I investigated the prices almost a year ago desktop machines with server features ranged in price from $800 to $2400 [1]). Some machines in a home environment need significant reliability. For example when students are completing high-school their assignments have a lot of time invested in them. Losing an assignment due to a computer problem shortly before it’s due in could impact their ability to get a place in the university course that they most desire! Then there is also data which is irreplaceable, one example I heard of was of a woman who’s computer had a factory pre-load of Windows, during a storm the machine rebooted and reinstalled itself to the factory defaults - wiping several years of baby photos… In both cases better backups would mostly solve the problem. For business use the common scenario is to have file servers storing all data and have very little data stored on the PC (ideally have no data on the PC). In this case a disk error would not lose any data (unless the swap space was corrupted and something important was paged out when the disk failed). For home use the backup requirements are quite small. If a student is working on an important assignment then they can back it up to removable media whenever they reach a milestone. Probably the best protection against disk errors destroying assignments would be a bulk purchase of USB flash storage sticks. Disk errors are usually easy to detect. Most errors are in the form of data which can not be read back, when that happens the OS will give an error message to the user explaining what happened. Then if you have good backups you revert to them and hope that you didn’t lose too much work in the mean-time (you also hope that your backups are actually readable - but that’s another issue). The less common errors are lost-writes - where the OS writes data to disk but the disk doesn’t store it. This is a little more difficult to discover as the drive will return bad data (maybe an old version of the file data or maybe data from a different file) and claim it to be good. The general idea nowadays is that a filesystem should check the consistency of the data it returns. Two new filesystems, ZFS from Sun [2] and BTRFS from Oracle [3] implement checksums of data stored on disk. ZFS is apparently production ready while BTRFS is apparently not nearly ready. I expect that from now on whenever anyone designs a filesystem for anything but the smallest machines (EG PDAs and phones) they will include data integrity mechanisms in the design. I believe that one such features become commonly used the need for RAID on low-end systems will dramatically decrease. A combination of good backups and knowing when your live data is corrupted will often be a good substitute for preserving the integrity of the live data. Not that RAID will necessarily protect your data - with most RAID configurations if a hard disk returns bad data and claims it to be good (the case of lost writes) then the system will not read data from other disks for checksum validation and the bad data will be accepted. It’s easy to compute checksums of important files and verify them later. One simple way of doing so is to compress the files, every file compression program that I’ve seen has some degree of error detection. Now the real problem with RAM which lacks ECC is that it can lose data without the user knowing. There is no possibility of software checks because any software which checks for data integrity could itself be mislead by memory errors. I once had a machine which experienced filesystem corruption on occasion, eventually I discovered that it had a memory error (memtest86+ reported a problem). I will never know whether some data was corrupted on disk because of this. Sifting through a large amount of stored data for some files which may have been corrupted due to memory errors is almost impossible. Especially when there was a period of weeks of unreliable operation of the machine in question. Checking the integrity of file data by using the verify option of a file compression utility, fsck on a filesystem that stores checksums on data, or any of the other methods is not difficult. I have a lot of important data on machines that don’t have ECC. One reason is that machines which have ECC cost more and have other trade-offs (more expensive parts, more noise, more electricity use, and the small supply makes it difficult to get good deals). Another is that there appear to be no laptops which support ECC (I use a laptop for most of my work). On the other hand RAID is very cheap and simple to implement, just buy a second hard disk and install software RAID - I think that all modern OSs support RAID as a standard installation option. So in spite of the fact that RAID does less good than a combination of ECC RAM and good backups (which are necessary even if you have RAID), it’s going to remain more popular in high-end desktop systems for a long time. The next development that seems interesting is the large portion of the PC market which is designed not to have the space for more than one hard disk. Such compact machines (known as Small Form Factor or SFF) could easily be designed to support ECC RAM. Hopefully the PC companies will add reliability features in one area while removing them in another.

    11 June 2008

    Evan Prodromou: 24 Prairial CCXVI

    Quick shout to announce an event here in Montreal. In response to the announced introduction of legislation to change Canadian copyright, commonly called the "Canadian DMCA" since the drafts presented so far have closely resembled that American law, the Montreal Chapter of Fair Copyright for Canada is holding an emergency organizational meeting Thursday, June 12th from 6-8PM. The event will be at Station C, 5369 Saint-Laurent #430, in the Mile End neighborhood of Montreal. It's open to anyone interested in learning more about this bill and helping to oppose it. tags:

    9 June 2008

    Christian Perrier: Samba 4 in experimental

    The (still alpha) 4.0.0 release of Samba is now in experimental. Both Samba 3 and Samba 4 packages are likely to coexist in the archive as Samba 3 is still for a while the production code for File and Print services while Samba 4 is the development branch of the Samba team to implement Active Directory domain control (among other things). Samba4 packages also bring a big bunch of libraries, most of which being used to related development such as Openchange. Expect some neat new packages in experimental, then unstable, pretty soon now. Please do NOT replace production servers running samba 3.0.* or samba 3.2.* by samba4....not yet..:-) The samba4 packages are maintained by Jelmer Vernooij, one of the two main developers for Samba 4 in the Samba Team (the other being Andrew Bartlett). Jelmer is someone we should stuck out from the NM queue as soon as possible..:-) Anyway, this is again another success of that small team we built around Samba and samba-related packaging. Thanks to all involved people (Steve Langasek, No l K the, Jelmer Vernooij, Peter Eisentraut, etc.).

    Matthew Garrett

    Things in your computer that you do not have the source code for:Anyone think of anything I've missed?

    Things people have suggested:
    More suggestions:

    David Pashley: Compiled Regexes in Spamassassin 3.2

    Spamassassin 3.2, which is available in Gutsy and Lenny, comes with a new feature to increase performance by compiling its regular expressions using re2c. It's very quick to enable. First, you need to install the required packages:
    apt-get install re2c libc6-dev gcc make
    Next, edit /etc/spamassassin/v320.pre and uncomment the line that says:
    loadplugin Mail::SpamAssassin::Plugin::Rule2XSBody
    Next pre-compile the regular expressions using sa-compile:
    femme:/etc/logcheck# sa-compile
    [18741] info: generic: base extraction starting. this can take a while...
    [18741] info: generic: extracting from rules of type body_0
    100% [===========================] 3293.83 rules/sec 00m00s DONE
    100% [===========================] 650.12 bases/sec 00m01s DONE
    [18741] info: body_0: 647 base strings extracted in 2 seconds
    [snip compiler output]
    make install
    Files found in blib/arch: installing files in blib/lib into architecture dependent library tree
    Installing /var/lib/spamassassin/compiled/3.002004/auto/Mail/SpamAssassin/CompiledRegexps/body_0/body_0.so
    Installing /tmp/.spamassassin18741hDrlUQtmp/ignored/man/man3/Mail::SpamAssassin::CompiledRegexps::body_0.3pm
    Writing /var/lib/spamassassin/compiled/3.002004/auto/Mail/SpamAssassin/CompiledRegexps/body_0/.packlist
    Appending installation info to /var/lib/spamassassin/compiled/3.002004/perllocal.pod
    cp /tmp/.spamassassin18741hDrlUQtmp/bases_body_0.pl /var/lib/spamassassin/compiled/3.002004/bases_body_0.pl
    cd /
    rm -rf /tmp/.spamassassin18741hDrlUQtmp
    
    Finally, restart spamassassin, and you should find it runs faster. You will need to run sa-compile every time you update your rules, or they won't take effect. If you get the following warning:
    Can't locate Mail/SpamAssassin/CompiledRegexps/body_0.pm in @INC
    you forgot to run sa-compile; re-run it and the error should go away.
    Read Comments (0)

    8 June 2008

    Manoj Srivastava: Manoj: Using org-mode with Ikiwiki

    Using org mode to write ikiwiki posts I have been much taken by org-mode in Emacs recently. Firstly, this has allowed me to actually effectively implement the tenets of Getting things done , which is amazing by itself. I always used to start with great enthusiasm, which rarely lasted longer than a week. Until org-mode. Then this is so amazingly flexible. It has better handling of tables and even a spreadsheet mode than anything I have used and it has a pretty decent HTML export function. I practically live in org-mode on work days now. I strongly recommend org-mode, from things like agendas, TODO lists, calendars, appointments, articles, and now Ikiwiki posts. I rarely ever use markdown, apart from blogging using Ikiwiki and thus, since I blog so infrequently, I rarely ever remember all the syntax. On the other hand, org-mode syntax is now at my finger tips so I decided it was time to write a Ikiwiki plugin so I can now write my blog posts using syntax I can actually remember. It is still rough around the edges firstly, this requires emacs23 (or, on Debian, there is the unofficial emacs-snapshot package), secondly, it uses hueristic hacks to prune away the bits of the stand alone HTML page that org-mode creates; all I can say is that it currently works for me, as you can see from this post. How git this is for mainlne inclusion is another matter. The code is here: org.pm. It can be dropped in to the Ikiwiki plug-in directory, and that s all you need. With this, Ikiwiki pre-process things in the input file, and then passes it to emacs to htmlize it.

    Andrew Pollock: [life] And we're back

    We actually got home last Sunday, but I've been too jetlagged, and then too busy with work to really have a chance to write anything... We got back in at around 2pm, but our bags fell victim to Heathrow's Terminal 5, and didn't arrive until around 11pm. I think I lasted until about 8pm, when I had to crash. Poor Sarah had to stay up until 11pm when they finally got around to delivering our suitcases. The week in Zurich was wonderful. The Zurich office is everything the photos make it out to be, and then some. One weird thing: apparently there's some Swiss regulation about how much you're allowed to cool a building in relation to the outdoor ambient temperature, so there's no air conditioning in the office. Instead, you can open the windows. Unfortunately, there's a fairly busy set of train tracks right beside the office, so it can get a tad noisy... It was a pretty warm week, I think around 30°C. I certainly prefer being colder rather than hotter when indoors, so found the lack of decent cooling to be a shame, given the rest of the building's features. That said, it was pretty amazing sweltering away in Zurich, and then looking up and seeing snow on the mountains. It was surprising how much of a temperature difference there was. On the last evening in Zurich, we went on a reconnoitre to try and find the river that we could hear (and see on the map) behind our apartment. We eventually found some street access to it, and it was another world back there. It was fairly thickly wooded, and the sunlight was heavily filtered through the trees, so it was cool and shady. The river was fairly fast flowing over some rocks in parts, which is what was making it so audible. The whole setting was absolutely beautiful. There were a couple of paths, and the whole thing felt like something out of a fairy tail. We think we stumbled onto a fox, but we're not sure. It's a shame we only discovered the place on the last day, as I'd have liked to have explored it further. I really liked Zurich. It was nice and flat, and had an excellent tram service. Monday Night Skate made me wish I'd packed my roller blades. We were wandering around on Monday night, and it felt like the entire city had donned skates and were going out. Apparently the authorities really get behind it, and close roads, and the police go along behind the pack and reopen the roads behind them and pick up the stragglers. Great way to encourage an active lifestyle. We've got all of our photos up now from the Europe trip, and they're here. This brings the total countries I've visited up to 10, excluding Hong Kong and Macau. I don't like selecting China when I generate this map, as I've never been to mainland China. Countries I've visited as of June 2008

    Axel Beckert: Bath Tub, Rubber Keyboard, Ratpoison and Opera

    I recently noticed that a very good way to safely read webcomics in the bath tub is an old laptop with a big screen (e.g. a IBM ThinkPad A-series like my 15” A31 which has a nice 1400×1050 resolution), a water proof keyboard, the screen-alike, keyboard only driven (hence the name) window manager ratpoison (other keyboard driven window managers like wmii or awesome probably will do as well as ratpoison) and a good keyboard driven web browser which can bind or by default has bound a key to follow <link rel="next" ... /> tags. Like Opera. Opera has bound the space bar to scroll one page down and if you reach the bottom of the page to go to the next page as labeled in the link tag. Additionally the full screen mode is helpful, too. Or the dream browser of all Emacs addicts, Conkeror, which has bound the function browser-follow-next to ]]. (Conkeror packages will hit Debian Experimental quite soon.) Or the GNOME feed reader Liferea which has bound Ctrl-Space by default to scroll down the content by one page and if you reach the bottom of the content go to the next unread item. With that equipment I can read my favourite web comics like Questionable Content (whose content seldomly is questionable :-) or Ozy and Millie (Think of a mixture of Calvin & Hobbes, Peanuts and Kevin & Kell) in the bath tub without drying my hands before reading the next comic or fearing water or health damage by the combination of water and computer. I just press one or two keys on the keyboard floating over my lap and have a good time.
    a keyboard floatiing in the bath tub close up of the floatiing keyboard
    BTW: I’ve got a blue, non-branded one (packaging reveals it as “AirTouch Keyboard”, probably manufactured by SanChuan Electronics, China) with swiss-german layout from ARP Datacom (whose website offers no permanent links and insists on session cookies *puke*), but those from Keysonic or from ROCK seem to be very similar — nowadays they are also available in illuminated, miscellaneous colors and wireless, but only IP65, probably because of the necessarily accessible battery compartment. But this kind of having fun still has optimisation potential: non-flexibel water-proof keyboard (IP67 recommended, so those IP66 keyboards and mice recently posted at UF LOTD are probably not tight enough), flat screen mounted above the bath tub, etc. ;-) Or maybe a completely water proof laptop if such thing exists — Does it? One more note: In Debian Sid and Lenny recently a new tool called keynav has been added, which allows you to control the mouse quickly using the keyboard only. So with Sid or Lenny, I don’t even need an waterproof mouse or trackball if an application insists on mouse usage. ;-)

    Biella Coleman: Anthropologocial Wonders and Myopias North and South

    Four years ago, the last time I was in Brazil, I came as an anthropologist-in-training to attend and give a talk at Debconf4 held in Porto Alegre. I have returned to this city in the south, again in the winter, but this time I have come back as an anthropologist who is giving a talk at the department of anthropology at the UFRGS, based on my research conducted many years ago. This time, thanks to my (really friendly and energetic) hosts, I am seeing far more of the actual city, its beautiful and legendary sunsets, its outlying neighborhoods, and even its subcultures. This has been my first foreign trip where I am interacting primarily with anthropologists and this has been really interesting for me. On the one hand, because of the language difference, there is a lot that is strange and hard for me to follow and I am acutely aware of the general and particular economic gulf between north and south and its impact on students (books are expensive, traveling for conferences is very difficult, and even applying to graduate schools in the north can be impossible because it costs so much to apply to each school!) On the other hand, the methods and subjects of study, the style of analysis, and the teaching are all very familiar, making me feel like I am part of a discipline that (thankfully) transcends national borders. Many of the projects I learned about–the tensions between free software advocacy and development, the role of memory among fan s of a corny country/Gaucho singer from this area and the surgical management of intersex (hermaphrodite) newborns were some of the most interesting projects I have come across and gobbled as much information about them as I could. I was also struck, yet again, by the deep myopias of anthropology, north and south. My own project, which focuses mostly on white, American and European hackers, often does not strike as culturally authentic enough because, well, the people I study are white, American and European (and I am slowly coming to see that if you study the so-called white and male or the North American/European elements of technology and the Internet and the law, you are probably white and male yourself and anything having to do with ethnicity/gender is usually the province of female academics, which I find really problematic). If I had carried out my first project in Brazil, where there is a foreign language, where there is a long tradition of studying various groups, then my project would have been stamped by that mysterious aura of authenticity/approval. On the other hand, two of the students here (those that worked on free software and the country singer) complained that their papers have yet to be accepted by the Brazilian Anthropological Association for being too strange, non-traditional, and it seems in their case, the problem is that they are studying urban Brazilian, popular culture (which for me would have just been just the thing to study). We are located far apart but find ourselves in similar positions by virtue of studying something geographically close to where we live, which is confused and misperceived as being culturally close to our world. But if there is something I wish to hammer into my students and other anthropologists is that there is tremendous plurality and multiplicity within our own societies. We can travel far, in the cultural sense, just by staying home and opening our eyes a little wider. Another nice experience is that I am learning a tremendous amount about free software politics and development from the graduate student, Luis Felipe, who is really responsible for getting me down here. He found me on IRC many moons ago and finally we have been able to compare notes and have long conversations about free software, the differences in how we can gather data due to gender, and a topic close to me, which is the tension between the political and apolitical in free software. Our relationship is at once based on friendship and also one of mentoring. And it reminds me of others who have also mentored me and how crucial this mentoring was to the development of my own work. One of my most important mentors, Chris Kelty, just published his book on Free Software (and I will soon write an entry about the book but the WHOLE THING IS ONLINE) and so it felt quite nice that we were discussing what I think his one of his most amazing chapters on the creation of the copyleft in our class on Friday. Not only was the topic about the genesis of the first free software license appropriate for a class on IP, but in many ways, because of his mentoring, I have gotten to where I am and so it felt also appropriate to honor and recognize that genealogy in my own work.

    7 June 2008

    David Nusinow: The Cognitive Load Of Web Development

    The most dynamic segment of the software industry that I'm aware of is web programming. Flamewars rage about frameworks and browsers and standards and whatnot, and the openness of the whole thing feels like some of the things that made me fall in love with computing as a whole and Free Software in particular. The problem with all that openness is that it's rather difficult to navigate. Aside from the obvious issues with browser incompatibilities, there's an enormous amount of software solutions on the server side, spanning languages, libraries, and frameworks. Each one is an ecosystem to itself, often being large and complicated.

    This makes it rather difficult to do certain aspects of web programming well. You are constantly switching between tools and languages, as well as coding paradigms, in order to build a web app in full. You'll be writing HTML, CSS, and Javascript on the client side, each with its own peculiarities. On the server side you'll probably have settled on one language with its own very different varieties of coding style and patterns. Some, like Ruby on Rails, are their own unique brand of language that makes you work both in the DSL (Rails in this case) and the parent language (Ruby here). Template engines often will do the same thing, such as in the case of Django.

    It's pretty obvious that having to juggle all these tools places a fairly large burden on the coder. Much to their credit, it's not difficult to work in any of these languages, but to work in them at a high level becomes more difficult the deeper and wider the software stack reaches. You get this a bit in systems programming, juggling C/C++ with a build system and a scripting language, but that's usually it. You can get that number of moving parts just on the client side of the web with HTML, CSS, and Javascript.

    Something interesting that seems to be happening is that people are moving to decrease this cognitive burden. One good example is the Google Web Toolkit which lets you write all your AJAXy client-side stuff in Java, and let it get compiled in to Javascript. That way you cut the languages you have to work in. A similar concept is behind Microsoft allowing Ruby (and Python, if I've heard correctly) in Silverlight. Alternately, you can bring Javascript (back) to the server side, which is the motivation behind Helma and Rhino on Rails (the latter of which I hope sees the light of day).

    Personally, I'm rooting for Javascript to not become an assembly language, but to take over the server side again. It's a capable and powerful language (much better than Java in my opinion), and we're collectively leveraging it very poorly. Rhino is interesting and has enormous potential, but it needs library binds in the worst way. Currently it's basically using Javascript to write Java, which is rather atrocious (COBOL in any language, etc) but with some Javascript-style wrappers around the Java classes it could be phenomenal. Alternately, there's Spidermonkey, which could be wrapped in a more capable shell with a good FFI, and we could easily have wrappers to our favorite Free Software libraries. This project seems far along here, but it's currently very windows-centric.

    There's a lot of potential here though. Unifying the development language for both web and system apps would benefit the Free Software desktop, and give us the ability to better integrate our stuff with the web. One of the best things about Free Software is that it dropped the barrier to entry in software development. I think we can repeat that success again here.

    MJ Ray: Bridgwater College Computing Advisory Panel

    Probably due to some mistake or just an excess of randomness in the world, our webmaster cooperative has been invited to Bridgwater College's Computing Advisory Panel meeting next week and it looks like I'm going. The interesting agenda items are:- 5. Current Curriculum offer 6. Computing and ICT Sector Developments - Education and Training - ICT Diplomas - Distance Learning - Specialist training / apprenticeships 7. Meeting Employer Needs I've asked some nearby cooperatives and social enterprises for comments and I'll probably highlight things like Linux opens London's Oyster and Specsavers sees clear benefits in open source when arguing for more free software use and emphasising "worker needs" rather than "employer needs", but please send me any other suggestions in a comment on this blog post or an email.

    6 June 2008

    Tore S. Bekkedal: Vitel A/S: Reach out and punch someone.

    (Playing Kraftwerk - The Telephone Call.) Warning: This is a long-wound complaint post. The only reason this is going to the Planet is because I have no method yet of disabling individual posts for syndication. Please skip, with my apologees for the noise, if you don’t want to listen to complaining. May contain nuts. May drive you nuts. Drove me nuts. Mmm, pistachio.My first cellphone! When I was around 11, I got my first cellphone. I was very proud. This was back when being 11 and having a cellphone was a real novelty; pagers were still being sold, and the NMT network was still running. Like many things I appreciate, I found it in a dumpster. It was ancient, even then: a Motorola MicroTAC brick, missing its antenna. I borrowed one of my grandmother’s hairpins, and voila, working phone - as long as you didn’t leave Oslo. It had a certain cyberpunk look to it, even then: I loved it. I remember that I first bought a NetCom pay-as-you-go card; I don’t quite remember the number, but it had “283″ in it at the end, if you want to wardial. For some reason, Dad needed a SIM card so I gave him mine. So I bought myself a new one, a Telenor card, pay-as-you-go, number 91 85 95 08. I must have been around 12 at the time. And that number served me very well. Fast-forward 6 years, to late March, 2006. I had just turned 18, and one of the first things I wanted to do was get a massively cheaper, proper subscription. I’d been procrastinating it for a bit - as I am wont to - but when a nice gentleman at the subway station told me about a nice new company called SEA Norway and their wonderful prices, I accepted. Big mistake. Here begins the saga of woe… Useless Vitel company logoFor a good while, I was very happy with their service - even though GPRS data never worked, I never used it anyway. Their name changed to Vitel somewhere along the way. I received a phone call from an enthusiastic salesman who told me how much cheaper Vitel’s power subscription is. It seemed very reasonable, so I went for it. One of the arguments he gave me was that I could collect all those utilities into one bill. I told him that I couldn’t have this: Since I’m a live-alone student, my power bill is covered entirely by the city council, so I need a proper bill to hand over every month. “Sure, that’s no problem”. I check and double-check that, and I stress it. Sure, no problem. OK, deal. Big, big mistake. They draw 2700 Kroner from my account, using the automated pay system (AvtaleGiro) that I had set up for the phone bill. 2700 Kroner (530 US Dollars) is a lot of money, especially when your monthly income, 5000 Kroner (1000 US Dollars) barely covers the necessities. So I cancel the autopay system. (I still have not received that money!) 23rd of January, my subscription is cut, since I hadn’t paid the bill. They hadn’t mailed it to me, and I’d told them I’d cut the AvtaleGiro, but I simply did not receive the bill. So I call them up in February some time and try to figure out a solution. “You didn’t pay the bill, so the subscription is closed.” -”So if I pay the bill, will you reopen it?” -”No, sorry. It is closed now.” -”So then I’ll have to transfer the number to another provider?” -”No, you can’t do that; it’s closed now.” -”You can’t refuse to transfer the number, that’s simply not legal.” -”Sure it is, the subscription’s been closed.” Urgh. So some time in the middle of March, before The Gathering, I call NetCom up: I want to transfer my number to them, please. Sure, no problem. No, this number is registered as active in our system. All is well. We’ll make the switchover on March 28th, and you will get your SIM card in the mail: It should be in your box by then. Fantastic! The 28th of March came and went with no great ceremony, not even a SIM card. No switchover occurred. I give them the benefit of the doubt for more than a fortnight. Some time in April, then, I get sufficiently annoyed to call them up and ask what had happened. “Oh, no, it seems Vitel is using the old transferral system”. -”OK?” -”Yes, you will have to send us a form with your signature by fax to carry this transfer out.” -”Sigh, alright. Where can I get this form?” -”I’ll email it to you.” Great. So I get the form. May was very busy, so I didn’t get an opportunity to fill the form out, and by now I was getting fairly tired of the whole thing anyway! The thing that finally got me, however, was a bill from Vitel - on the “closed” subscription - for not only the monthly fee, but also several kroner of SMS “content services” (Read: ring tones) and GPRS data (Which never even worked even when the subscription did!) So last Tuesday, June 3rd, I send them the damned fax. Wednesday, June 4th, I call NetCom up and ask what the status is of my subscription. “Oh, yes, this is now active with us.” -”Really?!” (I was happy in the way that only someone who’s gone 5 months without a working phone could be.) -”Sure, we’ve sent the SIM card to Bekkeveien (something).” -”…Bekkeveien? But I’ve never lived at any Bekkeveien. My last name is Bekkedal, but..” -”Alright, then that’s probably a mistake. What you can do, though, is go to a NetCom dealer and pick up a new SIM card.” -”Perfect! I’ll do that, then.” (I stress “pick up” for a reason which will later become apparent.) Goodie. I’ll get a working phone! I was very happy as I took the tram down to Aker Brygge and looked everywhere for the damned NetCom dealer. Having finally found it, I happily stride inside and ask for a SIM card for my subscription. “Sure. That’ll be 200 Kroner.” Argh! I hadn’t brought enough money! She did say pick up. Oh well. Aker Brygge is a center of business, I’m sure there are other shops. So I ask to use one of their phones to call NetCom. Sorry, no phones you can use. Sigh. Fine. Do you know of any phone booths around? They didn’t, but I found one anyway, finally. I deposit the necessary $1 for a local call (See, this is the problem with having a cellphone company run the phone booths!) and dial NetCom customer service. They take my number and - Sorry, that number isn’t yet active, it’s still registered to Vitel. ARGH! Fine. So what’s standing in the way? Well, Vitel can’t refuse it - (thanks, I know that already) - but the form that I was sent wasn’t the right form. The form I need to send is from Vitel to Vitel, authorizing the transfer.While I’m trying to hold a conversation with the customer support guy, some drunk guy is accusing me of stealing phones, and begins reading out loud the sponsor company names on the back of my DebConf T-shirt. But Vitel has already told me that I cannot transfer this number, under any circumstance! They consider it closed! “Yes, but you have to talk to Vitel.” -”I have!” -”Well, we can’t really do anything…” Alright. So I hang up and I try to call Vitel. But that doesn’t work: They closed at 3 PM!. So finally tired of Netcom failing to port my number for a while, I call Telenor. I talk to a very cheerful and helpful lady who admits to not knowing about the particulars of that situation. Fantastic! At least she isn’t going to fib it! She says that she has a vacation Thursday, but she can call me back on Friday after discussing the matter with the number transfers department. Wow! She’s actually going to confer with the relevant expertise! This is too good to be true! So she said she’d call me back Friday at 1PM with a status update. As I am writing, it is Friday at 11:01, I am anxiously awaiting the telephone call. But now I get a funny SMS message:
    From: NetCom Message from NetCom: Your number has been ordered by another operator. We would like to remind you that violation of the commitment period will be billed to the order of 1500 kroner. For questions, call 05050.
    My subscription with Vitel was made in 2006, more than the 12 months I and knowledgeable friends believe to be the maximum legal commitment period (I doubt I would have agreed to more anyway!). Besides, the text message keeps saying that it’s from NetCom. So my deduction is: It’s from NetCom, who I have registered an active subscription and are now complaining that I’ve moved the subscription I’ve opened with them (but which they have not successfully moved from Vitel), over to Telenor. I don’t think this is an unfair conclusion. So I call NetCom customer service - which to NetComs infinite credit is a number you always can call, almost like 112 (technically it’s accomplished completely differently, but you get my point. The number is never blocked in a blocked subscription.) And they say it’s not from NetCom, even though it says it’s from NetCom. So I’ll keep both of my readers posted on the developing situation. At least the text message indicates that something is happening with my subscription. I just want the dialtone!

    5 June 2008

    MJ Ray: World Environment Day

    Today is World Environment Day 2008 and also Recycle Week in the UK. Locally, our poor recycling record has attracted attention. The main Somerset CC area averages 50% recycling and composting, while North Somerset only does 36%. This will cost us money, as explained under the subtly-titled local newspaper article 'REDUCE WASTE NOW OR FACE A 12M FINE' It's very annoying to read a Conservative councillor say
    "The message we really have to get out to people is that this is their problem too."
    We know it's a bloody problem. Have you tried to use your stupid recycling system? Criticism of the stupid collections is strangely absent from the local Lib-Dem's Mid table place shouldn't be good enough when it comes to recycling (Mike Bell) and I can't even find recycling mentioned on local Labour sites - I think those two groups may have been in coalition when the stupid system was introduced. It's left to local residents to explain the problem. Local waste collections have been simplified recently but it still seems bloody awkward, taking different types of waste to different places. The most recent edition of the council's North Somerset Life magazine explained that they don't collect plastic bottles from the doorstep because it would add 10 per year to our local tax. I'm damn sure it already costs me more than that to store them and ferry them into town over a year. The only people rewarded by saving that 10 seem to be those who live near the town centre collection point, drive to that Tesco anyway, or don't recycle plastics. Why reward them? Speaking of Tesco... those national and local favourites have applied to build Two more Tesco stores for Weston [The Weston Mercury] including one directly opposite another supermarket. The application number is 08/P/1230/F if you want to respond. The planning system seems my main chance to protect my local environment at the moment. Next Monday evening at 7.30pm, 08/P/1070/F - Erection of 5 storey building to provide 14 flats, restaurant and office with basement parking following demolition of restaurant will be considered by a Kewstoke village council planning meeting in the village hall. As I understand it, the public may make statements at the start of the meeting and 2 Kewstoke Road is currently the Castle.

    2 June 2008

    Russell Coker: Moving a Mail Server

    Nowadays it seems that most serious mail servers (IE mail servers suitable for running an ISP) use one file per message. In the old days (before about 1996) almost all Internet email was stored in Mbox format [1]. In Mbox you have a large number of messages in a single file, most users would have a single file with all their mail and the advanced users would have multiple files for storing different categories of mail. A significant problem with Mbox is that it was necessary to read the entire file to determine how many messages were stored, as determining the number of messages was the first thing that was done in a POP connection this caused significant performance problems for POP servers. Even more serious problems occurred when messages were deleted as the Mbox file needed to be compacted. Maildir is a mail storage method developed by Dan Bernstein based around the idea of one file per message [2]. It solves the performance problems of Mbox and also solves some reliability issues (file locking is not needed). It was invented in 1996 and has since become widely used in Unix messaging systems. The Cyrus IMAP server [3] uses a format similar to Maildir. The most significant difference is that the Cyrus data is regarded as being private to the Cyrus system (IE you are not supposed to mess with it) while Maildir is designed to be used by any tools that you wish (EG my Maildir-Bulletin project [4]). One down-side to such formats that many people don’t realise (except at the worst time) is the the difficulty in performing backups. As a test I used LVM volume stored on a RAID-1 array of two 20G 7200rpm IDE disks with 343M of data used (according to “df -h” and 39358 inodes in use (as there were 5000 accounts with maildir storage that means 25,000 directories for the home directories and Maildir directories). So there were 14,358 files. To create a tar file of that (written to /dev/null via dd to avoid tar’s optimisation of /dev/null) took 230.6 seconds, 105MB of data was transferred for a transfer rate of 456KB/s. It seems that tar stores the data in a more space efficient manner than the Ext3 filesystem (105MB vs 343MB). For comparison either of the two disks can deliver 40MB/s for the inner tracks. So it seems that unless the amount of used space is less than 1% of the total disk space it will be faster to transfer a filesystem image. If you have disks that are faster than your network (EG old IDE disks can sustain 40MB/s transfer rates on machines with 100baseT networking and RAID arrays can easily sustain hundreds of megabytes a second on machines with gigabit Ethernet networking) then compression has the potential to improve the speed. Of course the fastest way of transferring such data is to connect the disks to the new machine, this is usually possible when using IDE disks but the vast number of combinations of SCSI bus, disk format, and RAID controller makes it almost impossible on systems with hardware RAID. The first test I made of compression was on a 1GHz Athlon system which could compress (via gzip -1) 100M of data with four seconds of CPU time. This means that compression has the potential to reduce the overall transfer time (the machine in question has 100baseT networking and no realistic option of adding Gig-E). The next test I made was on a 3.2GHz Pentium-4 Xeon system. It compressed 1000M of data in 77 seconds (it didn’t have the same data as the Athlon system so it can’t be directly compared), as 1000M would take something like 10 or 12 seconds to transfer at Gig-E speeds that obviously isn’t a viable option. The gzip -1 compression however compressed the data to 57% of it’s original size, the fact that it compresses so well with gzip -1 suggests to me that there might be a compression method that uses less CPU time while still getting a worth-while amount of compression. If anyone can suggest such a compression method then I would be very interested to try it out. The goal would be a program that can compress 1G of data in significantly less than 10 seconds on a 3.2GHz P4. Without compression the time taken to transfer 500G of data at Gig-E speeds will probably approach two hours. Not a good amount of down-time for a service that runs 24*7. Particularly given that some time would be spent in getting the new machine to actually use the data. As for how to design a system to not have these problems, I’ll write a future post with some ideas for how to alleviate that.

    1 June 2008

    Enrico Zini: Playing with a Holux M-241

    Playing with a Holux M-241 Here is how to download tracks from the Holux M-241:
    gpsbabel -t -r -w -i m241 -f /dev/ttyUSB0 -o gpx -F  date +'%Y-%m-%d-%H%M%S' .gpx
    
    
    It might work also via bluetooth, but I have not tried yet. Now, until M-241 support will be released in a stable version of gpsbabel, here is how to compile the version from CVS. Get the sources:
    cvs -d:pserver:anonymous@gpsbabel.cvs.sourceforge.net:/cvsroot/gpsbabel login
    cvs -z3 -d:pserver:anonymous@gpsbabel.cvs.sourceforge.net:/cvsroot/gpsbabel co -P gpsbabel
    
    
    Untar this to debianise the sources. If it does to compile because of some errors in lmx.c, apply this patch. Then you can install the resulting package and (hopefully) be happy. Note, after downloading the logs, gpsbabel currently turns on logging. Here is a patch to disable that behaviour. Finally, if you want to hack around a little on the unit, you can play with mtkbabel: the source code is simple, and most of the MTK protocol is implemented, so you can easily feed your own commands to the MTK. Documentation about the commands can be found here: The links are taken from a post in the GPSPasSion forum. For a source of ideas of what commands you can send, you can look into the source code of BT747. For example, to set the M-241 to 2Hz fix, you can add this to mtkbabel:
    packet_send('PMTK300,500,0,0,0.0,0.0');
    $ret = packet_wait('PMTK001,300,3');
    
    
    And if you want to set the MTK to an insane 5Hz fix rate, to take really fine grained gpx traces with your laptop, you can use this:
    packet_send('PMTK300,200,0,0,0.0,0.0');
    $ret = packet_wait('PMTK001,300,3');
    
    

    31 May 2008

    Christian Perrier: News from samba packages development

    There has been some hype around samba this week. On May 21st, the Samba Team released Samba 3.0.29, an update from their stable branch. The Debian package maintainers quickly built and uploaded a package for unstable/lenny the very same day. Then, on 27th, we got notified of CVE 2008-1005, which affects both etch, lenny and unstable (not the 3.2 versions we have in experimental). I contacted the security team and, after some brief discussions, I uploaded with their blessing, a fixed package to stable-security (3.0.24-6etch10). Then, I immediately uploaded samba 3.0.30 to unstable (1:3.0.30-1), that version being the Samba Team response to the security issue. All this lead to DSA 1590. Up to now, everything perfect. In the meantime, Karolin, the Samba Team's release manager announced the Releace Candidate 1 of Samba 3.2.0. So, yesterday (Saturday), I started building it....which succeeded. Then I started to screw up..:) Instead of uploading that version to experimental, as planned, I uploaded 1:3.2.0~rc1-1 to unstable. Yes, AGAIN (I already did that in April). There's no excuse for that, except distraction and failure to use our tools properly ("dch -r"). Thankfully, I noticed that nearly immediately, but not immediately enough. So, I had to re-upload 3.0.30 to unstable (which is urgent: that's a security fix) and increase the epoch (we already had one as this is the second identical screwage): 2:3.0.30-2. Sorry, autobuilders... Yesterday evening, I uploaded 3.2.0-rc1 again to experimental (2:3.2.0~rc1-2). Then, finally, I went on the last bit of remaining work: spend some time on Jelmer Vernooij's samba4 packages. Jelmer is one of the two lead developers of Samba 4, a huge masterpiece still under heavy delveopment. To give it more exposure, it is planned to upload it a few times in experimental, then move it to unstable, block it so that it does not enter testing, and make some noise about this. I finally uploaded samba4 this morning (triple checking that I was uploading to experimental!) and samba4 4.0.0~alpha4~20080522-1 is now waiting for NEW processing (which might take time: this is a big piece of new stuff). I'm sad for the screwage because, apart from it, that was nearly a perfect process. But, anyway, I'm still proud of the result: 4 successful uploads for samba in a little more than a week (which included 4 day away). It could even have been 5 as the samba version in sarge is affected by the security issue, but we don't support sarge anymore.

    30 May 2008

    Cyril Brulebois: Blender 2.46 in experimental

    Just a quick word: Blender 2.46 has been uploaded to experimental a week ago, and is available on the following architectures:
    $ rmadison -s experimental blender
    blender   2.46+dfsg-1    experimental   source, alpha, amd64, i386, ia64, mipsel, powerpc, s390
    
    The move to unstable is planned once ftgl 2.1.3 has been released and uploaded to unstable. This release s highlight: Big Buck Bunny, as seen on the splash screen: Blender 2.46 - splash screen

    29 May 2008

    Joerg Jaspert: I'm still alive

    While I have been a little quiet with ftpmaster posts lately - I am still alive. I just had some other bits to do and didn’t get large enough changes done that would warrant an own post. Since my last post earlier this months there have only been 3 notable changes, namely And only a few minutes ago I sent a “Bits from” mail to our debian-devel-announce mailinglist, which includes a summary of the recent activities and changes in our team, with the addition of Thomas and Kalle, me getting FTPMaster and Anthony and James resigning. We also ask for volunteers which we might add to the team later on. See, I write might. We keep the right to reject people, for whatever reason. Hopefully there won’t be many rejects, but still. And we will see how the volunteers get the tasks done we plan to assign to them. You can expect to see a little more in the near future from me again, I don’t plan on staying that quiet forever.
    And in other news - I recently got another group added, mirroradm, as it did help to fix some mirror issues with packages.debian.org. With that I also got the mirrors@ alias additionally routed into my INBOX. Thats a nicely spammed role address and my tip for you, if you should ever get offered access to (any) role address - run as fast as you can, unless you have very good spam filters! :) Luckily my filters are pretty good, as I’m on a good number of role addresses for a long time already, only a few dozen spams getting through a day (and a thousand filtered away). Comments: 0

    Next.

    Previous.