Search Results: "will"

10 December 2008

Alexis Sukrieh: My two cents about the RubyGems polemic

Hey planet, it s been a very long time since I took my e-pen to write something down here, my work is taking pretty much all my time nowdays. Anyway, it seems that there is sort of a polemic rising here and there about RubyGems and its packaging philosophy. I d like to take that opportunity to contribute to the debate with the experience I had at Yoolink. Yes, we provide Ruby on Rails applications and are defacto dealing (or trying to) with Ruby and its gems. Wouter enlightened the main reason why it s such a mess to put into production a Rails app and to maintain it : the Ruby on Rails ecosystem is unstable. If you start writing your app, there are very big chances that when it gets mature for production mode, one or more of the Ruby libs you depend on will have a new version. And sadly, chances are there too that one of those libs will provide an API change. This can happen in a lot of places in a Rails app, because of its modular structure: plugins, gems or even Rails core upgrades (try to upgrade a real-working Rails app from 1.x to 2.x if you like tricky games). I agree with Joss when he said that Debian packages and Ruby gems are not designed for the same purpose, the latter are for a developer ecosystem and the first suitable for long-working production ecosystems. That s why setting up a complex Rails app in production is a nightmare : no Debian packages for all your Ruby needs on the first hand and API-unstable and hard-to-upgrade gems on the other hand. At Yoolink, the only working way we ve found to maintain in production our Rails apps is to freeze pretty much every Ruby-thing we can. Our production servers are then Rails/Ruby free (nothing installed system-wide) and the apps provides all their Ruby dependencies in their vendor/ directory. You can t do that for native gems but that works for 90% of a random Rails app s dependencies. To conclude, I d say that most of these issues come from the philosophy of the Ruby/Rails people : they don t fear to change anything they want from one release to another, just take a look at the gettext support in Rails 2.2 and you ll understand what I mean . It s like they don t even take in mind that people are now using their stuff, in real world, with customers relying on their apps. It s not a pet project anymore guys, if Rails want to remain the killer app it is, it will have to be someting stable and maintainable.

David Welton: Slicehost vs Linode

Since hosting is the major expense for DedaSys LLC, and obviously a critical part of much of what I do, getting the right one is very important. Naturally, "the right one" for me may not be the right one for everyone. I am a fair sysadmin for small numbers of machines, so I don't mind doing that myself - I don't need, nor want to pay for hand holding.

Since this post has become fairly popular, I am going to link to my Linode affiliate URL if you'd like to sign up with Linode - thanks! Other than running my sites there, I have no other affiliation with Linode. And, if you're more a fan of Slicehost, here is my referral page on Slicehost.

I recently moved everything over to Slicehost on the recommendations of friends, and am so far fairly happy with the experience. It's cheaper, simpler, and more flexible than Layered Tech was. To boot, I also like the fact that Slicehost is run out of St. Louis, Missouri, rather than some really expensive tech center like San Francisco or Boston. Hosting is basically a commodity, and much like you wouldn't want to put a factory in San Francisco, you don't want your hosting their either. However, I discovered something annoying about Slicehost: they use x86_64 servers, which, per se, doesn't really matter to me - I use open source code that can run on any number of architectures. The problem is that this particular architecture uses more memory than plain old x86. Significantly more. My Layered Tech server with 1 gig of memory was hitting the swap space a bit, but the same code on a slice was swapping quite heavily, despite the fact that I'd moved PostgreSQL off to its own slice. Since I pretty much exclusively run Rails, I decided to look into Phusion's "Ruby Enterprise Edition", which is basically just some nice hacking on Ruby's garbage collection mechanism. What they've done is nice, and I may end up using it, but ultimately it's buying me space that I would also gain from simply moving back to x86. With that in mind, I decided to take another look at what appears to be emerging as one of Slicehost's main competitors, Linode, who *do* use x86 servers. Here are my results, which are admittedly not all that scientific, but what the heck - you're getting them for free, and they were pretty quick to whip up. Service, Support, Setup But before we begin with the numbers, I want to add a few words of caution. One of the big things about hosting, to me at least, is how they deal with unexpected random negative events. Connections going down, disk breakage, DoS attacks, and so on. It's really hard to get an idea of just what sort of people you're dealing with until you've "ridden the river together". I don't really know what sort of response times, uptime, and all that either Slicehost or Linode have, so there are potentially some big intangibles there that are not quite as easy to draw pretty charts for. In terms of the console/setup/management tools, I liked Slicehost's simplicity more - Linode gives you more options, but they're a bit fiddly and cluttered feeling at times. For instance, Linode lets you pick how much swap space to give your disk image, but doesn't include a bit of javascript to balance out the swap and regular partitions, so that if you type in a larger swap number, you have to do the math and subtract that from the other number. Annoying, but not a big deal. Linode lets you pick various data centers in the US (no Europe, yet). Slicehost gives you the option to do backups of your disk images, which is nice, and something that Linode lacks. Comparing plans Since I'm interested in comparing hardware and machines I actually have access to, here are the plans I am currently signed up with, and the salient numbers:

	Slicehost 1024	Slicehost 512	Linode 720
Memory	1024	512	720
Bandwidth	400	200	400
Disk	40	20	24
Cost	70	38	39.95

Memory / Dollar:

Bandwidth / Dollar:

Disk / Dollar:

Linode mostly comes out ahead, but not by that much, except for bandwidth. However, let's take a crude look at x86_64 vs x86 memory usage. First, ./script/console (a Rails console) for the same code base on two different machines, showing the virtual memory size (VSZ) and resident set size (RSS):

And just to look at something else that's a bit smaller, I ran the following script:

puts File.open("/proc/self/status", "r").read

And got the following data:

Being charitable with those numbers, we get x86 taking up 77% the memory that x86_64 does. So let's add that to our comparison of how much memory you're getting per dollar on Slicehost and Linode:

Wow! That's a fairly significant difference. Performance That's most of what we can glean from published numbers, and running a few simple experiments. However, there's another factor that's important: performance. "Virtual Private Server" systems, or simply VPS's, got a bad name in the past because they were "overbooked" - too many virtual servers competing for the same CPU resources on one machine. Slicehost and Linode both look like they want to avoid that kind of bad reputation, and so far all the systems I have used have felt snappy and responsive. Now here's where we get really unscientific... I decided to try and do something, even if it wasn't much, to get some kind of objective measurement of what kind of CPU I was getting on the machines. This is really unscientific, because the machines do have other duties (I don't have the cash, time or inclination to set up random servers just for testing), and of course, who knows what else is sharing the computer with my systems: if you get a relatively unused computer, you can apparently pick up extra cycles for yourself, over and above your guaranteed minimum. But c'est la vie, and so I decided to do some numbers anyway. I picked a C implementation of mandelbrot from the language shootout more or less "for the hell of it" (I'm going to keep repeating "unscientific" and invite someone to do better tests than I have). I ran this code every 20 minutes for a couple days, and then averaged out the run times:

As expected, the higher power Slicehost 1024 machine wins out in terms of raw speed (less time is better), but not by that much over the Linode system. Indeed, when we factor in the relative prices, Linode comes out ahead, again:

So it looks like we're not paying a CPU penalty for having more memory, bandwidth, disk, etc... Notes

Nothing in these statistics is indicative of future directions that both Linode and Slicehost might take. Slicehost was recently acquired by Rackspace, and that could have effects on the service they provide. Maybe one or both will come under financial pressure and start "overbooking" their servers.
The spreadsheet I used is available via Google docs, for the curious. I did most of the work in OpenOffice, and then uploaded it though, because working with the web spreadsheet was kind of painful. Unfortunately, you can't select non-contiguous data ranges in the Google spreadsheet, so the labels are interspersed with the data. Yuck. Also, Google docs doesn't handle 'time' values nicely, which is what the performance data was originally. The spreadsheet is here: http://spreadsheets.google.com/pub?key=pVDeZF6ire2YwATBYb4mBKQ
The standard deviation on the processing times is higher for both slices than the linode system. Not quite sure what that means in terms of what's going on under the hood, but I thought it was interesting.

Russell Coker: New version of Bonnie++ and Violin Memory

I have just released version 1.03e of my Bonnie++ benchmark [1]. The only change is support for direct IO in Bonnie++ (via the -D command-line parameter). The patch for this was written by Dave Murch of Violin Memory [2]. Violin specialise in 2RU storage servers based on DRAM and/or Flash storage. One of their products is designed to handle a sustained load of 100,000 write IOPS (in 4K blocks) and 200,000 read IOPS per second for it’s 10 year life (but it’s not clear whether you could do 100,000 writes AND 200,000 reads in a second). The only pricing information that they have online is a claim that flash costs less than $50 per gig, while that would be quite affordable for dozens of gigs and not really expensive for hundreds of gigs, as they are discussing a device with 4TB capacity it sounds rather expensive - but of course it would be a lot cheaper than using hard disks if you need that combination of capacity and performance. I wonder how much benefit you would get from using a Violin device to manage the journals for 100 servers in a data center. It seems that 1000 writes per second is near the upper end of the capacity of a 2RU server for many common work-loads, this is of course just a rough estimation based on observations of some servers that I run. If the main storage was on a SAN then using data journaling and putting the journals on a Violin device seems likely to improve latency (data is committed faster and the application can report success to the client sooner) while also reducing the load on the SAN disks (which are really expensive). Now given that their price point is less than $50 per gig, it seems that a virtual hosting provider could provide really fast storage to their customers for a quite affordable price. $5 per month per gig for flash storage in a virtual hosting environment would be an attractive option for many people. Currently if you have a small service that you want hosted a virtual server is the best way to do it, and as most providers offer little information on the disk IO capacity of their services it seems quite unlikely that anyone has taken any serious steps to prevent high load from one customer from degrading the performance of the rest. With flash storage you not only get a much higher number of writes per second, but one customer writing data won’t seriously impact read speed for other customers (with hard drive one process that does a lot of writes can cripple the performance of processes that do reads). The experimental versions of Bonnie++ have better support for testing some of these usage scenarios. One new feature is measuring the worst-case latency of all operations in each section of the test run. I will soon release Bonnie++ version 1.99 which includes direct IO support, it should show some significant benefits for all usage cases involving Violin devices, ZFS (when configured with multiple types of storage hardware), NetApp Filers, and other advanced storage options. For a while I have been dithering about the exact feature list of Bonnie++ 2.x. After some pressure from a contributor to the OpenSolaris project I have decided to freeze the feature list at the current 1.94 level plus direct IO support. This doesn’t mean that I will stop adding new features in the 2.0x branch, but I will avoid doing anything that can change the results. So in future benchmark results made from Bonnie++ version 1.94 can be directly compared to results that will be made from version 2.0 and above. There is one minor issue, new versions of GCC have in the past made differences to some of the benchmark results (the per-character IO test was the main one) - but that’s not my problem. As far as I am concerned Bonnie++ benchmarks everything from the compiler to the mass storage device in terms of disk IO performance. If you compare two systems with different kernels, different versions of GCC, or other differences then it’s up to you to make appropriate notes of what was changed. This means that the OpenSolaris people can now cease using the 1.0x branch of Bonnie++, and other distributions can do the same if they wish. I have just uploaded version 1.03e to Debian and will request that it goes in Lenny - I believe that it is way too late to put 1.9x in Lenny. But once Lenny is released I will upload version 2.00 to Debian/Unstable and that will be the only version supported in Debian after that time.

[1] http://www.coker.com.au/bonnie++/

[2] http://violin-memory.com/

MJ Ray: Environmental Policies and Computing

I’ve not written about this for a while, but I’ve just started seeing Apple’s Greenwash adverts, so I went and found the latest Greenpeace electronics report which says:

“Apple’s score increases slightly to 4.3 points, but the company drops to 14th position. Apple scores well for putting products on the market whose key components are free of brominated flame retardants (BFRs) and PVC vinyl plastic [...] All Apple products should be free of these substances by the end of 2008, which will challenge other PC makers to follow their lead. [...] Apple scores poorly on most e-waste criteria, except for reporting a recycling rate in 2006 of 18% as a percentage of sales 7 years ago. It does slightly better on energy criteria for disclosing the carbon footprint of every model of product although not exactly what is being evaluated in the criterion.”

I feel that it’s hard to push the large electronics companies by direct action and purchasing decisions (although I will continue to try). Regulations must be introduced too and now the EC has issued a Code of Conduct on Data Centres Energy Efficiency [TreeHugger report]. TTLLP has been slowly getting out of data centres for the last few years - between the toxic e-waste in servers and rising energy costs, with most of our competitors ignoring both of them, it was too hard to do the right thing and be profitable. I think the market is ready for a step-change to green data centres and we can’t do that alone. Expect a new TTLLP-branded venture early next year… Finally, tomorrow, on 11 December, our governments will decide Europe’s response to climate change for the next 12 years. Take 2 minutes to tell them to keep global warming below 2 C.

9 December 2008

Gerfried Fuchs: Better backports.org Support

Since last weekend backports.org has two more services supporting tracking of what's going on. I will list them in chronological order: Tracking of Security issues in backports packages: This was one of the many topics that was discussed at the Security Team meeting in Essen at the end of last month and Florian Weimer implemented a first version of tracking security issues within backports.org. It currently compares the backported version against the one fixed for unstable, so at the moment it still has some false-positives (e.g. libspf2 because the fix was taken from lenny-security), but this is still a big step forward and helpful to track outstanding issues here, too. Diffs between etch-backports and lenny: Something similar was available at some other places before but strangely got discontinued. In here you can see which packages in backports are older (meaning only debian revision difference), outdated (newer upstream version in lenny) newer (backported from unstable?), have a wrong version schema or are not available in lenny (propably even removed from unstable). It will hopefully help people to get their backports in sync. At least it's an indicator of how well the packages are tracked. Hope you consider them already as useful as myself, even though there is obvious space for improvements. But they are both quite helpful already in their current state so that shouldn't hinder you from using them. :)

Peter Eisentraut: When non ASCII names will finally work? (part 2)

no comment

Bastian Venthur: Lenny s release date III

Two months since my last status update of my initial guess for Lenny s release date, time for an update: My guess for Lenny's release III

I don t know what the current official release target for Lenny is, but I still think we re heading towards Spring/Summer 2009. The last related mail on -devel-announce is from October 2008 from Alexander who hopes that we ll release Lenny at least by the end of 2008. I ve read an article with an interview with Steve McIntyre somewhere on The Register where he said:

Bastian s being a little bit pessimistic based on the data he s looking at, but as far as I know, he hasn t spoken to any of the release team. As far as Lenny goes, we re almost there, he told The Register.

That interview was in the beginning of October. It doesn t bother me too much, that we release late. What bothers me is that we don t correct our estimated release dates even when it s obvious that there is no way we will release in time. It s really sad that we keep our users in the dark about the fact that we can t release in time until it s already too late. I believe many users would appreciate if we d honestly say: We don t know when we release, but currently we have x RC bugs open and looking at the last releases we think it will take roughly n months to fix them and release, instead of the good old We release when it s ready . Another important point: Unstable is frozen for almost five months now. If you buy a current laptop, chances are hight that it might not fully work with Debian bleeding edge unstable, since it s horrible outdated. Some of the newer stuff can be obtained from experimental if you re brave enough, but if you re not, unstable is getting more and more unattractive compared to other distribution s current versions. This is really something we should try to fix after we released Lenny.

Lucas Nussbaum: The Project membership procedures GR

So, the call for vote on the “Project membership procedures” GR was sent. Since I took part in the discussions that lead to that GR, I feel obliged to explain my point of view. What this GR isn’t about
This GR isn’t about the merits of Joerg Jaspert’s proposal. I personally don’t think that the changes he wants to implement are what we need in Debian, but even if I fully agreed with those changes, I would still disagree with the process that led to those changes. Also, this GR isn’t about Joerg Jaspert himself. I deeply respect him and his awesome work for Debian. And I’m sure that this vote could have been triggered by decisions announced by someone else. What this GR is about
I think that the question we are asked to vote on is:

Some people in Debian have a special role, and special privileges. Does it allow them to change important aspects of our project, without the approval of all the other developers?

Or, put differently:

Where is our limit between Doocracy and Democracy?

I’m all for listening to developers who contribute(d) a lot. I think that most people, when skimming through posts in a big thread on -devel@, will look for posts from people who did a lot of work for Debian, or who usually post interesting ideas. However, that doesn’t mean that those people should feel empowered to decide changes to important parts of our project, without first discussing those changes with all other developers, and making sure that those changes are consensual. So, I hope that we will send a clear message that says: you are very welcome to propose changes, but please make sure that they are properly discussed within the developers before they are implemented. Joerg’s initial mail on d-d-a, with a few s/decision/proposal/, and a final question phrased like “Any comments? Suggestions? Other ideas?”, could have been the start of an interesting discussion about how we can integrate non-developing contributors in Debian. And if too many developers disagreed, we could have had an opinion poll (it would be great to have an infrastructure for running polls inside Debian, like what Jeroen did three times a long time ago), or even a GR, like we did for Debian Maintainers. For this actual vote, there are two similar options on the ballot (choices 1 and 2, respectively “amendment text A” and the original text on the vote page). I proposed this amendement (choice 1) because the original text is unclear, and mixes several different questions, forcing voters to answer several (secondary) questions at once, like:

Do we recognize that there are contributors that can’t become members of the project currently, and that should?
Do we want to thank DAM for bringing this up? (implicit question or not: … now that we are trying to release?)
Do we want DAM to work on a new proposal?

The amendment aims at going straight to the point. And yes, it is only asking the DAMs, but if the developers ask the DAMs not to do something, and the DAMs do it anyway, we clearly have a bigger problem. I believe that the way those decisions were announced was a mistake (that unfortunately none of the people involved have been willing to admit), not that the DAMs are trying to take-over Debian, so there’s no need for a stronger wording.

Wouter Verhelst: The advantages of a home-grown comment system

A while ago, I was investigating the possibility to move blog backend to something else; something like, say, drupal, so that I wouldn't have to maintain it too much myself. For those who care, this blog is a very strange contraption based on subversion, blosxom, php, and a postres backend for the comments. I've now dropped that migration thought, for a very simple reason: comment spam. A while back, the spammers had found my blog, and started placing comments using a bot. They might think that these comments will eventually show up on my website, but that's wrong; I always moderate away spam. First, they posted a few comments. I changed a few details about the form, and the bots (who only went to the POST URL, never checking whether the form they requested was unchanged) failed miserably. Then, they posted loads and loads of comments, in a weird attempt to overrun me with comments. One psql run and one DELETE statement later, those comments were gone -- and one minor form change later, the bots failed miserably, again. This went on for a few months, until the stupid spammer finally understood that he was wasting loads of his time in trying to post something on a website that wasn't ever going to enable his comments, anyway. Or so I thought. You see, the thing they're really after is my pagerank. In itself, grep.be is nothing special -- but planet.grep.be is, and apparently some of that pagerank score trickles to the other grep.be subdomains. Or, at least, that's what the spammers seem to think, in case it isn't (I wouldn't know). When the comment spam on my blog had been silent for a few weeks, I found that the spammers had found my gallery installation, and had posted some several thousand comments there. One DELETE statement later they were gone, but a few hours later some of them were back. I don't know the gallery source code well enough to pull the same trick here (disable comments until manual moderation), and unless I want to do something like using a CAPTCHA on my gallery site (I don't) or disabling comments altogether (I don't, either), I'll just have to resort to removing these comments from the database. At least by doing that, I can remove them all in one go. And at least the spammers are easily recognizable. For now. Perhaps I'll have to look into changing gallery's source code after all... or, perhaps, just use something else. Sigh. Stupid spammers

Josselin Mouette: RubyGem is from Mars, apt-get is from Earth

There is again some ongoing debate about Ruy gems and how to integrate them cleanly into the operating system. Let s put aside the fact such tools are useful on Windows and MacOS which don t have decent packaging systems. This is not a reason to dictate stupid requirements for other operating systems. Still, even without that, gems, eggs and the like are great tools for development; tools that developers can t work without today. They allow to easily setup a development environment with everything needed, at the latest versions. However they were not meant for integration and production environments, and will never be suitable for them. We re living in another world indeed: a world where servers must be up 24/7, where security issues flood your inbox every day, where you need dozens of servers running the very same software. This world is the Earth. Python eggs and Ruby gems were designed for the same target audience and with similar requirements; as such, they share the same deficiencies when it comes to other audiences:

They replace the operating system s packaging tool. This is very nice to not wait for the packagers when you want the latest development version, but you lose track of what is actually installed. In production, you need to know exactly what s installed and at what version. You also need to know what you will need to update in case of a security issue.
They install packages automatically on the system. This is really the biggest WTF; there are some reasons to do that to begin with, but they are all wrong.
They use a specific file format. This requirement comes mostly from the Windows world, and it really makes no sense on Unix. It makes it much harder to integrate complex applications using components in different languages.
They ship all files at the same place, happily violating the FHS and making it even harder to integrate with other things.

Python tools (setuptools and easy_install) have shifted their philosophy and brought a few small improvements which allow us to easily work around issues 1-3. These tools now include all you need to build a development environment in any directory that will not have an impact on the system packaging, making a clear distinction between the two targets. Furthermore, the metadata is shipped separately and can safely be included in a distribution package. Similarly, modules will not be installed automatically on your system. I have yet to see such a move from Ruby developers. However, what we are still missing is something along the lines of dh_make_perl: starting from a CPAN module, it will generate a clean and working Debian package in a few minutes. There have been some similar attempts for Python, but nothing as efficient. And I am certainly guilty of not having done something in this area, as one of the most knowledgeable of issues we encounter while packaging Python modules. As a note, I should add the underlying issue behind all of this is still the same: the NIH syndrome. Developers are reinventing the wheel, engine and transmission. Which is not that bad per se, but by not looking at existing solutions for the problem of making a car move, they are inventing a square wheel, a steam-powered engine and a superconductor-powered magnetic transmission. For example, while trying to fix the 4th of items listed earlier and they are really willing to fix it and several more general issues, setuptools developers forgot to look at the existing solutions:

API versioning in /usr/include/package-X.Y
ABI versioning of libraries
Build/install-time requirements gathering using an extensible format with pkg-config
Automated generation of config.h containing all system-specific macros (including paths), with autoconf
The Mono Global assembly cache which solves some of the specific issues of interpreted languages

Certainly, the existing tools are everything but perfect, and not necessarily suitable for Ruby and Python, but some of the design ideas behind them are clearly the good solutions to some of the issues of software development and integration. Adapting these ideas to radically different languages is going to take more time than just throwing other layers of symlink farms.

8 December 2008

Gunnar Wolf: Apt-get and gems: Different planets, right. But it must not be the war of the worlds!

Thanks to some unexplained comments on some oldish entries on my blog, I found -with a couple of days of delay- Rubigem is from Mars, Apt-get is from Venus, in Pelle's weblog. And no, I have not yet read the huge amount of comments generated from it... Still, I replied with the following text - And I am leaving this blog post in place to remind me to further extend my opinions later on.
Wow... Quite a bit of comments. And yes, given that the author wrote a (very well phrased and balanced) post, I feel obliged to reply. But given that he refered to me first, I'll just skip the chatter for later - I'm tired this time of day ;-)
Pelle, I agree with you - This problem is because we are from two very different mindsets. I have already said so - http://www.gwolf.org/soft/debian+rails is a witness to that point.
But I do not think the divide is between sysadmins and developers. I am a developer that grew from the sysadmin stance, but that's not AFAICT that much the fact in Debian.
Thing is, in a distribution, we try to cater for common users. I have a couple of Rails apps under development that I expect to be able to package for Debian, and I think can be very useful for the general public.
Now, how is the user experience when you install a desktop application, in whatever language/framework it is written? You don't care what the platform is - you care that it integrates nicely with your environment. Yes, the webapp arena is a bit more difficult - but we have achieved quite a bit of advance in that way. Feel like using a PHP webapp? Just install it, and it's there. A Python webapp? Same thing. A Perl webapp? As long as you don't do some black magic (and that's one of the main factors that motivated me away from mod_perl), the same: Just ask apt-get to install it and you are set.
But... What about installing a Rails application? From a package manager? For a user who does not really care about what design philosophy you followed, who might not even know what a MVC pattern is?
Thing is, distributions aim at _users_. And yes, I have gradually adopted a user's point of view. I very seldom install anything not available as a .deb - and if I do, I try to keep it clean enough so I can package it for my personal use later on.
Anyway... I will post a copy of this message in my blog (http://gwolf.org/), partly as a reminder to come back here and read the rest of the buzz. And to go to the other post referenced here. And, of course, I invite other people involved in Ruby and Debian to continue sharing this - I am sure I am not the only person (or, in more fairness, that Debian's pkg-ruby-extras team is not the only team) interested in bridging this huge divide and get to a point we can interact better - And I am sure that among the Rubyists many people will also value having their code usable by non-developers as well.

Jonathan McDowell: We need a STFU option

The Debian Project recognises that on occasion the General Resolution process is dominated by a vocal minority of the project members, while the majority are uninterested in the topic under discussion. Without wishing to curtail the democratic process it is felt that it should be possible for the silent majority to indicate its desire to move on from the topic of a GR rather than continuing to rehash the same arguments ad infinitum. In order to facilitate this the project resolves to add a further option to all ballets, "No Further Discussion". It is hoped that in the event of the "No Further Discussion" option being the winner of any vote that the project as a whole will accept that discussion of the subject of the vote is not considered to be a worthwhile use of project time or resources, and that we should therefore concentrate on more important matters. (I thought about signing this post, but decided against it)

Phil Hands: HTTPS VirtualHosts

Until a couple of weeks ago, I was under the impression that one could only have a single HTTPS site per IP address, but it seems I was wrong. The procedure is described here on the CACert wiki. In short, you need multiple SubjectAltName fields on your server's certificate, such that all the VirtualHost names that you want to work are either directly mentioned, or will match via wildcards. The page above includes a link to a nice script that generates a key and CSR (Certificate Signing Request) ready to be pasted into CAcert's Server Certificate page. Note: the CN is pretty much ignored by some browsers, so you'll want to put the machine's main name as one of the SubjectAltNames as well. BTW if you get a warning like:

  [warn] _default_ VirtualHost overlap on port 443, the first has precedence

you probably need to add

  NameVirtualHost *:443

to your /etc/apache2/sites-enabled/ssl before the <VirtualHost *:443> line. Of course, this glosses over the details of doing things like setting your name and address in the certificate, but since CAcert will strip all that out anyway, it only matters if you wanted to get it signed by someone else. Even so, this should get you started -- you can always edit those details into the csr script.

Francois Marier: Setting the default git branch in a bare repository

A bare repository in git does not include a checkout. So in order to set the default branch that users will get after they clone, you cannot use git-checkout mybranch.

Instead, if you want the default branch to be something other than master, you need to do this:

git symbolic-ref HEAD refs/heads/mybranch

Which will update the HEAD file in your repository so that it contains:

ref: refs/heads/mybranch

This is well documented in the git-symbolic-ref manpage but it's not necessarily the first place you'd think of looking.

7 December 2008

Joey Hess: haskell and xmonad

I came back from Thanksgiving in California to find my copy of Real World Haskell had arrived. I've spent the last several days making my way through it up through chapter 10. I'd read a few of the first chapters last year when I was struggling to learn Haskell, and posted enough comments that I'm amoung the (many many) people credited in the Preface. Amusing, since I still don't consider myself proficient in the language. Reading back through those and on through Chapter Nine has swapped back in the basics I learned last year, and built on them. I was able to get all the way to Chapter Ten before hitting real brain-melting stumbling blocks. That's progess! I can't help but feel that Chapter Ten should come later, perhaps after monads are covered. Only other quibbles so far are some typographical problems introduced in typesetting the book, and a few bugs and problems that were pointed out in comments on the web site, but not fixed in time for the first edition.

Anyway, I know more than enough Haskell now to configure xmonad. While I'm still perfectly happy with awesome, xmonad has some cool things going for it. Playing with it this evening, I especially like:

Being able to compile my xmonad.hs and rely on Haskell's type checking to make sure I wrote it correctly and it will work.
Xmonad's integration with gnome. This was a nice suprise, and being able to just use the gnome panel in xmonad, rather than building my own panel, has made it a lot easier to get started with.
gnome panel showing xmonad workspaces
The power and beauty of xmonad's layouts. Start with the usual simple sorts of layouts like tiled, full screen and floating. These can be transformed in a myrid of ways (ie, rotated, decorations added), and can even be combined together to create more complex layouts. So I was quite easily able to configure it to alternate between a tabbed desktop layout that plays nicely with panels, and absolute unadorned full-screen mode:
```
myLayout = desktop   noBorders Full
where
  desktop = tabbed $ ewmhDesktopsLayout $ avoidStruts $ twoPane
  tabbed = addTabsAlways shrinkText myTheme
  twoPane = Tall 1 2/3 3/100
```
(And it wasn't too hard to enhance this with a special-purpose grid layout for the workspace I run pidgin on.)
That my xmonad.hs is small, easy to understand, and doesn't contain any boilerplate beyond main = xmonad $ gnomeConfig.
It lets me muck about with doing something real with Haskell without having to commit to using it in some project of my own.
It supports cycling all windows on screen so that each in turn moves into the main viewport, via XMonad.Actions.RotSlaves.rotAllUp. An action that has been on my window manager wish-list for ages. Implemented in an easy 3 lines of code.

My current set of gripes with xmonad is small:

mod-shift-space is documented somewhere, but not where you need it.
I applied addTabsAlways to a Tall layout, and got title-bar like "tabs". But how do I add multiple windows as tabs to a given frame, like in ion? Can't figure out how, or even if it's possible.
Moving the mouse cursor over one of the above title-bar-like things doesn't focus its window. Very annoying!
Mouse support is very lacking. I'd like to be able to drag tabs around the screen, and maybe drag a border between frames to resize, couldn't find modules implementing any of that.
Theme configuration is only partly factored out, some of it is scattered in several different places.
Reloading the config file takes ages. 1.9 mb executable has to be built and run.
The lack of a dynamic language shows; there's no way to open a prompt and type Haskell into the window manager. I rarely speak lua to awesome or ion3, but it's nice to have the ability there. I think it should be possible to do this with Haskell (see ghci).

Biella Coleman: Nerds, Geeks, and Nerd/Geek Grrrls

I have not sat behind the helm of teaching for very long but I already have a few tricks up my sleeve. One of them is that I assign some of my favorite readings at the end of the semester so as to counter the downtrodden and tepid spirit and mood (not to mention attention) of my students, which drops precipitously with each passing day. Let’s face it post Thanksgiving, we are all a little tired and I try to find the readings, which uplift, intrigue, and challenge cherished assumptions about marriage and sex. So far it seems to pay off and I often can tell because the conversational pitch and excitement in class is high and the student writings are good, great, even exceptional, which, again, is hard to produce/induce this late in the semester. Readers of this blog would probably be most interested in one of these lively readings, Ben Nugent’s American Nerd (and it might be interesting to hear how the European Nerd story would diverge or converge with this one). One of my students, an audio geek and Free Culture President/Free Software junkie, by the name of John Randall produced a very nice little response (not research) paper on the Nugent reading as well as a short piece by Sarah Seltzer from Bitch Magazine
The(Girl) Geek Stands Alone (and thanks to Joe> for cluing me into this piece). Seltzer piece basically argues, in her own words, the following:

Imagine this scene from a comedy: a group of female friends sit around smoking a bowl and working on the Wikipedia page for Lord of the Rings. Their fashion sense is decidedly iconoclastic and several sport thick-rimmed glasses. Without a trace of self-consciousness, they have a hilariously ribald discussion on the relative traits of elves and orcs.
Awesome as it is, you’ll never see this scene onscreen. No mainstream movie or TV series would dare group so many female nerds together, or celebrate them so unabashedly

So John’s whole response paper is here and here is the pdf. In the paper, he makes a number of excellent points but what I loved most about it was his very geeky move at the end of the paper to prove Sarah (somewhat wrong) by listing all the girl geeks that do and have appeared in mainstream (and not-so mainstream) entertainment venues/shows, etc. They are as follows and in his own words:

I will now showcase my own geekiness through my knowledge of geeky female characters. Why? Because I can. But also because I want to demonstrate that if you look hard enough for representations of female geekyiness in pop culture, you will find plenty. Moreover, if you pick the right ones, you can make them support your argument about gender relations, whatever that argument might be. Some of these charters and personalities are hardly gendered, some are hyper-sexual. Some are incredibly attractive but completely asexual. Some undergo a transformation into/out of geekiness, while others to not. Some are powerful, while some are powerless. Some (most?) celebrate their geekiness, others are tortured by it. They are all geeks take your pick: Aeon Flux, a sexy geek who’s technological gadgets give her super powers (Comic drawings then Charlize TheronAeon Flux) Wonder Woman, attractive pilot of an invisible plane Lara Croft, a female Indiana Jones in short shorts, wielding guns and cracking computer codes (CGI and then Angelina Jolie in Tomb Raider) She-Ra, who was way smarter than He-Man (Masters of the Universe cartoons) Gadget Hackwrench, beautiful chipmonk technician for Chip and Dale (Rescue Rangers cartoon) Velma, featuring eyeglasses, awkwardness and brains (Scooby Doo), Hermonie Granger, a geek who is temporarily rejected because she is a geek, remains a geek, and finds love and happiness (Harry Potter) Barbarella, who, through comic strips and a 1968 film, helped introduce science fiction and sex to young women (Barbarella) La Femme Nikita, a skillful, savvy, and very feminine girl who doubles as a covert spy Kate Libby, aka ‘Acid Burn’, uber-sexualized hacker (played by Angelina Jolie in Hackers) Kathryn Janeway, smart and powerful captain of the USS Voyager (Star Trek Voyager) Starbuck (Battlestar Galactica), Dana Scully, FBI agent with encyclopedic media knowledge. The bizzare subtex of non-realized sexual tension was part of the magic The X-Files. Willow Rosenberg, geeky sidekick turned geeky supervillian (Alyson Hannigan in buffy the Vampire Slayer) Michelle Flaherty, hyper-sexual band geek (Alyson Hannigan in American Pie series) Dr Ellie Sattler, heroniene scientist (Jurrasic Park) Ellie, scientis hero (played by both Jenna Malone and Jodi Foster in Carl Sagan’s Contact) Dawn Wiener (Heather Matarazzo in Welcome to the Dollhouse Enid and Rebecca (Thora Birch and Scarlett Johanson in Ghost World) just about every charater ever played by Jenna Malone (Donie Darko, The Dangerous Lives of Altar Boys, The United States of Leland, Saved!, etc) half of the charaters played within the last decade by Jodi Foster (Panic Room, The Dangerous Lives of Altar Boys, Flightplan, The Addams Family half of the charaters played by Christina Ricci (Mermaids, The Addams Family, Little Red Riding Hood, The Ice Storm, Buffalo ‘66, Prozac Nation, Pumpkin, Speed Racer) half of the characters played by Natalie Portman (The Professional, Mars Attacks!, Star Wars, V for Vendette, The Darjeeling Limited, Mr. Magorium’s Wonder Emporium, Garden State) Molly Ringwald. characters played by Molly Ringwald. Rock musician Ani DiFranco and geeky Riot Grrls everywhere. Sarah Vowell, NPR commentator celebrating her geeky life. Voiceover for geeky cartoon characters. Rachel Maddow, for being Rachel Maddow.

First, awesome list, though he forgot a few (like one of my favorites, Bionic Woman and a more recent one, Juno) and it is nice to have it in one compact place. But, I have to say, I still agree to some degree with Sarah Setlzer, though I also agree with John. On the one hand there are representations and it is as important just to strut this stuff publicly as it is to claim that there is not enough female geeky representations in mainstream media. This is what John has done quite nicely. One the other hand, as he himself says ” if you look hard enough for representations of female geekyiness in pop culture, you will find plenty.” I think those words, “if you look hard enough” also speaks volumes of the continued disparity that does exist. One should not have to look “hard,” and the only blockbusters, so to speak, which feature a female geek, is Tomb Raider, which for being so hyper-sexualized is not so geeky to me, no matter how good she is with the gadgets. That said, what I find so important, and have emphasized in different contexts, is the need for what I think of simultaneous positive and negative form of critique, the former being about pointing to already exisitng examples to get people jazzed and excited and to put things in perspective. The later form of critique, negative critique, identifies a lack, a void to fill, just the type of excellent commentary in the Seltzer piece… But now for the most important question, who has John overlooked?

Jon Dowland: 1984

The UK now has it's own equivalent to China's great firewall. My current ISP "bethere" is one of several which have opted into using it. If I try to access the wikipedia article on the Scorpion's album "Virgin Killer" I receive a "404 not found response" which does not originate from Wikipedia. This album had a controversial picture of a young girl naked on the cover. The censoring proxy has blocked access to both wikimedia's copy of the album cover but also the entire article about it (including discussion about the album cover controversy). There are two things in particular that really annoy me about the way this has been implemented. Firstly, the response I get for a censored page is an unadorned 404. At the very least it should be an explanation of why I cannot access the material, and who to contact if there has been a mistake (no doubt in the eyes of our moral superiors there are no mistakes). Secondly, my ISP has signed up for this and implemented it without telling me. I will shortly be moving both my broadband and that of my parents away from them. The scary thing for me about George Orwell's 1984 was not just the total control but how ruthlessly efficiently it was implemented. So far this has been implemented predictably poorly, first of all it's trivial to circumvent the block on wikipedia, at least; secondly, they've inadvertently buggered up editing Wikipedia for those affected.

Neil Williams: langupdate for Emdebian Grip - borne from edos-debcheck

Whilst building more and more packages into Emdebian Grip, I came across gettext
which depends on libglib2.0-0 which surprised me somewhat. However, this is extremely useful because langupdate - the Emdebian-only utility that I wrote to correlate Emdebian TDebs with installed packages and supported locales - also uses libglib2.0-0 as it is one of my favourite libraries for all my C code. So now there is no need to rewrite langupdate to not use libglib2.0-0 (I wasn't looking forward to that particular task so it's even better).
Secondly, I thought about using reportbug --kudos edos-debcheck but AFAICT that merely sends an email to Stefano Zacchiroli so I thought I'd put the "kudos" into a blog post for more people to read:
If you haven't used edos-debcheck before you probably haven't done any work at all with repositories - edos-debcheck is becoming fundamental to everything I do. Emdebian simply could not function without these checks - it would take hours to identify the missing dependencies. (I know because before edos-debcheck was released, I did spend (s/spend/waste/) hours upon hours on a fruitless chase through the dependency trees in Debian.)
Consider my current task - adding packages to Emdebian Grip in such a way as to retain the smallest possible package set in which all dependencies are met (excluding Recommends that do not apply in Emdebian). edos-debcheck makes it almost trivial:



codehelp@ant:/org/emdebian$ ./scripts/grip-binary.sh libxml2

....

codehelp@ant:/org/emdebian$ edos-debcheck -explain -failures < grip/dists/unstable/main/binary-amd64/Packages

Parsing package file...  0.0 seconds     135 packages

Generating constraints...  0.0 seconds

gettext (= 0.17-6em1): FAILED              

The following constraints cannot be satisfied:

  gettext (= 0.17-6em1) depends on gettext-base  NOT AVAILABLE 

Checking packages... 0.0 seconds  

codehelp@ant:/org/emdebian$ ./scripts/grip-binary.sh gettext-base

Simple. A second script then goes around behind and ensures that each binary added to the repo has the corresponding source package also added. The key is that Emdebian Grip does not have to have all binaries from that source - we only use the binaries that are needed, either directly or as direct dependencies of others.
Zack - and all the people who worked on edos-debcheck - your code is just indispensable to me.

6 December 2008

Kartik Mistry: Kartik Mistry

* Inspired by Lior’s post, here is my 6 month adventure with Mac OS X. Since, I have dual-boot with Debian on my Macbook, and it works perfect; there are only 6 hours for Mac for me at work. Still, I can use ‘Free and Open Source’ softwares as much as possible

Firefox Adium Songbird Colloquy GIMP OpenOffice.org VLC Vim Frozen Bubble X-Moto Nope. Still, I am use Mail and iPhoto/Photo Booth. Shame on me, but will move to alternatives soon! Note to potential Linux developers who want to buy macbook: Don’t buy it. You can get better performance and don’t have to fix things and you can spend that much time in other useful work!

Biella Coleman: FLOSS Manuals: How to Bypass Internet Censorship

If you don’t know about this great project, you should. The nitty gritty details are to be found here. FLOSS Manuals Release Circumvention Book, How To Bypass Internet
Censorship December 4, 2008, Amsterdam A new book released by FLOSS Manuals, How to Bypass Internet Censorship,
describes circumvention tools and explains why you might want to use
them, and honestly describes the risks you must consider before
circumventing blockers or monitors. Blockers and monitors restrict
access to areas of the Internet, and this book describes simple
techniques for bypassing those restrictions. The book can be read or
downloaded for free as a PDF from flossmanuals.net, or you can purchase
a high-quality printed copy of the 200 page book through Lulu, an
on-demand printer, at http://www.lulu.com/content/4904448 for 10.83
($14.00). The growth of the Internet has been paralleled by attempts to control
how people use it, motivated by a desire to protect children,
businesses, personal information, the capacity of networks, or moral
interests, for example. Some of these concerns involve allowing people
to control their own experience of the Internet (for instance, letting
people use spam-filtering tools to prevent spam from being delivered to
their own e-mail accounts), but others involve restricting how other
people can use the Internet and what those other people can and can’t
access. The latter case causes significant conflicts and disagreements
when the people whose access is restricted don’t agree that the blocking
is appropriate or in their interest. Problems also arise when blocking
mechanisms and filters reduce access to useful business, health,
educational, and other information. Because of concerns about the effect of internet blocking mechanisms,
and the implications of censorship, many individuals and groups are
working hard to ensure that the Internet, and the information on it, are
freely available to everyone who wants it. There is a vast amount of
energy, from commercial, non-profit and volunteer groups, devoted to
creating tools and techniques to bypass Internet censorship. Some
techniques require no special software, just a knowledge of where to
look for the same information. Programmers have developed a variety of
more capable tools, which address different types of filtering and
blocking. These tools, often called “circumvention tools” help Internet
users access information that they might not otherwise be able to see.
This book documents simple circumvention techniques such as a cached
file or web proxy, and also describes more complex methods using Tor,
which stands for The Onion Router, involving a sophisticated network of
proxy servers. How to Bypass Internet Censorship was written by eight writers in a
FLOSS Manuals ‘book sprint’ - a week-long intensive writing session, and
it also includes content from many different authors’ previous works on
the subject.
How to Bypass Internet Censorship will always be available for free from
the FLOSS Manuals Website. Each sale of the book generates $2.50 (USD).
100% of this income goes back into the development of more manuals about
free software. About FLOSS Manuals
http://www.flossmanuals.net
FLOSS Manuals is a non-profit foundation and community creating a
collection of manuals that explain how to install and use a range of
free and open source software. The manuals are friendly and simple, and
they are intended to encourage people to explore the wide range of free
and open source software. FLOSS stands for Free, Libre Open Source
Software, and FLOSS Manuals intends to provide free manuals for free
software. The manuals on FLOSS Manuals are written by a community of
people, writers, editors, and technicians do a variety of things to keep
the manuals as up to date and accurate as possible. The way in which
FLOSS Manuals are written mirrors the way in which FLOSS software itself
is written: by a community who contribute to and maintain the content. FLOSS Manuals produces printed books, PDF books, and HTML output. Each
chapter from each manual can be recombined with other chapters to create
a new manual, which we call remix capability. An embed API lets you use
FLOSS Manuals to write the content and then embed the content into your
website. For more information contact:
Adam Hyde
Founder, FLOSS Manuals
adam@flossmanuals.net

Next.

Previous.