Search Results: "tar"

26 October 2011

Ingo Juergensmann: Roundcube doesn't work anymore because of suhosin

Well, yesterday out of nothing my webmailer roundcube started to refuse to work. At least as I remember it. For some reasons reloading the Inbox just showed the "Loading..." message on the screen, but there was no list of mails anymore. Funny enough other folders do actually work as before. But anyway, doing an update did not help and improve anything. (I really don't know whether I updated before or after because of the first occurence of this issue.) There's an entry in syslog when loading the Inbox folder:
Oct 26 07:24:59 muaddib suhosin[32432]: ALERT - Include filename ('http://www.gnu.org/s/hello/manual/automake/ ?.php') is an URL that is not allowed (attacker '127.0.0.1', file '/usr/share/roundcube/program/include/iniset.php', line 110
This lead to bug #1488086 in the Roundcube issue tracker which states:
This messages made me wonder why suhosin thinks there's an include going on. Line 111 of iniset.php shows: include_once("$filename.php"); It seems like roundcube wants to include what is displayed in the subject, which happens to be a url - and suhosin legitimately blocks this attempt. In short, I can send an email to a user on a suhosin protected mail server and make his inbox unavailable. Needless to say, the user cannot delete this email himself via RoundCube. In my case, I had to delete the email file on the server to make roundcube show the inbox again.
In Debian there's bug #619411 that is related to PATH setting in iniset.php, but I'm not sure if this is really related to #1488086 in the Roundcube issue tracker and my problem? However, disabling suhosin doesn't seem the right way to "solve" this issue and the trac issue tracker suggests a security related problem. Anyway, I filed this as bug #646675 in Debian, waiting for the bug number. But when someone else knows some quick fixes or something I can try, please speak up! :-) UPDATE: It seems as if some mail triggered this issue like reported in the Roundcube ticket. After filtering my mails with Iceweasel, I'm being able to read my Inbox now again.

Russ Allbery: Review: Scrum and XP from the Trenches

Review: Scrum and XP from the Trenches, by Henrik Kniberg
Publisher: InfoQ
Copyright: 2007
ISBN: 1-4303-2264-0
Format: Trade paperback
Pages: 127
For those not already familiar with it, Scrum is a project management methodology that's most often seen in the agile software development community. It attempts to do for project management what agile techniques do for software development (and, other than test-driven development, I think is more useful than most of agile). I'm not going to explain it here in detail, but if you do software development or project management of software-style projects (including integration and computing infrastructure), it's worth your attention. It provides some nice techniques for dealing with some of the banes of technical projects, particularly constantly changing requirements and the difficulties of estimates. Scrum and XP from the Trenches is subtitled "How we do Scrum" and is exactly what it says on the tin. It is a fairly detailed description of how Crisp (an agile Java development company in Sweden) does Scrum (with a little discussion of Extreme Programming). It doesn't explain what Scrum is, although it does provide a few pointers to resources if you've never heard of it (but this isn't the sort of book you'd be buying if you'd never heard of it). It also doesn't try to convince you that Scrum is wonderful, which is a nice trap to avoid. It gets right down to business and packs quite a bit of relentlessly practical content into a small number of pages. I've been doing Scrum for about a year, across two large projects, both times not entirely following the full Scrum model. Both times, most of the team was learning Scrum during the project. I've gone to some Scrum training and read a bit, largely on-line, but I haven't made an exhaustive study of it. I think I'm exactly the ideal audience for this book, although my guess is that people who have been doing Scrum for longer will also get something out of it. This is not an introduction; I don't think one is likely to get much benefit out of it without a couple of Scrum projects under one's belt so that one has a feel for the problems and failure modes. As Kniberg accurately points out, Scrum is a framework, not a precise rule set, and it can't be applied blindly in one situation using analysis from another. For that audience, though, this book is great. Kniberg has used Scrum in a variety of different situations and tried multiple solutions to many of its problems, so he can provide guidance not only on his preferred solutions but on the other solutions he's tried and why they don't work. I started taking extensive notes just a few pages in; it was exactly the book I needed to read for the full-project retrospective on how Scrum worked for a just-concluded work project, and for planning what changes we'll want to make for the next project. He goes through every step of the Scrum cycle and offers advice on each step, particularly for the sprint planning (which was extremely helpful). He then walks through release planning and advance estimation (one of the best parts of the book; I've seen the concept of a focus factor before, but this is one of the best and clearest explanations I've seen), talks about XP (Extreme Programming) techniques and testing, and then discusses how to coordinate multiple Scrum teams in a larger organization. The chapter on testing was the reason why I bought the book; the rest of the book lives up to it. As mentioned, my Scrum exposure is not broad, so Kniberg may just be following different orthodoxies. That said, he broke with the ones I was previously familiar with in some useful places. One is to attach a time unit to story points, something that I was told was close to anethema, but as presented here looks very useful. Another, not so much a break but an idea that I'd not encountered elsewhere, is downtime between sprints, an brilliant solution to a problem that I'd not even fully articulated but which had been plaguing our Scrum process. He also has some suggestions about communication that, while fairly obvious and common-sense stuff, still were things that hadn't occurred to me and that we should adopt. I paid for a hard-copy version of this book from Lulu because I like things like this on paper and am happy to support the author financially, but you can download a PDF file and read it for free from InfoQ and see whether it works as well for you as it did for me. If you're ramping up on Scrum, seeing some problems, and looking for good ideas on how to fix them, I highly recommend it. I'd love to read two or three more case studies like this to get some more perspectives on how to make Scrum work. The paper book will likely be making a circuit around our office to serve as fodder for our upcoming Scrum projects. Rating: 8 out of 10

25 October 2011

Sylvestre Ledru: Removal of sun-java6 from Debian

As said in one of my previous blog post, we, Debian, were going to remove sun-java6 packages from the archive once critical security issues would come. It was supposed to happen sooner and later and here we are. The version 29 (see the #645881 Debian bug) of the Oracle proprietary JVM has been released. It fixes some critical issues but, because of the DLJ removal, Debian is no longer able to distribute it. Unfortunately, we have no other choice than asking for a removal from the archive and promote the use of the openjdk 6 & 7.
The conclusion is:
apt-get --purge remove sun-java6-jre && apt-get install openjdk-7-jre

24 October 2011

Biella Coleman: Occupy S o Paulo Under Threat

Occupy Sao Paulo Under Threat I was in Brazil last week and had the pleasure of participating in a teach-in at Occupy S o Paulo, a vibrant camp set up last week in the heart of the old down town. There are now under threat, please see below for details and spread the word. *** On October 15, a group of nearly 300 activists began an occupation of S o Paulo in the Valley of Anhagabau, one of the sites of the first rallies for direct elections during the end of the Brazilian military dictatorship in the 1980s. After a week of peaceful encampment, educational and cultural programs, and creating a sustainable community for not just themselves but many homeless people in downtown S o Paulo, the Occupy S o Paulo movement is coming under increased police threat. Today, Monday October 24, the governor of the state of S o Paulo, Geraldo Alckmin is holding two special events. First, he is hosting Florida Governor Rick Scott (R). Second, Alckmin has decided to hold a parade of 3000 military police right next to the encampment (see photo above). After a week of police harrassment and a pending court case for them to hold the right to pitch their tents, the Occupy S o Paulo movement sees this as an escalation of the harassment they have already faced by city police. Further, the presence of 3000 military police next to 300 occupiers is clearly meant to intimidate both occupiers and members of the public who have been coming up to the encampment and learning about the movement. Please take the time to call or email the governor and the secretary of public security of the state of S o Paulo to condemn this action. You can also send messages of solidarity to the S o Paulo occupation at occupysampaenglish@googlegroups.com. To contact Governor Geraldo Alckmin s office of citizen and organizational relations: Fill out a comment form at: http://www.saopaulo.sp.gov.br/en/fale/fale.php
Phone: 55-11-2193-8463 To contact the Secretary of Public Security, Ant nio Ferreiro Pinto:
Email: seguranca@sp.gov.br
Phone: 55-11-3291-8500

Colin Watson: Quality in Ubuntu 12.04 LTS

As is natural for an LTS cycle, lots of people are thinking and talking about work focused on quality rather than features. With Canonical extending LTS support to five years on the desktop for 12.04, much of this is quite rightly focused on the desktop. I'm really not a desktop hacker in any way, shape, or form, though. I spent my first few years in Ubuntu working mainly on the installer - I still do, although I do some other things now too - and I used to say only half-jokingly that my job was done once X started. Of course there are plenty of bugs I can fix, but I wanted to see if I could do something with a bit more structure, so I got to thinking about projects we could work on at the foundations level that would make a big difference. Image build pipeline One difficulty we have is that quite a few of our bugs - especially installer bugs, although this goes for some other things too - are only really caught when people are doing coordinated image testing just before a milestone release. Now, it takes a while to do all the builds and then it takes a while to test them. The excellent work of the QA team has meant that testing is much quicker now than it used to be, and a certain amount of smoke-testing is automated (particularly for server images). On the other hand, the build phase has only got longer as we've added more flavours and architectures, particularly as some parts of the process are still serialised per architecture or subarchitecture so ARM builds in particular take a very long time indeed. Exact timings are a bit difficult to get for various reasons, but I think the minimum time between a developer uploading a fix and us having a full set of candidate images on all architectures including that fix is currently somewhere north of eight hours, and that's with people cutting corners and pulling strings which is a suboptimal thing to have to do around release time. This obviously makes us reluctant to respin for anything short of showstopper bugs. If we could get things down to something closer to two hours, respins would be a much less horrible proposition and so we might be able to fix a few bugs that are serious but not showstoppers, not to mention that the release team would feel less burned out. We discussed this problem at the release sprint, and came up with a laundry list of improvements; I've scheduled this for discussion at UDS in case we can think of any more. Please come along if you're interested! One thing in particular that I'm working on is refactoring Germinate, a tool which dates right back to our first meeting before Ubuntu was even called Ubuntu and whose job is to expand dependencies starting from our lists of "seed" packages; we use this, among other things, to generate Task fields in the archive and to decide which packages to copy into our images. This was acceptably quick in 2004, but now that we run it forty times (eight flavours multiplied by five architectures) at the end of every publisher run it's actually become rather a serious performance problem: cron.germinate takes about ten minutes, which is over a third of the typical publisher runtime. It parses Packages files eight times as often as it needs to, Sources files forty times as often as it needs to, and recalculates the dependency tree of the base system five times as often as it needs to. I am confident that we can significantly reduce the runtime here, and I think there's some hope that we might be able to move the publisher back to a 30-minute cycle, which would increase the velocity of Ubuntu development in general. Maintaining the development release Our release cycle always starts with syncing and merging packages from Debian unstable (or testing in the case of LTS cycles). The vast majority of packages in Ubuntu arrive this way, and generally speaking if we didn't do this we would fall behind in ways that would be difficult to recover from later. However, this does mean that we get a "big bang" of changes at the start of the cycle, and it takes a while for the archive to be usable again. Furthermore, even once we've taken care of this, we have a long-established rhythm where the first part of the cycle is mainly about feature development and the second part of the cycle is mainly about stabilisation. As a result, we've got used to the archive being fairly broken for the first few months, and we even tell people that they shouldn't expect things to work reliably until somewhere approaching beta. This makes some kind of sense from the inside. But how are you supposed to do feature development that relies on other things in the development release? In the first few years of Ubuntu, this question didn't matter very much. Nearly all the people doing serious feature development were themselves serious Ubuntu developers; they were capable of fixing problems in the development release as they went along, and while it got in their way a little bit it wasn't all that big a deal. Now, though, we have people focusing on things like Unity development, and we shouldn't assume that just because somebody is (say) an OpenGL expert or a window management expert that they should be able to recover from arbitrary failures in development release upgrades. One of the best things we could do to help the 12.04 desktop be more stable is to have the entire system be less unstable as we go along, so that developers further up the stack don't have to be distracted by things wobbling underneath them. Plus, it's just good software engineering to keep the basics working as you go along: it should always build, it should always install, it should always upgrade. Ubuntu is too big to do something like having everyone stop any time the build breaks, the way you might do in a smaller project, but we shouldn't let things slide for months either. I've been talking to Rick Spencer and the other Ubuntu engineering leads at Canonical about this. Canonical has a system of "rotations", where you can go off to another team for a while if you're in need of a change or want to branch out a bit; so I proposed that we allow our engineers to spend a month or two at a time on what I'm calling the +1 Maintenance Team, whose job is simply to keep the development release buildable, installable, and upgradeable at all times. Rick has been very receptive to this, and we're going to be running this as a trial throughout the 12.04 cycle, with probably about three people at a time. As well as being professional archive gardeners, these people will also work on developing infrastructure to help us keep better track of what we need to do. For instance, we could deploy better tools from Debian QA to help us track uninstallable packages, or we could enhance some of our many existing reports to have bug links and/or comment facilities, or we could spruce up the weather report; there are lots of things we could do to make our own lives easier. By 12.04, I would like, in no particular order: Of course, this overlaps to a certain extent with the kinds of things that the MOTU team have been doing for years, not to mention with what all developers should be doing to keep their own houses in reasonable order, and I'd like us to work together on this; we're trying to provide some extra hands here to make Ubuntu better for everyone, not take over! I would love this to be an opportunity to re-energise MOTU and bring some new people on board. I've registered a couple of blueprints (priorities, infrastructure) for discussion at UDS. These are deliberately open-ended skeleton sessions, and I'll try to make sure they're scheduled fairly early in the week, so that we have time for break-out sessions later on. If you're interested, please come along and give your feedback!

Russell Coker: Stop HRL

Today I attended the Stop HRL demonstration [1]. The government plans to spend $100,000,000 of federal money and $50,000,000 of Victorian state money to build a new coal power station. The state government has imposed some unreasonable restrictions on renewable energy which includes allowing a single person who objects within 2Km of a wind turbine to block the project while also not allowing anyone to object to coal power plants or fracking (both of which have proven health implications). Today VCAT had a hearing about this issue, HRL wants to double the size of the proposed coal power plant while Environment Victoria, Doctors for the Environment Australia, local climate action group Locals Into Victoria s Environment Martin Shield from Climate Action Moreland are opposing the plan. What the government should be doing is permitting (if not encouraging) renewable energy production, particularly wind power which will drive down energy prices by undercutting the energy auctions due to it s almost zero marginal cost. The government should not be spending tax payer money on new coal projects due to climate change and air pollution in addition to the fact that coal is simply more expensive in the long term. As usual it s disappointing that the National party which supposedly represents farmers is supporting the government in such things. The pollution caused by coal power and fracking ends up damaging farmland. The Liberal party doesn t have a majority on it s own in either house of the state government, so this proposed coal power plant could be stopped immediately if the National party supported renewable power, or even if they merely opposed giving $50,000,000 of state tax money to HRL. The demonstration went reasonably well, there was about 80 people there when I arrived and by the time it ended there were more than 150 people. I took the above picture at the start and it missed many people who were off to the right. I think that s a good result for a demonstration that had little publicity and was held in business hours with really bad weather. Update: On Sunday the 13th of November at 13:00-14:30 there is a Going Backwards Under Baillieu demonstration at Parliament House. This is to mark the negative affects that the Liberal state government has had on the environment of Victoria.

Gregor Herrmann: managing $HOME

I'm not the first one who thought about managing their dotfiles & scripts in a (D)VCS in order to be able to have them in sync across different machines, & I even haven't re-invented the wheel for doing so. short report on my experiences:

after looking around a bit I decided to try movein (info page with links to downloads & a README, blog post, git repo) because it looked liked it would easily do what I'd like to do without getting in my way too much; which means e.g.: no symlink farm, & only one simple command to manage the whole setup.

movein is a simple shell script that allows to setup & manage multiple modular repos tracking different sets of files quite easily, taking advantage of the powers of git & mr. & being written in shell means I also was able to make a few changes, which stew kindly accepted & applied to his tree already.

in fact I'm just starting to use it for real, but it looks good so far, & I think I'm going to stick with it for the time being.

2 minor observations: </promotion_time>

Dirk Eddelbuettel: Kurt Elling and the Kl vers Big Band at the Green Mill

Twenty-four hours ago I was in heaven. Or to more properly recapitulate and situate this, fourty-eight hours ago I was in hell. Now this requires some context. Earlier in the year, I had tweeted briefly about a Kurt Elling live concert still streaming on NPR, and truth be told, I have listened to Kurt Elling a whole lot ever since. The most recent album The Gate is fantastic, the Grammy-winner Dedicated to You (which retakes the very famous Coltrane/Hartman album) is beyond words, and The Messenger is worth it just for 'Nature Boy' and the funkiest-ever 'April in Paris'. I also saw him as part of a larger and otherwise rather neat tour. But I needed to see the man himself. So following what one can see from his touring calendar, I jotted down two October dates. Kurt Elling at the Green Mill. Fine. That was earlier in the summer. Summer has a tendency of evaporating quickly, so here we were in October. And getting going late on Friday, so by the time we got the Green Mill, a long line had formed and we ended up paying tribute to the line for fourty-five minutes with nothing to show. Sad. Home we went, and no live music. But somehow we mustered the energy to go again last night and thought we were early, arriving about 20 minutes before the show was to start. Ha! Others had appeared at 3pm, by 5pm all seats were gone. We were literally the last ones the get in after a short spell in a shorter line, and standing room-only it was. Oh, but what a treat we got. The set for these two days was Kurt Elling with the fourteen piece Kl vers Big Band from Denmark. They played for three marvellous sets, and well over four hours (with two 25 minute breaks). Pieces often alternated between big band arrangements and smaller pieces by Elling's standard rhythm band, and covered standards as well as Elling's catalogue. As I mentioned earlier on a short post (with pictures) on Google+, it is essentially impossible to not fall for Elling's stage presence when being so close to the stage. The man is just that good. Elling and the Kl vers Big Band will now play for six nights at Birdland in New York followed by a night each in Washington, DC, and Boston. If you're around, do yourself a favour and go out to see them.

23 October 2011

Asheesh Laroia: RFBP: Request for birthday present/package

There is a program that I love: bb. bb is a demo of the famous ASCII Art library, aalib.
     dT8  8Tb     
    dT 8  8 Tb    
   dT  8  8  Tb   
<PROJECT><PROJECT>
 dT    8  8    Tb 
dT     8  8     Tb
bb is a demo-scene-type program that shows how awesome automatic ASCII art is. The personalities of the people who made bb shine through. It's surely one of my favorite programs in Debian, up there with alpine. It's been in Debian since 1998. bb has a serious bug, however: BB's "graphics" freeze when music starts. Here's the issue. So if your system (like most GNU/Linux systems) uses pulseaudio for sound, then bb is broken. That means every Ubuntu user and most desktop Debian users can't use it. There are a few possible fixes, depending on where you'd want to solve the problem. If you just want bb to run on your own machine, without recompiling anything, you can adjust pulseaudio's configuration (in /etc/pulse/default.pa) to disable esound support. If you want to do that, just comment out this line:
 load-module module-esound-protocol-unix
We could also possibly patch bb so that it asks libmikmod not to use its esound "support." I think the smarter thing to do is to adjust libmikmod. Since its esound support seems to be just plain broken, it should be removed. At very least, it should not be the default when ALSA output is available. There is a new upstream release of libmikmod, maybe the esound output is fixed. In Debian, libmikmod is orphaned. When a package is orphaned, it means that a new person must step in and adopt the package. Debian packages need ongoing care and commitment to fix issues and make changes like this that benefit the users. In this case, you'd need to understand some C and be willing to maintain a shared library. Maintaining a library in Debian requires attention to detail, but it is quite doable. Since you would be adopting an existing package, most of the work is already done for you. I would also be quite willing to answer questions. If you're not a Debian developer, I would happily sponsor uploads of this package into Debian so that the fixes are part of the distribution. So: who will maintain libmikmod and fix bb? Could it be you? It would make a really great birthday present if the amazing bb program worked in the next Debian release. Leave a comment if you have questions or are interested! P.S. In a pinch, I can be convinced to maintain libmikmod myself, but I think this is a great opportunity for someone new to Debian to make a big difference.

Gregor Herrmann: RC bugs 2011/42

this week my activities were split between some freshly-reported pkg-perl bugs, & looking at some of the older RC bugs. for most of the ones in the latter category I found again ready-to-be-applied patches, many of them from Ubuntu again. details:

Uwe Hermann: The TrekStor eBook Reader 3.0 (EBR30-a), review and dissection

The TrekStor eBook Reader 3.0, front
The TrekStor eBook Reader 3.0, front on There a many, many, e-book reader devices available these days, and they're quickly becoming pretty affordable. The currently cheapest device in Germany (that I know of) is the TrekStor eBook Reader 3.0, model number EBR30-a, at 59.- Euros via Weltbild or Hugendubel. The device has an 800x480 7" TFT (yep, no e-ink), 2100mAh battery, it can display PDFs, EPUB, and TXT files (and Adobe DRM crap, which I don't really care about), it has an accelerometer which allows for landscape/portrait switching, it can play MP3, OGG, WAV, and WMA audio files (headphone jack), it can display pictures (BMP, GIF, JPG, even PNG, though that's not mentioned in the vendor's specs), and it has 2GB internal storage for books/music/pictures. Uploading of (non-DRM) content is done by a simple file copy, it enumerates as a standard USB mass storage device with FAT filesystem. It's a relatively nice reader for the price, I've read a few PDFs (datasheets, presentations) on it in the subway/train while listening to music from the device and it's quite OK for my purposes. So much for the review part. However, I didn't really buy it for reading books on it, I was more interested in taking it apart, of course ;-) My hope was that it would turn out to be a really cheap device running Linux/U-Boot which would be perfect for playing around with embedded Linux stuff. Unfortunately, I wasn't so lucky (it seems). The TrekStor eBook Reader 3.0, opened I've posted a few photos of the device and its hardware components on my flickr account and over at randomprojects.org, together with all the information I was able to find out so far. Here's a quick summary: The TrekStor eBook Reader 3.0, CPU There are public datasheets for most of the hardware components (see randomprojects.org for links), but unfortunately the most important one (for the CPU) is not yet found/identified. I was told that the CPU/SoC is probably based on an ARM9 (ARM926EJ-S) core and the firmware running on it seems to be some uCos-based RTOS (not Linux, unfortunately). So far I was not able to find out the vendor name or website of the "FI E200" CPU/SoC (let alone any datasheets), any hints would be highly appreciated. I checked arm.com: Processor Licensees, but the only two companies whose name starts with "F" having licensed an ARM9 core are Fujitsu and Freescale, which doesn't fit, I think? I could (and probably will) check the PCB for RX/TX lines on an UART and/or JTAG pads (none are obviously labelled), and given that it's and ARM9 core there is a good chance that OpenOCD can be used and that a standard cross-gcc for ARM will work. However, that is all pretty pointless until it's clear which SoC exactly is used, and thus whether there is already Linux and/or U-Boot support for it and/or whether datasheets are available so that the respective code could be written. Without datasheets, this is going to be a pretty painful experience, not really worth investing much time, IMHO. If anyone knows more about the vendor/device and respective datasheets, please let me know. Thanks!

Niels Thykier: Testing testing migration

If you have been following Lintian s development closely, you will probably have noticed that I have not really done anything there for the past week. Instead I have turned my focus on our testing migration script, britney2. First, I have created a minimal test suite[1]. It started as 4 simple tests and by now it contains about 30 tests. The size of each test is rather small; the largest tests are about 1600 binary packages in total[2], but most are 2-20 binary packages in total. Thus the test suite is rather fast compared to a live data sample , which easily takes more than 10 minutes for a single run. Unfortunately, hand-crafting the test data is somewhat annoying and easy to get wrong. The test suite has a somewhat unfair focus on auto-hint [3] cases, so the current britney2 fails up to 14 tests. Some of these appears to fail because the auto-hinter (for some reason) receives incomplete information about the situation. To my knowledge we not been able to debug the situation, but Adam has a refactor branch that does not seem to have this issue. Personally I am hoping it will soon be merged into the master branch, especially because it seems to simplify a lot of common operations. Joachim Breitner (who has been working on a SAT-solver based britney) also contributed a couple of test cases[4]. Allegedly, SAT-britney does rather well on the test suite, failing only 2 tests as far as I can tell[5]. On the other hand, it does solve a some of the more interesting cases britney2 does not solve. On a more mathematical note, the britney2 implementation behaves like a function[6] with an attractive fixed point[7]. This is interesting, because for some cases it may take britney2 a couple of iterations to reach the right solution. This fixed point is somewhat simple to find by using the following steps (pseudo-code):
// Runtime complexity O(n * br * diff), where "n" is the number of iterations until
// a fixed point is reached, "br" is the complexity of "run_britney" and "diff" is
// the runtime of the "last != current" comparison.
function find_fixed_point(initial);
    last = run_britney(initial)
    current = run_britney(last)
    while last != current ; do
        last = current
        current = run_britney(last)
    od
    return current
end
This gives us a simple way to test if britney will eventually solve the issue herself (and when she will do it). Currently britney2 is automatically run twice a day, so for every 2 iterations (beyond the first) roughly translates to a 24-hours delay. So far the test suite does not have a lot of problems that requires more than one iteration. Personally I would be pleased if it turned out to stay that way as the test suite coverage grows. If you are interested in playing around with this, you can get sources from: Footnotes: [1] http://lists.debian.org/debian-release/2011/10/msg00178.html [2] These tests are auto-generated, so it is merely an up-scaled pattern . [3] Basically if two (or more) packages needs to migrate into testing at the exact same time, they need to be hinted in. [4] Not to mention all the copy-waste errors he pointed out in mine. Apparently, SAT-britney has stricter requirements to the data than britney2. :P [5] I assume the test called sat-britney-death (created by Joachim) was named that way for a reason. The second failure is caused by SAT britney not reading hints (yet?), so the approve tpu package test case should fail. [6] A function that maps an archive into another archive erh, I mean, it maps a set of packages into another set of packages :P [7] http://en.wikipedia.org/wiki/Fixed_point_%28mathematics%29 Assuming my claim to be true, the function will have more than one fixed point. The obtained fixed point depends on the initial state of testing. As an example:
y depends on x
x in testing has RC bugs If x is not in testing, it cannot migrate to testing (due to its RC bugs). If x is not in testing, then y cannot migrate into testing. But if x starts in testing, then y may be able to migrate. This can happen if x migrated to testing before an RC bug was filed against it. (Dis-)Proving my claim is an exercise left for the reader.

Luca Falavigna: Stats, more stats and, guess what? Even more stats!

We all love stats, don t we? So, here we go! Let s start with a graph: NEW graph It shows the number of packages in the NEW queue since last year. You can see a big drop during April 2011, and a reasonably low rate during the last six months. You could think fellow Debian Developers stopped to upload NEW packages. Sorry, you re wrong! :) Since Squeeze release, 3.832 .changes files with NEW components were processed by dak, with an average of 14,85 NEW packages per day. On the FTP Team side, we had 3.732 accepts (14,47 per day), 339 rejects (1,31 per day) and 178 comments to maintainers (0,69 per day).
Who were the most prolific maintainers who got a NEW processing? Here is our special top ten:
  1. Debian Haskell Group (362 packages)
  2. Debian Perl Group (343 packages)
  3. Debian Java Maintainers (161 packages)
  4. Debian Ruby Extras Maintainers (124 packages)
  5. Debian Multimedia Maintainers (100 packages)
  6. Debian Fonts Task Force (96 packages)
  7. Debian Med Packaging Team (79 packages)
  8. Debian Install System Team (61 packages)
  9. Debian Javascript Maintainers (54 packages)
  10. Debian Python Modules Team (50 packages)
That s bad packaging teams cannot bake cookies!
Let s do the same with Changed By, this time:
  1. Ben Hutchings (159 packages)
  2. Joachim Breitner (138 packages)
  3. Clint Adams (134 packages)
  4. Jonas Smedegaard (124 packages)
  5. TANIGUCHI Takaki (97 packages)
  6. Nicholas Bamber (61 packages)
  7. Alessio Treglia (60 packages)
  8. maximilian attems (54 packages)
  9. David Paleino (51 packages)
  10. Torsten Werner (45 packages)
Much better now go and heat up your ovens, we know who you are ;)
Another nice aspect to look at is the speed of NEW processing. Some maintainers were very happy for a fast NEW processing, someone even complained for having been too quick! :) So, let s find out which upload was the quickest ever. Try to gamble a bit before reading the answer, to see whether you are near to the real value ;) Alessio Treglia, you probably already know, because your gwc_0.21.16~dfsg-1 upload has been processed in 41 seconds (yes, forty-one seconds!). Here s an excerpt from ftp-master log to certify it:
20110516120252 process-upload dak Processing changes file gwc_0.21.16~dfsg-1_amd64.changes
20110516120258 process-upload dak Moving to new gwc_0.21.16~dfsg-1_amd64.changes
20110516120339 process-new tolimar NEW ACCEPT: gwc_0.21.16~dfsg-1_amd64.changes
Alex was the super-fast FTP Team member behind the quickest accept, do you want to beat him? Join FTP Team ;)

Andrew Pollock: [life] Miss Representation

A few months after Zoe was born, it hit me that we didn't just have to raise a baby to grow up happy and healthy, we were raising a girl, and had to worry about making sure she had a healthy self-image and mind as well, and having to worry about how the world she was coming into was going to have an impact on that. I think it might have been around the time that I became aware of the book Cinderella Ate My Daughter that I had this realisation. I started taking more of an interest in, what I guess I'd call feminist issues. I became a seed funder for the Ada Initiative, and I tend to prick my ears up when I hear about feminist issues. So when an email went by on the parents list at work about a screening of an independent documentary film called Miss Representation, I watched the trailer, and since I'm currently swanning around sans wife and child, ponied up the $10 to go and see a screening of it in Palo Alto, which was followed by a Q&A session with the writer/director/producer. I basically went along on the strength of the trailer, and being a father of a daughter, and didn't do much more reading into the who the writer/director/producer was. It turns out that Jennifer Siebel is Gavin Newsom's wife. I didn't figure that out until I got home and did my homework, so some of the references she made in the conversation after the screening didn't make a lot of sense to me at the time. The documentary itself was very poignant, but I found that after about 40 or 50 minutes, I felt that the point was made, and I would have preferred to see more about what to do about it. But I guess what to do about it is to be aware of the problem. I had some trouble accepting the statistic quoted at the beginning of the film that the average teenager spends more than 10 hours each day consuming media. If they sleep for 8 hours, and go to school for 6 hours, that means they're spending every other waking minute consuming media, and I just don't buy that. Apparently Jennifer is working on a couple of related follow-up documentaries, so it'll be interesting to see what they're about and what they're like, and how they're presented to the viewing public. One of the things that disappointed me a bit was this film doesn't look like it's going to do any sort of mainstream theatre run. I'm not sure why, but it seems like the way they're going is for more of a grassroots, small screening thing, possibly as a way of trying to get it added to (I presume private) school curricula. It is getting a couple of showings on Oprah Winfrey's cable channel OWN, but that doesn't seem like enough to me. I think it'd really cause people to sit up and take notice if it got half-decent box office numbers and a bit more press coverage as a result. It's the kind of thing I could see going viral if it gets more exposure. The other thing that really disappointed me was the lack of men in the audience. I think I could have counted on both hands the number of men there. But I have to say that I wouldn't have heard about it (yet, anyway) if it weren't for that email that went by on the mailing list at work, so maybe it just doesn't have much of a profile yet. I view this documentary in the same calibre as Super Size Me, or anything that Michael Moore has put out. It's a documentary trying to highlight an issue with American culture. It should get as widespread a viewing as possible to get the conversation going. Instead, as best I can tell, there's two DVDs billed as "educational material", with what I consider prohibitive pricetags. I want to purchase a copy of the film for posterity (if nothing more, it'll be interesting to show Zoe a snapshot of American culture from around when she was born), but I'd also like to get my hands on the age-appropriate educational stuff so that I can use it with Zoe. The likelihood of Zoe going to a school that incorporates the material into its curriculum is slim (she'll be educated in Australia), so I'd really like to be able to use it at home. I'll end with this thought provoking article that also did the rounds of the parents list at work a few months ago.

22 October 2011

Francois Marier: Reducing the size of Apache 301 and 302 responses

Looking through the Libravatar access logs, I found that most of the traffic we currently serve consists of 302 redirects to Gravatar. Optimising that path is therefore very important.

While Apache allows admins to provide custom error pages for things like 404 or 500, it's not quite that straightforward for 30x return codes.

Standard 301 / 302 responsesBy default, Apache (and most web servers out there) returns a fairly large HTML page along with a 30x redirection. Try it for yourself by disabling automatic redirections in Firefox (Preferences Advanced General Accessibility) or by installing the Request Policy add-on.

The 302 responses sent by Libravatar looked like this:
$ curl -i http://cdn.libravatar.org/avatar/12345678901234567890123456789012
HTTP/1.1 302 Found
Date: Wed, 21 Sep 2011 01:51:52 GMT
Server: Apache
Cache-Control: max-age=86400
Location: http://www.gravatar.com/avatar/12345678901234567890123456789012.jpg?r=g&s=80&d=http://cdn.libravatar.org/nobody/80.png
Vary: Accept-Encoding
Content-Length: 310
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="http://www.gravatar.com/avatar/12345678901234567890123456789012.jpg?r=g&s=80&d=http://cdn.libravatar.org/nobody/80.png">here</a>.</p>
</body></html>
As you can see, the body of the response is just as large as the headers and isn't really necessary.

Body-less 301 responsesAfter reading about the ErrorDocument directive, I created an empty file called 302 in the root of the web server and included this directive in my vhost configuration file:
ErrorDocument 302 /302
which made the responses look like this:
$ curl -i http://example.com/redir
HTTP/1.1 302 Found
Date: Wed, 21 Sep 2011 03:39:26 GMT
Server: Apache
Last-Modified: Wed, 21 Sep 2011 03:39:17 GMT
ETag: "8024d-0-4ad6b52201036"
Accept-Ranges: bytes
Content-Length: 0
Content-Type: text/plain

This one does have a completely empty body, however, there's an important problem with this solution: the Location header is missing! Not much point in reducing the size of the redirect if it's no longer working.

Custom 302 response pageThe next thing I tried (and ended up settling on) is this:
ErrorDocument 302 " "
which results in a 1-byte response (a single space) in the body:
$ curl -i http://example.com/redir
HTTP/1.1 302 Found
Date: Wed, 21 Sep 2011 03:37:50 GMT
Server: Apache
Location: http://www.example.com
Vary: Accept-Encoding
Content-Length: 1
Content-Type: text/html; charset=iso-8859-1

There is still a little bit of unnecessary information in this response (character set, Vary and Server headers), but it's a major improvement over the original.

If you know of any other ways to reduce this further, please leave a comment!

Vincent Sanders: I do not want anything NASty to happen

I have a lot of digital data to store, like most people I have photos, music, home movies, email and lots of other random data. Being a programmer I also tend to have huge piles of source code and builds lying about. If all that was not enough I work from home so I have copious mountains of work data too.

Many years ago I decided I wanted a single robust, backed up, file server for all of this. So I slapped together a machine from leftovers stuffed some drives in a software RAID array, served over NFS and CIFS and never looked back.

Over time the hardware has changed and the system upgraded but the basic approach of a custom built server has remained. When I needed a build engine to churn out hundreds of kernels a day for the ARM Linux autobuilder the system was expanded to cope and mid 2009 the current instantiation was created.

Current full height tower fileserverThe current system is a huge tower case (courtesy of Mark Hymers) containing a Core 2 Quad 2.33GHz (8 threads) with 8Gigabytes of memory and 13 drives across four SATA controllers split into several RAID arrays. Despite buying new drives at higher capacities I have tended to keep the old drives around for extra storage resulting in what you see here.

I recently looked at the power usage of this monster and realised I was paying a lot of money to spin rust which was simply uneconomic. Seriously, why did I have six sub 200Gigabyte drives running when a single 2T to replace them would pay for itself in power saved in under a month! In addition I no longer required the compute power available either, most definitely time for a downsize!

Several friends suggested a HP micro server might be just the thing. After examining and evaluating some other options (Thecus and QNAP NAS) I decided the HP route was most definitely the best value for money.

The HP Proliant micro server is a dual core Athlon II 1.3GHz system with a Gigabyte of memory, space for four SATA hard drives and a single 5 inch bay for an optical drive. All this in a roughly 250mm on a side cube.

My HP proliant microserverI went out and bought the server from ebuyer for 235 with free shipping and 100 cashback. I Immediately sent off the cash back paperwork so I would not forget(what an odd way to get discount) so total cost for the unit was 135. I then used Crucial to select a suitable memory upgrade to take the total to 2 Gigabytes of RAM for 14

The final piece of the solution was the drives for the storage. I decided the best capacity to cost ratio could be had from 2 TB drives and with four bays available would give a raw capacity of 8 TB or more usefully for this discussion 7.8 TiB

I did an experiment with 3x1 TB 7200 RPM drives from the existing server and determined that The overall system would not really benefit enough to justify the 50% price premium of 7200 RPM drives over 5400 RPM devices. I ended up getting four Samsung Spinpoint F4EG 2 TB drives for 230.

I also bought a black LG DVD-RW drive for 16 I would have also required a SATA data cable and a molex to SATA power cable if I had not already got them.

My HP microserver with the front door openPutting the components together was really simple. The internal layout and design of the enclosure mean it is easy to work with and has the feel of build quality I usually associate with HP and IBM server kit not something this small and inexpensive.

The provided documentation is good but unnecessary as most operations are obvious. They even provide the bolts to attach all the drives along with a wrench in the lockable front door, how thoughtful is that!

I then installed the system with Debian squeeze from the optical drive. Principally because I happened to have a network installer CD to hand although the BIOS does have network boot capability.

I used the installer to put the whole initial system together and did not have to resort to the command line even once, very impressed with how far D-I has come.

After asking several people for advice the general consensus was that I should create two partitions on each drive one for a RAID 1 /boot and one for a RAID 5 LVM area.

I did have to perform the entire install a second time because there is a gotcha with GUID Partition Table, RAID 1 boot drives and GRUB. You must have a small "BIOS" partition on the front of the drive or GRUB cannot install in the MBR and your system will not boot!

The partition layout I ended up with looks like:
Model: ATA SAMSUNG HD204UI (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 17.4kB 32.0MB 32.0MB bios_grub
3 32.0MB 1000MB 968MB raid
2 1000MB 2000GB 1999GB raid

The small Gigabyte partition was configured as a RAID 1 across all four drives and formatted with ext2 and mount point of /boot. The large space was configured as RAID 5 across all four drives with LVM on top. Logical volumes were allocated formatted ext3 (on advice from seevral people about ext4 instability they had observed) for 50 GiB root, 4 GiB swap and 1 TiB home space.

The normal Debian install proceeded and after the post install reboot I was presented with a login prompt. Absolutely no surprises at all no additional drivers required and a correctly running system.

Over the next few days I did the usual sysadmin stuff, rsynced data from the old fileserver including creating logical volumes for the various arrays from the old server none of which presented much of a problem. The 5.5TiB Raid 5 did however take a day or so to sync!

I used the microservers eSATA port to attach external drives I use for backup purposes which has also not been an issue so far.

I am currently running both the new and old systems for a few days and rsyncing data to the microserver until I am sure of it. Actually I will make the switch this weekend and shut the old system down and leave it till next weekend before I scrub the old drives.

Before I made it live I decided to run some benchmarks and gather some data just for interest.
Bonnie (Version 1.96) was run in the root logical volume (I repeated the tests in other volumes, there is sub 1% variation) the test used a 4GiB size and 16 files

Sequential OutputSequential InputRandom SeeksSequential CreateRandom Create
Per ChrBlockRewritePer ChrBlockCreateReadDeleteCreateReadDelete
/sec378K41M37M2216K330M412.811697+++++1833014246+++++14371
%CPU9711891301524+++2829+++22
Latency109ms681ms324ms116ms93389 s250ms29021 s814 s842 s362 s51 s61 s

Does not seem to be any notable issues there, the write speeds are a little lower than I might like but that is the cost of RAID 5 and 5400 RPM drives.

The rsync operations used to sync up the live data seem to manage just short of 20MiB/s for the home partition comprising of 250GiB in two and a half million files with the expected mix of file sizes. The video partition managed 33MiB/s on 1TiB of data in nine thousand files.

The bonnie tests were performed accessing the server over NFS with 24GiB size and 16 files.
Sequential OutputSequential InputRandom SeeksSequential CreateRandom Create
Per ChrBlockRewritePer ChrBlockCreateReadDeleteCreateReadDelete
/sec1733K29M19M4608K106M358.3146537142402157640821529
%CPU98249310108109997
Latency10894 s23242ms69159ms49772 s224ms250ms148ms24821 s157ms108ms2074 s719ms

or alternatively as percentages against the previous direct access values

Sequential OutputSequential InputRandom SeeksSequential CreateRandom Create
Per ChrBlockRewritePer ChrBlockCreateReadDeleteCreateReadDelete
/sec4646851213328712+++1311+++10
CPU1011850104347133+++3231+++31
Latency925121324893227795093049186462983440661178688

Not that that tells us much aside from that write is a bit slower over the network, read is gigabit network bandwidth limited and latency of disc over the network is generally poorer than direct.

In summary the total cost was 395 for a complete ready to use system with 5.5TiB of RAID 5 storage which can be NFS served at nearly 900Mbit/s. Overall I am happy with the result, my only real issue is the write performance is a little disappointing but it is good enough for what I need.

Adnan Hodzic: My DebConf11 summary and its after effects

Even though some three months have passed since DebConf11 has successfully ended, I still wanted to give you just a glimpse on how some parts were played through the eyes of a lunatic (read: organizer). Of course, blog post can t come close near of explaining anything but at least it should give some insight and hopefully some pointers to the future DebConf organizers. If you don t feel like reading this much text you can listen to last episode of This Week In Debian podcast and after it just head to Beginning of the end part. DebConf11 First of all I d like to apologize if I offended or hurt anyone in this whole process as it wasn t my intention and during DebConf organization this is nothing irregular, for you to get hurt or you hurting somebody else. One thing that got stuck with me this whole time is when Mart n Ferrari approached me after I did bid proposal of Bosnia/Herzegovina for DebConf11 back on DebConf9 and told me you have no idea what you just got yourself into I played cool and said something along the line of coure I do and boy did I lie. During DebConf organization you re bound to make some of your decisions instantaneously in which you ll lose something, the most you can do is assess what s the thing you can cut the cord on, even though if that thing or a person for that matter might have meant something to you. You ll lose things, relationships, friendships, contacts at the times sense of humor and maybe even common logic. Some of it comes back to you, some the very next day and for some it may take longer. Some of it was lost irreversibly, but even in that case it wasn t lost and could even be traced to some other cause. One thing that I m thankful for is that I didn t lose, not lose but even get distanced from those close to me, and that I didn t lose guy who was with me in this from very first day, my right hand, my partner in crime Velimir ( aroundthfur ). It s not even losing I m talking about here, because even by losing you re gaining something and this is part of your everyday life, it s just that your life frame kinda gets stuck on fast forward button. So once the ride s over you may feel bit dizzy asking yourself what just happened. During this time I was also contacted by Google recruiter regrading software engineer internship but to make a long story short I didn t make it due to my heavy involvement and my lack of time (read: fail). But I don t find this as losing either, first time I was contacted by them, next time they ll be contacted by me maybe even for different position this time (project manager? Big Smile. Even this DebConf wasn t our first bid we lost DebConf7 bid to Edinburgh, but it is perseverance that got us here. The moral I m trying to tell here is if you have a goal never back down from it. What I found to be our biggest problem and what I personally found as most painstaking in all of this was having and reporting to so many different sides and different groups while trying to keep them in same communication channel. Even my right hand (Velimir) was apart from me (Vienna), then place where two of us are coming from is not the place where DebConf is getting organized in. As in the end we ended up having all the parties separated and pulling to their own side, from Government of Republika Srpska to NGO to Debian to local team and to those who remained as undeclared . Option of cutting the cord in this case was missing as these were the components that DebCon11 was made of. Of course this meant you ll be missing workforce and there were days you d wake up driving >= 200 km/h (literally) with one hand on the wheel, other on the phone and your other right hand (Velimir) holding your RedBull. But you wouldn t do this just because you had to and there was no one else to do it, you d do it because you re responsible for the sake of the project and won t be just sitting there because by some unwritten hierarchy rules you re supposed to sit and supervise things. I d advise you do the same thing when you see there s no one else to do something, even if it means making coffee for one of the teams. I m not trying to scare you away from organizing your own DebConf, on the contrary after all I ve been through in past two years if someone asked me would I do it if I knew it was like this, without a doubt my answer would still be fsck yea . All the previous DebConf organizers told me about all of this, but why didn t I give up? For the same reason you won t give up and for the same reason why we believe in the same goal. If anything this is one of the stories you ll want to tell your kids about so When I was your age I used to sleep naked in the snow would turn to When I was your age I organized DebConf11 in Maybe this blog post wasn t even necessary and some things weren t supposed to be said, I guess sometimes I just can t shut up Smile For those who were on DebConf11 I hope you enjoyed and had a great time, otherwise I m saying all of this in vein. As organizer all you see is problems and you miss most of the confrence, but by what I know there are 11 candidate bids for DebConf13 and for me hearing that tells me the results of our efforts. DebConf11 after effects I could write a whole novel about this, but to keep it as short as possible, for last two years as a side project I was working on an idea of Government or some of its institutions migrating to Linux. At first I was somewhat loud about it, then after Microsoft heard about it and after they tried stopping the idea by trying to scare me by trying to interfere with my private life; as that didn t work its lobbyist came even near of obstructing the whole conference within the Government. For the sake of the conference, I convinced the Government that by supporting DebConf it doesn t mean they need to move to Linux and publicly stopped talking about it. I also convinced them that our only goal was to have successful conference and promote alternative options and open ideas. I wasn t lying as I saw this as new opportunity of them concluding on their own why they should or shouldn t not move, the better conference was the more chances of success we had. That s why I tried pushing as many representatives from various companies as in this case we would use reverse psychology where basically no one or few know what Linux or Debian for that matter is, but everybody knows who Google is, so if you have participants from i.e: Google or Austrian E-Health care system talking about how they are using your technology is better way to explain what s it all about really. Eventually Microsoft even had their first ever conference in Bosnia/Herzegovina and you wanna take a wild guess where it was held? Smile In the end we had a great conference, after the conference we were the ones that were approached by some big local companies interested in future co-operation and in the end a meeting with Mark Shuttleworth and President along with the core of Government was scheduled. Topic? Migration to Linux. For me personally this meeting went better then I could possibly even picture it, many topics were discussed and basically it was up to us/me to make a draft of the project plan and submit request for proposal to the Government. There was still some lobbying but it seemed as it all disappeared, runway was clear and open for the lift off. Necessary contacts with potential future partners were made and before any meetings were scheduled I announced I ll be taking a short break. I missed this years vacation, for matter of a fact I missed going to a vacation for last two years. I decided going to Dubrovnik for few days, just to recollect my thoughts, recompile my kernel and switch to lower gear as I was stuck in 6th gear for way too long. Beginning of the end Being on this vacation , even if it was only for few days this gave me an opportunity to look at it all from a different prospective, different angle and that s when it came to me. I realized that even before I was finished with one project without even realizing I was already involved with another one, one that would take much more time and one that might leave me wondering ~5 years later with question what if? After all if get involved with something I m not going to back out, it s either failure is not an option or I won t start doing it at all. Even if it brought me everything I wanted, how would you know if that s what you wanted if you haven t thought about it first? Having DebConf for the first time in this region already showed some (positive) results, but for seeing the actual results it may take a bit longer. We sowed the seeds now the best thing we can do is leave them as it is and see if anything grows out. Of course I could lean over them and protect them from any natural or any other type of hazards, this would also mean when I m not around it wouldn t know how to protect itself on its own. Besides all, interfering with natures way of doing things is rarely a good option. I consider myself to still be relatively young and threatening my well being over safe keeping some seeds and watching over if they grow into something or not doesn t really seem that attractive to me. Maybe one day I ll decide to leave the land, maybe I won t even get to see if anything grows out after all. Maybe some day someone will find this fruit, recognize its worth and claim it as it s own, maybe over time everybody forgets everything and even that this land once was a fertile ground. Maybe, maybe not. I m still a student, I was supposed to graduate last year but I took less classes last year so I wouldn t snap under all the pressure. That s why this year I have less classes and am supposed to graduate in May, that s less then a year until then and I just want to devote some time to myself, figure out what I want with me life, learn the things I want to learn, do the things I want to do. I want to turn back to working on Eclipse, I have some good plans for Eclipse and Android developers, lately I ve been slacking off way too much but I think I have a good excuse Smile In the end, all I know is that I still want to change the world, whether it s doing that inside of Google or in my own company, or some third place; or I ll just be laughing at this statement only time will show. I m hoping that now with all of this out of me I can finally finish the DebConf11 chapter of my life.

21 October 2011

Tollef Fog Heen: Today's rant about RPM

Before I start, I'll admit that I'm not a real RPM packager. Maype I'm approaching this from completely the wrong direction, what do I know? I'm in the process of packaging Varnish 3.0.2 which includes mangling the spec file. The top of the spec file reads:
%define v_rc
%define vd_rc % ?v_rc:-% ?v_rc 
Apparently, this is not legal, since we're trying to define v_rc as a macro with no body. It's however not possible to directly define it as an empty string which can later be tested on, you have to do something like:
%define v_rc % nil 
%define vd_rc % ?v_rc:-% ?v_rc 
Now, this doesn't work correctly either. % ?macro tests if macro is defined, not whether it's an empty string so instead of two lines, we have to write:
%define v_rc % nil 
%if 0% ?v_rc  != 0
%define vd_rc % ?v_rc:-% ?v_rc 
%endif
The 0 ?v_rc != 0 workaround is there so that we don't accidentially end up with == 0 which would be a syntax error. I think having four lines like that is pretty ugly, so I looked for a workaround and figured that, ok, I'll just rewrite every use of % vd_rc to % ?v_rc:-% ?v_rc . There are only a couple, so the damage is limited. Also, I'd then just comment out the v_rc definition, since that makes it clear what you should uncomment to have a release candidate version. In my naivety, I tried:
# %define v_rc ""
# is used as a comment character in spec files, but apparently not for defines. The define was still processed and the build process stopped pretty quickly. Luckily, doing # % define "" seems to work fine and is not processed. I have no idea how people put up with this or if I'm doing something very wrong. Feel free to point me at a better way of doing this, of course.

Russell Coker: Dedicated vs Virtual Servers

A common question about hosting is whether to use a dedicated server or a virtual server. Dedicated Servers If you use a dedicated server then you will face the risk of problems which interrupt the boot process. It seems that all the affordable dedicated server offerings lack any good remote management, so when the server doesn t boot you either have to raise a trouble ticket with the company running the Data-Center (DC) or use some sort of hack. Hetzner is a dedicated server company that I have had good experiences with [1], when a server in their DC fails to boot you can use their web based interface (at no extra charge or delay) to boot a Linux recovery environment which can then be used to fix whatever the problem may be. They also charge extra for hands-on support which could be used if the Linux recovery environment revealed no errors but the system just didn t work. This isn t nearly as good as using something like IPMI which permits remote console access to see error messages and more direct control of rebooting. The up-side of a dedicated server is performance. Some people think that avoiding virtualisation improves performance, but in practice most virtual servers use virtualisation technologies that have little overhead. A bigger performance issue than the virtualisation overhead is the fact that most companies running DCs have a range of hardware in their DC and your system (whether a virtual server or a dedicated server) will be on a random system from their DC. I have observed hosting companies to give different speed CPUs and for dedicated servers different amounts of RAM for the same price. I expect that the disk IO performance also varies a lot but I have no evidence. As long as the hosting company provides everything that they offered before you sign the contract you can t complain. It s worth noting that CPU performance is either poorly specified or absent in most offers and disk IO performance is almost never specified. One advantage of dedicated servers in this regard is that you get to know the details of the hardware and can therefore refuse certain low spec hardware. The real performance benefit of a dedicated server is that disk IO performance won t be hurt by other users of the same system. Disk IO is the real issue as CPU and RAM are easy to share fairly but disk performance is difficult to share and is also a significant bottleneck on many servers. Dedicated servers also have a higher minimum price due to the fact that a real server is being used which involves hardware purchase and rack space. Hetzner s offers which start at E29 per month are about as cheap as it s possible to get. But it appears that the E29 offer is for an old server new hardware starts at E49 per month which is still quite cheap. But no dedicated server compares to the virtual servers which can be rented for prices less than $10 per month. Virtual Servers A virtual server will typically have an effective management interface. You should expect to get web based access to the system console as well as ssh console access. If console access is not sufficient to recover the system then there is an option to boot from a recovery device. This allows you to avoid many situations that could potentially result in down-time and when things go wrong it allows you to recover faster. Linode is an example of a company that provides virtual servers and provides a great management interface [2]. It would take a lot of work with performance monitoring and graphing tools to give the performance overview that comes for free with the Linode interface. Disk IO performance can suck badly on virtual servers and it can happen suddenly and semi-randomly. If someone else who is using a virtual server on the same hardware is the target of a DoS attack then your performance can disappear. Performance for CPU is generally fairly reliable though. So a CPU bound server would be a better fit on the typical virtual server options than a disk IO bound server. Virtual servers are a lot cheaper at the low end so if you don t need the hardware capabilities of a minimal dedicated server (with 1G of RAM for Hetzner and a minimum of 8G of RAM for some other providers) then you can save a lot of money by getting a virtual server. Finally the options for running a virtual machine under a virtual machine aren t good, AFAIK the only options that would work on a commercial VPS offering are QEMU (an x86 CPU instruction emulator), Hercules (a S/370 S/390, and Z series IBM mainframe emulator), and similar CPU emulators. Please let me know if there are any other good options for running a virtual machine on a VPS. Now while these emulators are apparently good for debugging OS development they aren t something that are generally useful for running a virtual machine. I knew someone who ran his important servers under Hercules so that x86 exploits couldn t be used for attacking them, but apart from that CPU emulation isn t generally useful for servers. Summary If you want to have entire control of the hardware or if you want to run your own virtual machines that suit your needs (EG one with lots of RAM and another with lots of disk space) then a dedicated server is required. If you want to have minimal expense or the greatest ease of sysadmin use then a virtual server is a better option. But the cheapest option for virtual hosting is to rent a server from Hetzner, run Xen on it, and then rent out DomUs to other people. Apart from the inevitable pain that you experience if anything goes wrong with the Dom0 this is a great option. As an aside, if anyone knows of a reliable company that offers some benefits over Hetzner then please let me know. What I would Like to See There is no technical reason why a company like Linode couldn t make an offer which was a single DomU on a server taking up all available RAM, CPU, and disk space. Such an offer would be really compelling if it wasn t excessively expensive. That would give Linode ease of management and also a guarantee that no-one else could disrupt your system by doing a lot of disk IO. This would be really easy for Linode (or any virtual server provider) to implement. There is also no technical reason why a company like Linode couldn t allow their customers to rent all the capacity of a physical system and then subdivide it among DomUs as they wish. I have a few clients who would be better suited by Linode DomUs that are configured for their needs rather than stock Linode offerings (which never seem to offer the exact amounts of RAM, disk, or CPU that are required). Also if I had a Linode physical server that only had DomUs for my clients then I could make sure that none of them had excessive disk IO that affected the others. This would require many extra features in the Linode management web pages, so it seems unlikely that they will do it. Please let me know if there is someone doing this, it s obvious enough that someone must be doing it. Update: A Rimuhosting employee pointed out that they offer virtual servers on dedicated hardware which meets this criteria [3]. Rimuhosting allows you to use their VPS management system for running all the resources in a single server (so no-one else can slow your VM down) and also allow custom partitioning of a server into as many VMs as you desire.

20 October 2011

Pau Garcia i Quiles: The Secure Boot controversy

Red Hat s Matthew Garrett let the cat out of the bag about a month ago: when UEFI Secure Boot is adopted by mainboard manufacturers to satisfy Microsoft Windows 8 requirements, it may very well be the case that Linux and others (BSD, Haiku, Minix, OS/2, etc) will no longer boot. Matthew has written about it extensively and seems to know very well what the issues are (part I, part II), the details about signing binaries and why Linux does not support Secure Boot yet. The Free Software Foundation has also released a statement and started a campaign, which is, as usually, anti-Microsoft instead of pro-solutions. Now let me express my opinion on this matter: this is not Microsoft s fault. Facts Let s see what are the facts in this controversy: Problem The problem stems from the fact that most Linux vendors do not have the power to get their certificates in UEFI BIOS. Red Hat and Suse will for sure get their certificates bundled in server UEFI BIOS. Debian and Ubuntu? Maybe. NetBSD, OpenIndiana, Slackware, etc? No way. This is, in my humble opinion, a serious defect in the standard. A huge omission. Apparently while developing the Secure Boot specification everybody was busy talking about signed binaries, yet nobody thought for a second how the certificates will get into the UEFI BIOS. What should have been done The UEFI secure boot standard should have defined an organization (a Secure Boot Certification Authority ) that would issue and/or receive certificates from organizations/companies (Red Hat, Oracle, Ubuntu, Microsoft, Apple, etc) that want their binaries signed. This SBCA would also be in charge of verifying the background of those organizations. There is actually no need for a new organization: just use an existing one, such as Verisign, that carries on with this task for Microsoft for kernel-level binaries (AuthentiCode). Given that there is no Secure Boot Certification Authority, Microsoft asked BIOS (UEFI) developers and manufacturers to include their certificates, which looks 100% logical to me. The fact that Linux distributions do not have such power is unfortunate, but it is not Microsoft s fault at all. What can we do? Given its strong ties with Intel, AMD and others, maybe the Linux Foundation could start a task force and a Temporary Secure Boot Certification Authority to deal with UEFI BIOS manufacturers and developers. This task force and TSBCA would act as a proxy for minorities such as Linux, BSD, etc distributions. I am convinced this is our best chance to get something done in a reasonable amount of time. Complaining will not get us anything. Screaming at Microsoft will not get us anything. We need to propose solutions. Wait! Non-Microsoft certificates? Why? In addition to the missing Secure Boot Certification Authority, there is a second problem apparently nobody is talking about: what is the advantage mainboard manufacturers get from including non-Microsoft certificates? For instance: why would Gigabyte (or any other mainboard manufacturer) include the certificate for, say, Haiku? The benefit for Gigabyte would be negligible and if someone with ill-intentions gets Haiku s certificate, that piece of malware will be installable on all Gigabyte s mainboards.This would lead to manufacturer-targetted malware, which would be fatal to Gigabyte: oh, want to be immune to the-grandchild-of-Stuxnet? Buy (a computer with) an MSI mainboard, which does not include Haiku s certificate Given that 99% of desktops and laptops only run Windows, the result of this (yet unresolved) problem would be that manufacturers will only install Microsoft certificates, therefore they would be immune to malware signed with a Slackware certificate in the wild. If we are lucky, mainboard manufacturers will give us an utility to install more certificates under your own risk. The solution to the first problem looks easy to me. The solution to the second looks much more worrying to me.

Next.

Previous.