Search Results: "ag"

24 October 2011

Gregor Herrmann: managing $HOME

I'm not the first one who thought about managing their dotfiles & scripts in a (D)VCS in order to be able to have them in sync across different machines, & I even haven't re-invented the wheel for doing so. short report on my experiences:

after looking around a bit I decided to try movein (info page with links to downloads & a README, blog post, git repo) because it looked liked it would easily do what I'd like to do without getting in my way too much; which means e.g.: no symlink farm, & only one simple command to manage the whole setup.

movein is a simple shell script that allows to setup & manage multiple modular repos tracking different sets of files quite easily, taking advantage of the powers of git & mr. & being written in shell means I also was able to make a few changes, which stew kindly accepted & applied to his tree already.

in fact I'm just starting to use it for real, but it looks good so far, & I think I'm going to stick with it for the time being.

2 minor observations: </promotion_time>

Dirk Eddelbuettel: Kurt Elling and the Kl vers Big Band at the Green Mill

Twenty-four hours ago I was in heaven. Or to more properly recapitulate and situate this, fourty-eight hours ago I was in hell. Now this requires some context. Earlier in the year, I had tweeted briefly about a Kurt Elling live concert still streaming on NPR, and truth be told, I have listened to Kurt Elling a whole lot ever since. The most recent album The Gate is fantastic, the Grammy-winner Dedicated to You (which retakes the very famous Coltrane/Hartman album) is beyond words, and The Messenger is worth it just for 'Nature Boy' and the funkiest-ever 'April in Paris'. I also saw him as part of a larger and otherwise rather neat tour. But I needed to see the man himself. So following what one can see from his touring calendar, I jotted down two October dates. Kurt Elling at the Green Mill. Fine. That was earlier in the summer. Summer has a tendency of evaporating quickly, so here we were in October. And getting going late on Friday, so by the time we got the Green Mill, a long line had formed and we ended up paying tribute to the line for fourty-five minutes with nothing to show. Sad. Home we went, and no live music. But somehow we mustered the energy to go again last night and thought we were early, arriving about 20 minutes before the show was to start. Ha! Others had appeared at 3pm, by 5pm all seats were gone. We were literally the last ones the get in after a short spell in a shorter line, and standing room-only it was. Oh, but what a treat we got. The set for these two days was Kurt Elling with the fourteen piece Kl vers Big Band from Denmark. They played for three marvellous sets, and well over four hours (with two 25 minute breaks). Pieces often alternated between big band arrangements and smaller pieces by Elling's standard rhythm band, and covered standards as well as Elling's catalogue. As I mentioned earlier on a short post (with pictures) on Google+, it is essentially impossible to not fall for Elling's stage presence when being so close to the stage. The man is just that good. Elling and the Kl vers Big Band will now play for six nights at Birdland in New York followed by a night each in Washington, DC, and Boston. If you're around, do yourself a favour and go out to see them.

23 October 2011

Asheesh Laroia: RFBP: Request for birthday present/package

There is a program that I love: bb. bb is a demo of the famous ASCII Art library, aalib.
     dT8  8Tb     
    dT 8  8 Tb    
   dT  8  8  Tb   
<PROJECT><PROJECT>
 dT    8  8    Tb 
dT     8  8     Tb
bb is a demo-scene-type program that shows how awesome automatic ASCII art is. The personalities of the people who made bb shine through. It's surely one of my favorite programs in Debian, up there with alpine. It's been in Debian since 1998. bb has a serious bug, however: BB's "graphics" freeze when music starts. Here's the issue. So if your system (like most GNU/Linux systems) uses pulseaudio for sound, then bb is broken. That means every Ubuntu user and most desktop Debian users can't use it. There are a few possible fixes, depending on where you'd want to solve the problem. If you just want bb to run on your own machine, without recompiling anything, you can adjust pulseaudio's configuration (in /etc/pulse/default.pa) to disable esound support. If you want to do that, just comment out this line:
 load-module module-esound-protocol-unix
We could also possibly patch bb so that it asks libmikmod not to use its esound "support." I think the smarter thing to do is to adjust libmikmod. Since its esound support seems to be just plain broken, it should be removed. At very least, it should not be the default when ALSA output is available. There is a new upstream release of libmikmod, maybe the esound output is fixed. In Debian, libmikmod is orphaned. When a package is orphaned, it means that a new person must step in and adopt the package. Debian packages need ongoing care and commitment to fix issues and make changes like this that benefit the users. In this case, you'd need to understand some C and be willing to maintain a shared library. Maintaining a library in Debian requires attention to detail, but it is quite doable. Since you would be adopting an existing package, most of the work is already done for you. I would also be quite willing to answer questions. If you're not a Debian developer, I would happily sponsor uploads of this package into Debian so that the fixes are part of the distribution. So: who will maintain libmikmod and fix bb? Could it be you? It would make a really great birthday present if the amazing bb program worked in the next Debian release. Leave a comment if you have questions or are interested! P.S. In a pinch, I can be convinced to maintain libmikmod myself, but I think this is a great opportunity for someone new to Debian to make a big difference.

Gregor Herrmann: RC bugs 2011/42

this week my activities were split between some freshly-reported pkg-perl bugs, & looking at some of the older RC bugs. for most of the ones in the latter category I found again ready-to-be-applied patches, many of them from Ubuntu again. details:

Uwe Hermann: The TrekStor eBook Reader 3.0 (EBR30-a), review and dissection

The TrekStor eBook Reader 3.0, front
The TrekStor eBook Reader 3.0, front on There a many, many, e-book reader devices available these days, and they're quickly becoming pretty affordable. The currently cheapest device in Germany (that I know of) is the TrekStor eBook Reader 3.0, model number EBR30-a, at 59.- Euros via Weltbild or Hugendubel. The device has an 800x480 7" TFT (yep, no e-ink), 2100mAh battery, it can display PDFs, EPUB, and TXT files (and Adobe DRM crap, which I don't really care about), it has an accelerometer which allows for landscape/portrait switching, it can play MP3, OGG, WAV, and WMA audio files (headphone jack), it can display pictures (BMP, GIF, JPG, even PNG, though that's not mentioned in the vendor's specs), and it has 2GB internal storage for books/music/pictures. Uploading of (non-DRM) content is done by a simple file copy, it enumerates as a standard USB mass storage device with FAT filesystem. It's a relatively nice reader for the price, I've read a few PDFs (datasheets, presentations) on it in the subway/train while listening to music from the device and it's quite OK for my purposes. So much for the review part. However, I didn't really buy it for reading books on it, I was more interested in taking it apart, of course ;-) My hope was that it would turn out to be a really cheap device running Linux/U-Boot which would be perfect for playing around with embedded Linux stuff. Unfortunately, I wasn't so lucky (it seems). The TrekStor eBook Reader 3.0, opened I've posted a few photos of the device and its hardware components on my flickr account and over at randomprojects.org, together with all the information I was able to find out so far. Here's a quick summary: The TrekStor eBook Reader 3.0, CPU There are public datasheets for most of the hardware components (see randomprojects.org for links), but unfortunately the most important one (for the CPU) is not yet found/identified. I was told that the CPU/SoC is probably based on an ARM9 (ARM926EJ-S) core and the firmware running on it seems to be some uCos-based RTOS (not Linux, unfortunately). So far I was not able to find out the vendor name or website of the "FI E200" CPU/SoC (let alone any datasheets), any hints would be highly appreciated. I checked arm.com: Processor Licensees, but the only two companies whose name starts with "F" having licensed an ARM9 core are Fujitsu and Freescale, which doesn't fit, I think? I could (and probably will) check the PCB for RX/TX lines on an UART and/or JTAG pads (none are obviously labelled), and given that it's and ARM9 core there is a good chance that OpenOCD can be used and that a standard cross-gcc for ARM will work. However, that is all pretty pointless until it's clear which SoC exactly is used, and thus whether there is already Linux and/or U-Boot support for it and/or whether datasheets are available so that the respective code could be written. Without datasheets, this is going to be a pretty painful experience, not really worth investing much time, IMHO. If anyone knows more about the vendor/device and respective datasheets, please let me know. Thanks!

Sylvain Beucler: Portal-style teleporters in OpenGL

Valve's Portal game introduced an innovative teleporter system - where you basically can see through the teleporter and even walk seamlessly through it. Ever wondered how this is implemented? Check this Mini-Portal demo, along with its companion tutorials!
http://en.wikibooks.org/wiki/OpenGL_Programming/Mini-Portal
In this first part, we cover a basic working version that we will improve over the next few days. [Find more tutorials at the OpenGL wikibook.]

Niels Thykier: Testing testing migration

If you have been following Lintian s development closely, you will probably have noticed that I have not really done anything there for the past week. Instead I have turned my focus on our testing migration script, britney2. First, I have created a minimal test suite[1]. It started as 4 simple tests and by now it contains about 30 tests. The size of each test is rather small; the largest tests are about 1600 binary packages in total[2], but most are 2-20 binary packages in total. Thus the test suite is rather fast compared to a live data sample , which easily takes more than 10 minutes for a single run. Unfortunately, hand-crafting the test data is somewhat annoying and easy to get wrong. The test suite has a somewhat unfair focus on auto-hint [3] cases, so the current britney2 fails up to 14 tests. Some of these appears to fail because the auto-hinter (for some reason) receives incomplete information about the situation. To my knowledge we not been able to debug the situation, but Adam has a refactor branch that does not seem to have this issue. Personally I am hoping it will soon be merged into the master branch, especially because it seems to simplify a lot of common operations. Joachim Breitner (who has been working on a SAT-solver based britney) also contributed a couple of test cases[4]. Allegedly, SAT-britney does rather well on the test suite, failing only 2 tests as far as I can tell[5]. On the other hand, it does solve a some of the more interesting cases britney2 does not solve. On a more mathematical note, the britney2 implementation behaves like a function[6] with an attractive fixed point[7]. This is interesting, because for some cases it may take britney2 a couple of iterations to reach the right solution. This fixed point is somewhat simple to find by using the following steps (pseudo-code):
// Runtime complexity O(n * br * diff), where "n" is the number of iterations until
// a fixed point is reached, "br" is the complexity of "run_britney" and "diff" is
// the runtime of the "last != current" comparison.
function find_fixed_point(initial);
    last = run_britney(initial)
    current = run_britney(last)
    while last != current ; do
        last = current
        current = run_britney(last)
    od
    return current
end
This gives us a simple way to test if britney will eventually solve the issue herself (and when she will do it). Currently britney2 is automatically run twice a day, so for every 2 iterations (beyond the first) roughly translates to a 24-hours delay. So far the test suite does not have a lot of problems that requires more than one iteration. Personally I would be pleased if it turned out to stay that way as the test suite coverage grows. If you are interested in playing around with this, you can get sources from: Footnotes: [1] http://lists.debian.org/debian-release/2011/10/msg00178.html [2] These tests are auto-generated, so it is merely an up-scaled pattern . [3] Basically if two (or more) packages needs to migrate into testing at the exact same time, they need to be hinted in. [4] Not to mention all the copy-waste errors he pointed out in mine. Apparently, SAT-britney has stricter requirements to the data than britney2. :P [5] I assume the test called sat-britney-death (created by Joachim) was named that way for a reason. The second failure is caused by SAT britney not reading hints (yet?), so the approve tpu package test case should fail. [6] A function that maps an archive into another archive erh, I mean, it maps a set of packages into another set of packages :P [7] http://en.wikipedia.org/wiki/Fixed_point_%28mathematics%29 Assuming my claim to be true, the function will have more than one fixed point. The obtained fixed point depends on the initial state of testing. As an example:
y depends on x
x in testing has RC bugs If x is not in testing, it cannot migrate to testing (due to its RC bugs). If x is not in testing, then y cannot migrate into testing. But if x starts in testing, then y may be able to migrate. This can happen if x migrated to testing before an RC bug was filed against it. (Dis-)Proving my claim is an exercise left for the reader.

Luca Falavigna: Stats, more stats and, guess what? Even more stats!

We all love stats, don t we? So, here we go! Let s start with a graph: NEW graph It shows the number of packages in the NEW queue since last year. You can see a big drop during April 2011, and a reasonably low rate during the last six months. You could think fellow Debian Developers stopped to upload NEW packages. Sorry, you re wrong! :) Since Squeeze release, 3.832 .changes files with NEW components were processed by dak, with an average of 14,85 NEW packages per day. On the FTP Team side, we had 3.732 accepts (14,47 per day), 339 rejects (1,31 per day) and 178 comments to maintainers (0,69 per day).
Who were the most prolific maintainers who got a NEW processing? Here is our special top ten:
  1. Debian Haskell Group (362 packages)
  2. Debian Perl Group (343 packages)
  3. Debian Java Maintainers (161 packages)
  4. Debian Ruby Extras Maintainers (124 packages)
  5. Debian Multimedia Maintainers (100 packages)
  6. Debian Fonts Task Force (96 packages)
  7. Debian Med Packaging Team (79 packages)
  8. Debian Install System Team (61 packages)
  9. Debian Javascript Maintainers (54 packages)
  10. Debian Python Modules Team (50 packages)
That s bad packaging teams cannot bake cookies!
Let s do the same with Changed By, this time:
  1. Ben Hutchings (159 packages)
  2. Joachim Breitner (138 packages)
  3. Clint Adams (134 packages)
  4. Jonas Smedegaard (124 packages)
  5. TANIGUCHI Takaki (97 packages)
  6. Nicholas Bamber (61 packages)
  7. Alessio Treglia (60 packages)
  8. maximilian attems (54 packages)
  9. David Paleino (51 packages)
  10. Torsten Werner (45 packages)
Much better now go and heat up your ovens, we know who you are ;)
Another nice aspect to look at is the speed of NEW processing. Some maintainers were very happy for a fast NEW processing, someone even complained for having been too quick! :) So, let s find out which upload was the quickest ever. Try to gamble a bit before reading the answer, to see whether you are near to the real value ;) Alessio Treglia, you probably already know, because your gwc_0.21.16~dfsg-1 upload has been processed in 41 seconds (yes, forty-one seconds!). Here s an excerpt from ftp-master log to certify it:
20110516120252 process-upload dak Processing changes file gwc_0.21.16~dfsg-1_amd64.changes
20110516120258 process-upload dak Moving to new gwc_0.21.16~dfsg-1_amd64.changes
20110516120339 process-new tolimar NEW ACCEPT: gwc_0.21.16~dfsg-1_amd64.changes
Alex was the super-fast FTP Team member behind the quickest accept, do you want to beat him? Join FTP Team ;)

Andrew Pollock: [life] Miss Representation

A few months after Zoe was born, it hit me that we didn't just have to raise a baby to grow up happy and healthy, we were raising a girl, and had to worry about making sure she had a healthy self-image and mind as well, and having to worry about how the world she was coming into was going to have an impact on that. I think it might have been around the time that I became aware of the book Cinderella Ate My Daughter that I had this realisation. I started taking more of an interest in, what I guess I'd call feminist issues. I became a seed funder for the Ada Initiative, and I tend to prick my ears up when I hear about feminist issues. So when an email went by on the parents list at work about a screening of an independent documentary film called Miss Representation, I watched the trailer, and since I'm currently swanning around sans wife and child, ponied up the $10 to go and see a screening of it in Palo Alto, which was followed by a Q&A session with the writer/director/producer. I basically went along on the strength of the trailer, and being a father of a daughter, and didn't do much more reading into the who the writer/director/producer was. It turns out that Jennifer Siebel is Gavin Newsom's wife. I didn't figure that out until I got home and did my homework, so some of the references she made in the conversation after the screening didn't make a lot of sense to me at the time. The documentary itself was very poignant, but I found that after about 40 or 50 minutes, I felt that the point was made, and I would have preferred to see more about what to do about it. But I guess what to do about it is to be aware of the problem. I had some trouble accepting the statistic quoted at the beginning of the film that the average teenager spends more than 10 hours each day consuming media. If they sleep for 8 hours, and go to school for 6 hours, that means they're spending every other waking minute consuming media, and I just don't buy that. Apparently Jennifer is working on a couple of related follow-up documentaries, so it'll be interesting to see what they're about and what they're like, and how they're presented to the viewing public. One of the things that disappointed me a bit was this film doesn't look like it's going to do any sort of mainstream theatre run. I'm not sure why, but it seems like the way they're going is for more of a grassroots, small screening thing, possibly as a way of trying to get it added to (I presume private) school curricula. It is getting a couple of showings on Oprah Winfrey's cable channel OWN, but that doesn't seem like enough to me. I think it'd really cause people to sit up and take notice if it got half-decent box office numbers and a bit more press coverage as a result. It's the kind of thing I could see going viral if it gets more exposure. The other thing that really disappointed me was the lack of men in the audience. I think I could have counted on both hands the number of men there. But I have to say that I wouldn't have heard about it (yet, anyway) if it weren't for that email that went by on the mailing list at work, so maybe it just doesn't have much of a profile yet. I view this documentary in the same calibre as Super Size Me, or anything that Michael Moore has put out. It's a documentary trying to highlight an issue with American culture. It should get as widespread a viewing as possible to get the conversation going. Instead, as best I can tell, there's two DVDs billed as "educational material", with what I consider prohibitive pricetags. I want to purchase a copy of the film for posterity (if nothing more, it'll be interesting to show Zoe a snapshot of American culture from around when she was born), but I'd also like to get my hands on the age-appropriate educational stuff so that I can use it with Zoe. The likelihood of Zoe going to a school that incorporates the material into its curriculum is slim (she'll be educated in Australia), so I'd really like to be able to use it at home. I'll end with this thought provoking article that also did the rounds of the parents list at work a few months ago.

22 October 2011

Francois Marier: Reducing the size of Apache 301 and 302 responses

Looking through the Libravatar access logs, I found that most of the traffic we currently serve consists of 302 redirects to Gravatar. Optimising that path is therefore very important.

While Apache allows admins to provide custom error pages for things like 404 or 500, it's not quite that straightforward for 30x return codes.

Standard 301 / 302 responsesBy default, Apache (and most web servers out there) returns a fairly large HTML page along with a 30x redirection. Try it for yourself by disabling automatic redirections in Firefox (Preferences Advanced General Accessibility) or by installing the Request Policy add-on.

The 302 responses sent by Libravatar looked like this:
$ curl -i http://cdn.libravatar.org/avatar/12345678901234567890123456789012
HTTP/1.1 302 Found
Date: Wed, 21 Sep 2011 01:51:52 GMT
Server: Apache
Cache-Control: max-age=86400
Location: http://www.gravatar.com/avatar/12345678901234567890123456789012.jpg?r=g&s=80&d=http://cdn.libravatar.org/nobody/80.png
Vary: Accept-Encoding
Content-Length: 310
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="http://www.gravatar.com/avatar/12345678901234567890123456789012.jpg?r=g&s=80&d=http://cdn.libravatar.org/nobody/80.png">here</a>.</p>
</body></html>
As you can see, the body of the response is just as large as the headers and isn't really necessary.

Body-less 301 responsesAfter reading about the ErrorDocument directive, I created an empty file called 302 in the root of the web server and included this directive in my vhost configuration file:
ErrorDocument 302 /302
which made the responses look like this:
$ curl -i http://example.com/redir
HTTP/1.1 302 Found
Date: Wed, 21 Sep 2011 03:39:26 GMT
Server: Apache
Last-Modified: Wed, 21 Sep 2011 03:39:17 GMT
ETag: "8024d-0-4ad6b52201036"
Accept-Ranges: bytes
Content-Length: 0
Content-Type: text/plain

This one does have a completely empty body, however, there's an important problem with this solution: the Location header is missing! Not much point in reducing the size of the redirect if it's no longer working.

Custom 302 response pageThe next thing I tried (and ended up settling on) is this:
ErrorDocument 302 " "
which results in a 1-byte response (a single space) in the body:
$ curl -i http://example.com/redir
HTTP/1.1 302 Found
Date: Wed, 21 Sep 2011 03:37:50 GMT
Server: Apache
Location: http://www.example.com
Vary: Accept-Encoding
Content-Length: 1
Content-Type: text/html; charset=iso-8859-1

There is still a little bit of unnecessary information in this response (character set, Vary and Server headers), but it's a major improvement over the original.

If you know of any other ways to reduce this further, please leave a comment!

Vincent Sanders: I do not want anything NASty to happen

I have a lot of digital data to store, like most people I have photos, music, home movies, email and lots of other random data. Being a programmer I also tend to have huge piles of source code and builds lying about. If all that was not enough I work from home so I have copious mountains of work data too.

Many years ago I decided I wanted a single robust, backed up, file server for all of this. So I slapped together a machine from leftovers stuffed some drives in a software RAID array, served over NFS and CIFS and never looked back.

Over time the hardware has changed and the system upgraded but the basic approach of a custom built server has remained. When I needed a build engine to churn out hundreds of kernels a day for the ARM Linux autobuilder the system was expanded to cope and mid 2009 the current instantiation was created.

Current full height tower fileserverThe current system is a huge tower case (courtesy of Mark Hymers) containing a Core 2 Quad 2.33GHz (8 threads) with 8Gigabytes of memory and 13 drives across four SATA controllers split into several RAID arrays. Despite buying new drives at higher capacities I have tended to keep the old drives around for extra storage resulting in what you see here.

I recently looked at the power usage of this monster and realised I was paying a lot of money to spin rust which was simply uneconomic. Seriously, why did I have six sub 200Gigabyte drives running when a single 2T to replace them would pay for itself in power saved in under a month! In addition I no longer required the compute power available either, most definitely time for a downsize!

Several friends suggested a HP micro server might be just the thing. After examining and evaluating some other options (Thecus and QNAP NAS) I decided the HP route was most definitely the best value for money.

The HP Proliant micro server is a dual core Athlon II 1.3GHz system with a Gigabyte of memory, space for four SATA hard drives and a single 5 inch bay for an optical drive. All this in a roughly 250mm on a side cube.

My HP proliant microserverI went out and bought the server from ebuyer for 235 with free shipping and 100 cashback. I Immediately sent off the cash back paperwork so I would not forget(what an odd way to get discount) so total cost for the unit was 135. I then used Crucial to select a suitable memory upgrade to take the total to 2 Gigabytes of RAM for 14

The final piece of the solution was the drives for the storage. I decided the best capacity to cost ratio could be had from 2 TB drives and with four bays available would give a raw capacity of 8 TB or more usefully for this discussion 7.8 TiB

I did an experiment with 3x1 TB 7200 RPM drives from the existing server and determined that The overall system would not really benefit enough to justify the 50% price premium of 7200 RPM drives over 5400 RPM devices. I ended up getting four Samsung Spinpoint F4EG 2 TB drives for 230.

I also bought a black LG DVD-RW drive for 16 I would have also required a SATA data cable and a molex to SATA power cable if I had not already got them.

My HP microserver with the front door openPutting the components together was really simple. The internal layout and design of the enclosure mean it is easy to work with and has the feel of build quality I usually associate with HP and IBM server kit not something this small and inexpensive.

The provided documentation is good but unnecessary as most operations are obvious. They even provide the bolts to attach all the drives along with a wrench in the lockable front door, how thoughtful is that!

I then installed the system with Debian squeeze from the optical drive. Principally because I happened to have a network installer CD to hand although the BIOS does have network boot capability.

I used the installer to put the whole initial system together and did not have to resort to the command line even once, very impressed with how far D-I has come.

After asking several people for advice the general consensus was that I should create two partitions on each drive one for a RAID 1 /boot and one for a RAID 5 LVM area.

I did have to perform the entire install a second time because there is a gotcha with GUID Partition Table, RAID 1 boot drives and GRUB. You must have a small "BIOS" partition on the front of the drive or GRUB cannot install in the MBR and your system will not boot!

The partition layout I ended up with looks like:
Model: ATA SAMSUNG HD204UI (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 17.4kB 32.0MB 32.0MB bios_grub
3 32.0MB 1000MB 968MB raid
2 1000MB 2000GB 1999GB raid

The small Gigabyte partition was configured as a RAID 1 across all four drives and formatted with ext2 and mount point of /boot. The large space was configured as RAID 5 across all four drives with LVM on top. Logical volumes were allocated formatted ext3 (on advice from seevral people about ext4 instability they had observed) for 50 GiB root, 4 GiB swap and 1 TiB home space.

The normal Debian install proceeded and after the post install reboot I was presented with a login prompt. Absolutely no surprises at all no additional drivers required and a correctly running system.

Over the next few days I did the usual sysadmin stuff, rsynced data from the old fileserver including creating logical volumes for the various arrays from the old server none of which presented much of a problem. The 5.5TiB Raid 5 did however take a day or so to sync!

I used the microservers eSATA port to attach external drives I use for backup purposes which has also not been an issue so far.

I am currently running both the new and old systems for a few days and rsyncing data to the microserver until I am sure of it. Actually I will make the switch this weekend and shut the old system down and leave it till next weekend before I scrub the old drives.

Before I made it live I decided to run some benchmarks and gather some data just for interest.
Bonnie (Version 1.96) was run in the root logical volume (I repeated the tests in other volumes, there is sub 1% variation) the test used a 4GiB size and 16 files

Sequential OutputSequential InputRandom SeeksSequential CreateRandom Create
Per ChrBlockRewritePer ChrBlockCreateReadDeleteCreateReadDelete
/sec378K41M37M2216K330M412.811697+++++1833014246+++++14371
%CPU9711891301524+++2829+++22
Latency109ms681ms324ms116ms93389 s250ms29021 s814 s842 s362 s51 s61 s

Does not seem to be any notable issues there, the write speeds are a little lower than I might like but that is the cost of RAID 5 and 5400 RPM drives.

The rsync operations used to sync up the live data seem to manage just short of 20MiB/s for the home partition comprising of 250GiB in two and a half million files with the expected mix of file sizes. The video partition managed 33MiB/s on 1TiB of data in nine thousand files.

The bonnie tests were performed accessing the server over NFS with 24GiB size and 16 files.
Sequential OutputSequential InputRandom SeeksSequential CreateRandom Create
Per ChrBlockRewritePer ChrBlockCreateReadDeleteCreateReadDelete
/sec1733K29M19M4608K106M358.3146537142402157640821529
%CPU98249310108109997
Latency10894 s23242ms69159ms49772 s224ms250ms148ms24821 s157ms108ms2074 s719ms

or alternatively as percentages against the previous direct access values

Sequential OutputSequential InputRandom SeeksSequential CreateRandom Create
Per ChrBlockRewritePer ChrBlockCreateReadDeleteCreateReadDelete
/sec4646851213328712+++1311+++10
CPU1011850104347133+++3231+++31
Latency925121324893227795093049186462983440661178688

Not that that tells us much aside from that write is a bit slower over the network, read is gigabit network bandwidth limited and latency of disc over the network is generally poorer than direct.

In summary the total cost was 395 for a complete ready to use system with 5.5TiB of RAID 5 storage which can be NFS served at nearly 900Mbit/s. Overall I am happy with the result, my only real issue is the write performance is a little disappointing but it is good enough for what I need.

Adnan Hodzic: My DebConf11 summary and its after effects

Even though some three months have passed since DebConf11 has successfully ended, I still wanted to give you just a glimpse on how some parts were played through the eyes of a lunatic (read: organizer). Of course, blog post can t come close near of explaining anything but at least it should give some insight and hopefully some pointers to the future DebConf organizers. If you don t feel like reading this much text you can listen to last episode of This Week In Debian podcast and after it just head to Beginning of the end part. DebConf11 First of all I d like to apologize if I offended or hurt anyone in this whole process as it wasn t my intention and during DebConf organization this is nothing irregular, for you to get hurt or you hurting somebody else. One thing that got stuck with me this whole time is when Mart n Ferrari approached me after I did bid proposal of Bosnia/Herzegovina for DebConf11 back on DebConf9 and told me you have no idea what you just got yourself into I played cool and said something along the line of coure I do and boy did I lie. During DebConf organization you re bound to make some of your decisions instantaneously in which you ll lose something, the most you can do is assess what s the thing you can cut the cord on, even though if that thing or a person for that matter might have meant something to you. You ll lose things, relationships, friendships, contacts at the times sense of humor and maybe even common logic. Some of it comes back to you, some the very next day and for some it may take longer. Some of it was lost irreversibly, but even in that case it wasn t lost and could even be traced to some other cause. One thing that I m thankful for is that I didn t lose, not lose but even get distanced from those close to me, and that I didn t lose guy who was with me in this from very first day, my right hand, my partner in crime Velimir ( aroundthfur ). It s not even losing I m talking about here, because even by losing you re gaining something and this is part of your everyday life, it s just that your life frame kinda gets stuck on fast forward button. So once the ride s over you may feel bit dizzy asking yourself what just happened. During this time I was also contacted by Google recruiter regrading software engineer internship but to make a long story short I didn t make it due to my heavy involvement and my lack of time (read: fail). But I don t find this as losing either, first time I was contacted by them, next time they ll be contacted by me maybe even for different position this time (project manager? Big Smile. Even this DebConf wasn t our first bid we lost DebConf7 bid to Edinburgh, but it is perseverance that got us here. The moral I m trying to tell here is if you have a goal never back down from it. What I found to be our biggest problem and what I personally found as most painstaking in all of this was having and reporting to so many different sides and different groups while trying to keep them in same communication channel. Even my right hand (Velimir) was apart from me (Vienna), then place where two of us are coming from is not the place where DebConf is getting organized in. As in the end we ended up having all the parties separated and pulling to their own side, from Government of Republika Srpska to NGO to Debian to local team and to those who remained as undeclared . Option of cutting the cord in this case was missing as these were the components that DebCon11 was made of. Of course this meant you ll be missing workforce and there were days you d wake up driving >= 200 km/h (literally) with one hand on the wheel, other on the phone and your other right hand (Velimir) holding your RedBull. But you wouldn t do this just because you had to and there was no one else to do it, you d do it because you re responsible for the sake of the project and won t be just sitting there because by some unwritten hierarchy rules you re supposed to sit and supervise things. I d advise you do the same thing when you see there s no one else to do something, even if it means making coffee for one of the teams. I m not trying to scare you away from organizing your own DebConf, on the contrary after all I ve been through in past two years if someone asked me would I do it if I knew it was like this, without a doubt my answer would still be fsck yea . All the previous DebConf organizers told me about all of this, but why didn t I give up? For the same reason you won t give up and for the same reason why we believe in the same goal. If anything this is one of the stories you ll want to tell your kids about so When I was your age I used to sleep naked in the snow would turn to When I was your age I organized DebConf11 in Maybe this blog post wasn t even necessary and some things weren t supposed to be said, I guess sometimes I just can t shut up Smile For those who were on DebConf11 I hope you enjoyed and had a great time, otherwise I m saying all of this in vein. As organizer all you see is problems and you miss most of the confrence, but by what I know there are 11 candidate bids for DebConf13 and for me hearing that tells me the results of our efforts. DebConf11 after effects I could write a whole novel about this, but to keep it as short as possible, for last two years as a side project I was working on an idea of Government or some of its institutions migrating to Linux. At first I was somewhat loud about it, then after Microsoft heard about it and after they tried stopping the idea by trying to scare me by trying to interfere with my private life; as that didn t work its lobbyist came even near of obstructing the whole conference within the Government. For the sake of the conference, I convinced the Government that by supporting DebConf it doesn t mean they need to move to Linux and publicly stopped talking about it. I also convinced them that our only goal was to have successful conference and promote alternative options and open ideas. I wasn t lying as I saw this as new opportunity of them concluding on their own why they should or shouldn t not move, the better conference was the more chances of success we had. That s why I tried pushing as many representatives from various companies as in this case we would use reverse psychology where basically no one or few know what Linux or Debian for that matter is, but everybody knows who Google is, so if you have participants from i.e: Google or Austrian E-Health care system talking about how they are using your technology is better way to explain what s it all about really. Eventually Microsoft even had their first ever conference in Bosnia/Herzegovina and you wanna take a wild guess where it was held? Smile In the end we had a great conference, after the conference we were the ones that were approached by some big local companies interested in future co-operation and in the end a meeting with Mark Shuttleworth and President along with the core of Government was scheduled. Topic? Migration to Linux. For me personally this meeting went better then I could possibly even picture it, many topics were discussed and basically it was up to us/me to make a draft of the project plan and submit request for proposal to the Government. There was still some lobbying but it seemed as it all disappeared, runway was clear and open for the lift off. Necessary contacts with potential future partners were made and before any meetings were scheduled I announced I ll be taking a short break. I missed this years vacation, for matter of a fact I missed going to a vacation for last two years. I decided going to Dubrovnik for few days, just to recollect my thoughts, recompile my kernel and switch to lower gear as I was stuck in 6th gear for way too long. Beginning of the end Being on this vacation , even if it was only for few days this gave me an opportunity to look at it all from a different prospective, different angle and that s when it came to me. I realized that even before I was finished with one project without even realizing I was already involved with another one, one that would take much more time and one that might leave me wondering ~5 years later with question what if? After all if get involved with something I m not going to back out, it s either failure is not an option or I won t start doing it at all. Even if it brought me everything I wanted, how would you know if that s what you wanted if you haven t thought about it first? Having DebConf for the first time in this region already showed some (positive) results, but for seeing the actual results it may take a bit longer. We sowed the seeds now the best thing we can do is leave them as it is and see if anything grows out. Of course I could lean over them and protect them from any natural or any other type of hazards, this would also mean when I m not around it wouldn t know how to protect itself on its own. Besides all, interfering with natures way of doing things is rarely a good option. I consider myself to still be relatively young and threatening my well being over safe keeping some seeds and watching over if they grow into something or not doesn t really seem that attractive to me. Maybe one day I ll decide to leave the land, maybe I won t even get to see if anything grows out after all. Maybe some day someone will find this fruit, recognize its worth and claim it as it s own, maybe over time everybody forgets everything and even that this land once was a fertile ground. Maybe, maybe not. I m still a student, I was supposed to graduate last year but I took less classes last year so I wouldn t snap under all the pressure. That s why this year I have less classes and am supposed to graduate in May, that s less then a year until then and I just want to devote some time to myself, figure out what I want with me life, learn the things I want to learn, do the things I want to do. I want to turn back to working on Eclipse, I have some good plans for Eclipse and Android developers, lately I ve been slacking off way too much but I think I have a good excuse Smile In the end, all I know is that I still want to change the world, whether it s doing that inside of Google or in my own company, or some third place; or I ll just be laughing at this statement only time will show. I m hoping that now with all of this out of me I can finally finish the DebConf11 chapter of my life.

21 October 2011

Tollef Fog Heen: Today's rant about RPM

Before I start, I'll admit that I'm not a real RPM packager. Maype I'm approaching this from completely the wrong direction, what do I know? I'm in the process of packaging Varnish 3.0.2 which includes mangling the spec file. The top of the spec file reads:
%define v_rc
%define vd_rc % ?v_rc:-% ?v_rc 
Apparently, this is not legal, since we're trying to define v_rc as a macro with no body. It's however not possible to directly define it as an empty string which can later be tested on, you have to do something like:
%define v_rc % nil 
%define vd_rc % ?v_rc:-% ?v_rc 
Now, this doesn't work correctly either. % ?macro tests if macro is defined, not whether it's an empty string so instead of two lines, we have to write:
%define v_rc % nil 
%if 0% ?v_rc  != 0
%define vd_rc % ?v_rc:-% ?v_rc 
%endif
The 0 ?v_rc != 0 workaround is there so that we don't accidentially end up with == 0 which would be a syntax error. I think having four lines like that is pretty ugly, so I looked for a workaround and figured that, ok, I'll just rewrite every use of % vd_rc to % ?v_rc:-% ?v_rc . There are only a couple, so the damage is limited. Also, I'd then just comment out the v_rc definition, since that makes it clear what you should uncomment to have a release candidate version. In my naivety, I tried:
# %define v_rc ""
# is used as a comment character in spec files, but apparently not for defines. The define was still processed and the build process stopped pretty quickly. Luckily, doing # % define "" seems to work fine and is not processed. I have no idea how people put up with this or if I'm doing something very wrong. Feel free to point me at a better way of doing this, of course.

Russell Coker: Occupy Main Street

The Occupy Wall St blog has an informative summary of attempts to reclaim the American political process which has been pwned badly by financiers in recent times [1]. The basic concept is that people who represent the 99% of the population who aren t super rich have protests in Wall St and now other business areas. Care2 has an interesting article about US marines opposing the brutal actions of police against Occupy Wall St protesters [2], apparently they treated Iraqis better than US police are treating Americans. The movement has spread to other locations, the OccupyTogether.org site has information on related events all around the world [3]. We have an ongoing event in Melbourne, Australia. It s been going for a week and yesterday Robert Doyle (the Mayor of Melbourne) ordered police to disperse the protest, so riot police and mounted police forced the protesters out of the city square [4]. According to the news reports there were only 100 people there at the time, here is a Google Maps link to the location, as you can see 100 people would not take up much of that space, not even with banners etc. The smart move would have been for the government to ignore it all until the protesters got bored. Now of course we will have more and bigger protests. The use of riot police will probably be considered as a good thing by some of the more aggressive protesters, but anyone who doesn t want to make the government (and the corporations that control it) look bad would consider it a gross error. Robert Doyle needs to be replaced, the liberal reason for replacing him is that we just don t want unnecessary force used against peaceful protesters. The conservative reason for replacing him is that he s grossly incompetent, he transformed a small protest that wasn t getting much media attention and appeared to be losing interest into a large protest with a lot of media attention. It will be interesting to see what happens next.

Russell Coker: Dedicated vs Virtual Servers

A common question about hosting is whether to use a dedicated server or a virtual server. Dedicated Servers If you use a dedicated server then you will face the risk of problems which interrupt the boot process. It seems that all the affordable dedicated server offerings lack any good remote management, so when the server doesn t boot you either have to raise a trouble ticket with the company running the Data-Center (DC) or use some sort of hack. Hetzner is a dedicated server company that I have had good experiences with [1], when a server in their DC fails to boot you can use their web based interface (at no extra charge or delay) to boot a Linux recovery environment which can then be used to fix whatever the problem may be. They also charge extra for hands-on support which could be used if the Linux recovery environment revealed no errors but the system just didn t work. This isn t nearly as good as using something like IPMI which permits remote console access to see error messages and more direct control of rebooting. The up-side of a dedicated server is performance. Some people think that avoiding virtualisation improves performance, but in practice most virtual servers use virtualisation technologies that have little overhead. A bigger performance issue than the virtualisation overhead is the fact that most companies running DCs have a range of hardware in their DC and your system (whether a virtual server or a dedicated server) will be on a random system from their DC. I have observed hosting companies to give different speed CPUs and for dedicated servers different amounts of RAM for the same price. I expect that the disk IO performance also varies a lot but I have no evidence. As long as the hosting company provides everything that they offered before you sign the contract you can t complain. It s worth noting that CPU performance is either poorly specified or absent in most offers and disk IO performance is almost never specified. One advantage of dedicated servers in this regard is that you get to know the details of the hardware and can therefore refuse certain low spec hardware. The real performance benefit of a dedicated server is that disk IO performance won t be hurt by other users of the same system. Disk IO is the real issue as CPU and RAM are easy to share fairly but disk performance is difficult to share and is also a significant bottleneck on many servers. Dedicated servers also have a higher minimum price due to the fact that a real server is being used which involves hardware purchase and rack space. Hetzner s offers which start at E29 per month are about as cheap as it s possible to get. But it appears that the E29 offer is for an old server new hardware starts at E49 per month which is still quite cheap. But no dedicated server compares to the virtual servers which can be rented for prices less than $10 per month. Virtual Servers A virtual server will typically have an effective management interface. You should expect to get web based access to the system console as well as ssh console access. If console access is not sufficient to recover the system then there is an option to boot from a recovery device. This allows you to avoid many situations that could potentially result in down-time and when things go wrong it allows you to recover faster. Linode is an example of a company that provides virtual servers and provides a great management interface [2]. It would take a lot of work with performance monitoring and graphing tools to give the performance overview that comes for free with the Linode interface. Disk IO performance can suck badly on virtual servers and it can happen suddenly and semi-randomly. If someone else who is using a virtual server on the same hardware is the target of a DoS attack then your performance can disappear. Performance for CPU is generally fairly reliable though. So a CPU bound server would be a better fit on the typical virtual server options than a disk IO bound server. Virtual servers are a lot cheaper at the low end so if you don t need the hardware capabilities of a minimal dedicated server (with 1G of RAM for Hetzner and a minimum of 8G of RAM for some other providers) then you can save a lot of money by getting a virtual server. Finally the options for running a virtual machine under a virtual machine aren t good, AFAIK the only options that would work on a commercial VPS offering are QEMU (an x86 CPU instruction emulator), Hercules (a S/370 S/390, and Z series IBM mainframe emulator), and similar CPU emulators. Please let me know if there are any other good options for running a virtual machine on a VPS. Now while these emulators are apparently good for debugging OS development they aren t something that are generally useful for running a virtual machine. I knew someone who ran his important servers under Hercules so that x86 exploits couldn t be used for attacking them, but apart from that CPU emulation isn t generally useful for servers. Summary If you want to have entire control of the hardware or if you want to run your own virtual machines that suit your needs (EG one with lots of RAM and another with lots of disk space) then a dedicated server is required. If you want to have minimal expense or the greatest ease of sysadmin use then a virtual server is a better option. But the cheapest option for virtual hosting is to rent a server from Hetzner, run Xen on it, and then rent out DomUs to other people. Apart from the inevitable pain that you experience if anything goes wrong with the Dom0 this is a great option. As an aside, if anyone knows of a reliable company that offers some benefits over Hetzner then please let me know. What I would Like to See There is no technical reason why a company like Linode couldn t make an offer which was a single DomU on a server taking up all available RAM, CPU, and disk space. Such an offer would be really compelling if it wasn t excessively expensive. That would give Linode ease of management and also a guarantee that no-one else could disrupt your system by doing a lot of disk IO. This would be really easy for Linode (or any virtual server provider) to implement. There is also no technical reason why a company like Linode couldn t allow their customers to rent all the capacity of a physical system and then subdivide it among DomUs as they wish. I have a few clients who would be better suited by Linode DomUs that are configured for their needs rather than stock Linode offerings (which never seem to offer the exact amounts of RAM, disk, or CPU that are required). Also if I had a Linode physical server that only had DomUs for my clients then I could make sure that none of them had excessive disk IO that affected the others. This would require many extra features in the Linode management web pages, so it seems unlikely that they will do it. Please let me know if there is someone doing this, it s obvious enough that someone must be doing it. Update: A Rimuhosting employee pointed out that they offer virtual servers on dedicated hardware which meets this criteria [3]. Rimuhosting allows you to use their VPS management system for running all the resources in a single server (so no-one else can slow your VM down) and also allow custom partitioning of a server into as many VMs as you desire.

20 October 2011

Sylvain Beucler: Bounding box

Quickie OpenGL wikibook tutorial to show how to display a bounding box around your object :)
http://en.wikibooks.org/wiki/OpenGL_Programming/Bounding_box

Kai Wasserb ch: WEB.DE offers Phish

[UPDATE 2011-10-26] The story has developed.[/UPDATE] Recently I was talking with somebody about the WEB.DE toolbar, and today I finally got around to look something up I was told then. And while I searched for the information on the download page for the WEB.DE toolbar, I stumbled over the download URL for the installer:
http://wa.ui-portal.de/webde/webde/s?produkte.browserdownload.link.download.ff7&bd_mc=undef_undef&ns_type=pdf&ns_url=http://dl.web.de/browser/firefox/WEB.DE_MFF7_Setup.exe
There I was magically attracted by the ns_url parameter, because it smelled like something where you could just pass any URL along. And indeed you can: http://wa.ui-portal.de/webde/webde/s?produkte.browserdownload.link.download.ff7&bd_mc=undef_undef&ns_type=pdf&ns_url=http://bundestrojaner.net/inhalt-bundestrojaner-gratis-download-3.html. That should redirect you to http://bundestrojaner.net/inhalt-bundestrojaner-gratis-download-3.html instead of http://dl.web.de/browser/firefox/WEB.DE_MFF7_Setup.exe. This is a serious flaw, as it might allow an attacker to request a username and password combination from an unsuspecting user. With a harvest page designed to look like another WEB.DE page you should have a high return rate. This is, by the way, a flaw on the OWASP Top 10 (2010 edition). I've informed somebody working for United Internet and expect the bug to be resolved pretty soon, hopefully it won't work anylonger when you read this.

Pau Garcia i Quiles: The Secure Boot controversy

Red Hat s Matthew Garrett let the cat out of the bag about a month ago: when UEFI Secure Boot is adopted by mainboard manufacturers to satisfy Microsoft Windows 8 requirements, it may very well be the case that Linux and others (BSD, Haiku, Minix, OS/2, etc) will no longer boot. Matthew has written about it extensively and seems to know very well what the issues are (part I, part II), the details about signing binaries and why Linux does not support Secure Boot yet. The Free Software Foundation has also released a statement and started a campaign, which is, as usually, anti-Microsoft instead of pro-solutions. Now let me express my opinion on this matter: this is not Microsoft s fault. Facts Let s see what are the facts in this controversy: Problem The problem stems from the fact that most Linux vendors do not have the power to get their certificates in UEFI BIOS. Red Hat and Suse will for sure get their certificates bundled in server UEFI BIOS. Debian and Ubuntu? Maybe. NetBSD, OpenIndiana, Slackware, etc? No way. This is, in my humble opinion, a serious defect in the standard. A huge omission. Apparently while developing the Secure Boot specification everybody was busy talking about signed binaries, yet nobody thought for a second how the certificates will get into the UEFI BIOS. What should have been done The UEFI secure boot standard should have defined an organization (a Secure Boot Certification Authority ) that would issue and/or receive certificates from organizations/companies (Red Hat, Oracle, Ubuntu, Microsoft, Apple, etc) that want their binaries signed. This SBCA would also be in charge of verifying the background of those organizations. There is actually no need for a new organization: just use an existing one, such as Verisign, that carries on with this task for Microsoft for kernel-level binaries (AuthentiCode). Given that there is no Secure Boot Certification Authority, Microsoft asked BIOS (UEFI) developers and manufacturers to include their certificates, which looks 100% logical to me. The fact that Linux distributions do not have such power is unfortunate, but it is not Microsoft s fault at all. What can we do? Given its strong ties with Intel, AMD and others, maybe the Linux Foundation could start a task force and a Temporary Secure Boot Certification Authority to deal with UEFI BIOS manufacturers and developers. This task force and TSBCA would act as a proxy for minorities such as Linux, BSD, etc distributions. I am convinced this is our best chance to get something done in a reasonable amount of time. Complaining will not get us anything. Screaming at Microsoft will not get us anything. We need to propose solutions. Wait! Non-Microsoft certificates? Why? In addition to the missing Secure Boot Certification Authority, there is a second problem apparently nobody is talking about: what is the advantage mainboard manufacturers get from including non-Microsoft certificates? For instance: why would Gigabyte (or any other mainboard manufacturer) include the certificate for, say, Haiku? The benefit for Gigabyte would be negligible and if someone with ill-intentions gets Haiku s certificate, that piece of malware will be installable on all Gigabyte s mainboards.This would lead to manufacturer-targetted malware, which would be fatal to Gigabyte: oh, want to be immune to the-grandchild-of-Stuxnet? Buy (a computer with) an MSI mainboard, which does not include Haiku s certificate Given that 99% of desktops and laptops only run Windows, the result of this (yet unresolved) problem would be that manufacturers will only install Microsoft certificates, therefore they would be immune to malware signed with a Slackware certificate in the wild. If we are lucky, mainboard manufacturers will give us an utility to install more certificates under your own risk. The solution to the first problem looks easy to me. The solution to the second looks much more worrying to me.

Andrew Pollock: [life] Sponsoring a child in Africa

I've been wanting to sponsor a child in Africa for many, many years. I remember seeing a photo of a sponsored child on a couple of sets of friends' mantelpieces in Canberra, and saying to myself how I should really do that too. Of course, for no really good reason, fast-forwarding to 6+ years later, I still hadn't gotten around to it. Until last month. I was walking out of the Apple Store in Palo Alto. I was accosted by one of those people on the street, with the clipboard and all that. I usually manage to brush these sorts of people off. I really hate solicited donations. I'd rather consciously make a donation to a charity, than do it because someone on the street was paid to ask me to, or knocked on my door, or whatever. But I was distracted, or in a good mood, or something, and so I missed the initial opportunity to deflect her. I'm really glad I did, because as I said, this is something that I've wanted to do for years, so I barely hesitated, and signed up on the spot. I received the welcome kit in the mail a couple of days ago. I've been allocated a 6 year old boy named Armando, from Mozambique. I'm looking forward to corresponding with him. Our friends, and former neighbours, the Harvetts have moved back to South Africa, and I hope one day to be able to visit them. Since we'll be in the vicinity, I hope one day to be able to visit Armando as well. I'm pleased to see that Save the Children gets a good rap in Charity Navigator. I was worried that I'd get home and find out I'd signed up with one of those charities that blows half their money on overhead, but that's not the case.

19 October 2011

Yves-Alexis Perez: Debian grsec kernels

I received recently a mail about my attempt to provide Grsecurity kernels in Debian. The sender found the bug by accident, and asked me why I didn't do some more publicity here. So here we are. I won't go into details on what grsecurity is, it's fairly complex. But it's basically a hardening patch for the Linux kernel, with three main components: A lot of this touches low level stuff in the kernel, especially memory management. Ideally this patch would be pushed upstream, but Brad Spengler (grsecurity main developper) already said he wasn't interested in upstreaming it and upstream already said the patch was too huge and invasive to include it like that (especially since the original authors aren't interested in maintaining it upstream). There's an ongoing effort to split the patch and merge things little by little, but in the meanwhile having a mid-term solution would be nice. I know Debian users rebuilding grsecurity-patched kernels themselves, and I know some of them would appreciate having them included in the Debian kernel. Fortunately, the linux-2.6 source package has a nice feature which is called featureset. Basically it's a way to build some (binary) packages using a different set of patches and a different config. For example this was used to provide xen/openvz/vserver patchsets, and is now used to provide rt kernels. So I though it'd be nice to provide a grsec featureset, and starting doing the work. I have a working setup for producing those kernels, so I've opened a wishlist bug against the kernel (#605090) to have this merged. Those packages follow the sid kernel. There's an ongoing work for Squeeze, but it's a bit harder there because both the grsecurity patchset and the Debian kernel ship a whole lot of backports to the Linux kernel, meaning the grsecurity patch doesn't apply directly to the Debian source package. Basically I need to remove some of the hunks (since they are already applied to the source) and port some others (since there are some backported code not present in the vanilla 2.6.32, for example the drm code). Until the patches are merged and the bug is closed, I host some of the built packages at:

deb http://molly.corsac.net/~corsac/debian/kernel-grsec/packages/ sid/ The repository is signed by my key which you can add to your apt setup using apt-key add. If you want to rebuild the packages yourself, here's the method:

mkdir kernel-grsec
cd kernel-grsec
svn checkout svn://svn.debian.org/svn/kernel/dists/sid/linux-2.6
git clone git://anonscm.debian.org/users/corsac/grsec-patches.git
wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.0.tar.bz2
wget http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.0.tar.bz2.sign
gpg --verify linux-3.0.tar.bz2
cd linux-2.6
apt-get build-dep linux-2.6
export QUILT_PATCHES=../grsec-patches
quilt push -a
python debian/bin/genorig.py ../linux-3.0.tar.bz2
debian/rules orig
fakeroot debian/rules source
fakeroot make -f debian/rules.gen binary-arch_amd64_grsec_amd64 You could also do dpkg-buildpackage, pdebuild or whatever. Kernel handbook is a nice reading too if you want more information on how to rebuild Debian kernels. The quilt push -a may fail if you checkout an svn version more recent than mine. I try to keep patches up to date but I usually have some delay. Note that installing the kernel will require installing linux-grsec-base package. Binary is not yet available on my mirror but you can easily build it. Source can be found on git.debian.org. If you're interested by this, don't hesitate to mail me or the bug.

Next.

Previous.