Search Results: "ema"

25 June 2008

David Moreno Garza: YAPC::NA 2008

So I attended the conference last week in Chicago, IL. I was just been procrastinating to write about this.

As a (almost) complete outsider of the Perl community, I have countered thoughts about the whole event:

After all, it was quite a nice and new experience. I'm not totally sure I'll attend YAPC::NA 2009 in Pittsburgh, PA, we'll see.

Biella Coleman: The Cultural Significance of Free Software

When it comes to research and writing, anthropologists are notorious for being slow as molasses. It takes, on average, 9 years to finish grad school and then another 3-6 years before you write and publish your first book. Such is the sometimes frustrating pace of academia. But sometimes it is worth the wait as this new book, Two Bits, the Cultural Significance of Free Software by the anthropologist of science and technology Chris Kelty, makes clear. I am pretty intimate with the details of the book as ck was one of my dissertation committee members. I have read earlier versions and was fortunate enough to teach the final version to a small group of graduate students last spring. While there have been previous books on the subject of open source or peer-to-peer production more generally, this is the first book to delve deep, analytically and historically, into the cultural significance of free software. And the best part is that ck managed (I am not sure how but he did) to convince Duke University Press to release the whole book under a CC license. So download it to your heart’s delight and if you find it useful, do order a treeware copy (as you can’t beat that for easy and comfortable reading). For the developers out there, I would suggest starting with the introduction, which provides an initial overview of some of the main themes he develops and then I would dig into part two where the historical chapters reside and then loop back to the theoretical chapters in part one or continue to part three. Trust me, these will fascinate and surprise even those geeks with a relatively deep historical sense of this world. In fact, my very favorite academic piece on free software exists in these pages: and is the chapter on the history of the actual conflict between James Gosling and Richard Stallman that led to the creation of the GNU General Public License. Here is a small excerpt to whet your appetite:
In brief the controversy was this: in 1983 James Gosling decided to sell his version of EMACS a version written in C for UNIX called GOSMACS to a commercial software vendor called Unipress. GOSMACS, the second most famous implementation of EMACS (after Stallman s itself ), was written when Gosling was a graduate student at Carnegie Mellon University. For years, Gosling had distributed GOSMACS by himself and had run a mailing list on Usenet, on which he answered queries and discussed extensions. Gosling had explicitly asked people not to redistribute the program, but to come back to him (or send interested parties to him directly) for new versions, making GOSMACS more of a benevolent dictatorship than a commune. Gosling maintained his authority, but graciously accepted revisions and bug-fixes and extensions from users, incorporating them into new releases. Stallman s system, by contrast, allowed users to distribute their extensions themselves, as well as have them included in the official EMACS. By 1983, Gosling had decided he was unable to effectively maintain and support GOSMACS a task he considered the proper role of a corporation.
[ . . .]
Indeed, the general irony of this complicated situation was certainly not as evident as it might have been given the emotional tone of the debates: Stallman was using code from Gosling based on permission Gosling had given to Labalme, but Labalme had written code for Gosling which Gosling had commercialized without telling Labalme conceivably, but not likely, the same code. Furthermore, all of them were creating software that had been originally conceived in large part by Stallman (but based on ideas and work on TECO, an editor written twenty years before EMACS), who was now busy rewriting the very software Gosling had rewritten for UNIX. The once proud hacker ethic that Labalme mentions would thus amount not so much to an explicit belief in sharing so much as a fast-and-loose practice of making contributions and fixes without documenting them, giving oral permission to use and reuse, and losing records that may or may not have existed hardly a noble enterprise.
[. . .]
The story is a lively one, punctuated with some great e-mail “moments,” and gives you a play-by-play sense of how it is that free software become so entangled with the law and how the unwritten norms that guided sharing were violated to then become transformed, explicit, and written. The other chapter that will also fascinate those who want a deeper sense of this history is on UNIX and its role in creating the technical and collaborative conditions necessary to secure the explosion of free software years later. The story of the norms of sharing source code is, not by accident, writes Kelty, also the history of the UNIX operating system. This connection has been made by others but not shown in any historical detail as it is done in this book. For those interested in more anthropological issues, the first few chapters is where this type of more conceptual theorizing is at. Chris Kelty introduces the master trope of his book: the recursive public which is an idea that feeds into a longer academic debate, inaugurated by Jurgen Habermas, on the role of publicity, dialogue, and circulation of ideas and texts in cementing a democratic sphere. Many others, such as Charles Taylor, Michael Warner, and Nancy Fraser have since refigured and rehashed the meaning publics and public debate but there have been few ethnographies on the topic. It is here where Chris Kelty intervenes most forcefully to conceptualize publics in explicitly ethnographic terms and in an arena technology and the Internet which is often left undertheorized by cultural theorists. There is a lot more in the book (the third section examines some of the various political modulations of free software, such as Connexions and Creative Commons) but instead of saying any more, I will just leave you with his definition of a recursive public and again urge you to check out the book for yourself!
A recursive public is a public that is vitally concerned with the material and practical maintenance and modification of the technical, legal, practical, and conceptual means of its own existence as a public; it is a collective independent of other forms of constituted power and is capable of speaking to existing forms of power through the production of actually existing alternatives.

Clint Adams: Relaxation

I was misled. That much is probably true. I certainly had no idea what I was getting myself into. Enough of that though; there's narrative fading away. The part I should take responsibility for is the decision I made to stray off the map. Perhaps it was not so much a decision as a compulsion. I looked at the map. I saw the suspicious unmarked area at the edge, and concluded immediately that there was something good there, something the people wanted to keep hidden from me. I was correct, but I presumed far too much about the terrain. I thought I would make a quick exploration of the secret place, then backtrack to the more mapped area and get back with the program. Enough of that though; I've nearly forgotten the story. First I should point out that, had the map been to scale, it should have taken me an leisurely fifteen minutes to get to my first destination. It should have, assuming the land would be as flat as the map that represented it. Instead, it turned out to be something one could cover in fifteen minutes if one had, say, a jetpack and spring boots. I'm probably wrong; I'd probably injure myself in the process and take longer. Anyway, the average incline was close to 45 , and luckily was mostly uphill. Luckily. When I got to a resting place, I was able to conclude that my supply of fluids was almost certainly insufficient for me to continue. On the other hand, there was probably no way I would return if I went back for more drink. I told myself that there would likely be a beverage-selling shack out in the middle of nowhere around the next bend or so. Having solved this probability problem, I pressed on into the blazing sun. I came to a path leading down to the sea. It was unclear what was down there, but it was clear that it seemed to be a million miles down, and that that path might be the only way back. Given my current state of exhaustion and hydration, I decided to pass on this opportunity. After continuing for a while, the features at the bottom of that path became visible. A very pretty cove was down there, and almost certainly no shack o' drinks. I then came upon a fork. A wide path led downward: to what, I could not see. To its left, a narrow and rough path led straight. Knowing that a descent probably meant climbing four to eight times the altitude back somehow, and assuming the less pleasant-looking path led somewhere better, I went left. After a while, the land began to slope down, ever so slightly. Little clusters of bottles and newspapers were left on the rocks here and there. Then came a sofa under a tent. I began to wonder what hippies were traveling this path. The slope grew greater, and I passed a bench in a tent. Then I came to a giant staircase, almost straight down. It was covered in weeds and insects. Some were vegan, some wanted my flesh. As I got toward the bottom I thought about how glad I was that I had come that way from the top and not the other way around. Oh, I was very glad. At the bottom was something resembling a plastic bottle graveyard and barrels among barrels filled with water. Rubber hoses seemed to run randomly between certain barrels. A kitchen stood at the edge of some barrels. I suddenly got the feeling that I was not supposed to be here. After discovering that a couple of escape routes were actually dead ends, I proceeded to descend a small staircase to a terraced garden. Hoses lay about the rows, presumably for watering the plants. A cot under a tent lay empty, and I hurried past hoping I would not encounter the owner. Then I was at the ocean. The coast was lined with huge boulders, and a wooden bridge led me to the first one, and another wooden bridge took me to the second. Then the bridges stopped. In the distance, I could see that pretty little cove from before. I took stock. I could either return whence I came, up that horrifying staircase, or I could try to traverse the rocky coast over to the cove, then maybe do a faceplant on the beach and let the tide drown me rather than climbing back up. It seemed like a no-brainer, so I began making my way toward the cove. It got harder. Several times I became very afraid, either of falling and smashing my head against a rock or falling into the water below, or some sequential combination thereof. In the end I turned out to be more afraid of going back, at least until I got to what I judged the point of no return. I was reasonably close to where I imagined the sandy beach started, but there was a huge and impassable boulder in the way. Also, if I went further there was no way I would be able to climb back up in case I needed to retreat. I was thinking I would need to retreat, because unless there was some magical hidden toehold, I would be forced to jump into the water and swim around. I did not want to jump into the water. I wanted my phone to stay unfried in the event that I would need to use it to get rescued. I also did not want to call anyone to get rescued, especially since I didn't really know how to explain where I was. With a heavy sigh, I gave up, turned, and went back. This time I moved more quickly, though incurring more damage to my hands and feet. No humans were visible at the crazy farm/garden and makeshift reservoir, but plenty of bugs were visible on the staircase. I made the mistake of trying to lean against the railing to catch my breath. When I finally got to the bench in the tent, I understood exactly what it was for. I sat down, took a sip of my remaining water, ate an apple, and kept going up. When I got to the couch, I sat again. This time I sat and tried to figure out a plan. I still clung to the belief that once I got back to somewhere less wild I'd be able to buy a drink. I noted that I only had a little bit of water left, that I was dehydrated, that I was sunburned, that my pulse was about 180, and that there were no dogs or Russian people wandering past me. I ate a pear and drank my last few drops of water. I kept sitting until my pulse was down to around 150, then I figured I needed to keep moving. I was experiencing most of the effects of caffeine, and for free. Oddly, I passed a dog a few steps later. It was just standing by a rock, looking at me. I decided not to converse. When I had nearly reached the fork again, a couple came my way. I thought about begging them for water. I decided not to be rude. I have no idea what happened to them. I'm betting they wisely decided not to go down the steps. Resolving that the only sane option was to go back all the way to where I absolutely knew I could buy something potable, I headed in that direction. Still, I took two risks. Instead of going up the exact way I had come, or taking a well-travelled path down, I took a narrow and flat dirt path. It was actually a bit of a dilemma. I started down the latter two more than once each. Finally I saw that the down path was rather V shaped, and I concluded correctly that the straight path would bypass that angular-half-pipe-like construct. When I finally got to a point I recognized, it was smooth sailing the rest of the way down. I bought water and a snack, since I had had almost nothing to eat that day. At this point it would have been prudent to call it quits and go off and gorge myself on seafood or something, but no, I was now in the mapped area and surely it would all be roses and manna. Finishing one bottle, I bought two more. Five minutes later I bought another. I had now overcompensated, but I was playing it safe. I continued my explorations. Bicyclists kept nearly running into me, and, aside from the obvious, I viewed this as a good sign. Bicyclists, especially those that can't seem to steer straight for some reason, avoid difficult terrain. This is why they take their bikes on subways, trains, and buses. I don't like bike-riding, but if I had my druthers, I would always go downhill. No up, no flat, just down. Like downhill skiing, just down. It is a good thing I don't ride a bike. Once I got to the point where the bicycles thinned out to nonexistence, I was at an intersection. The way I wanted to go was up. The way I didn't want to go, but was a reasonable option, was only very slightly up. Additional factors complicated this decision point. [elision] This passage has been marked friends-only and you will not be able to view it without logging in. Moving hurriedly away from there, I headed upward. After a couple of turns, I ran into Catholics. First there were dead Catholics. Then there were Catholics pre-occupied with dead Catholics. Then there were Catholics waiting to die. Then there were seminarians. Then there were teens with prayer books. When I reached the next pinnacle, there were more teens with prayer books. There were benches, and there were no teens on them, for the teens were standing or sitting on the ground, quietly contemplating the Word. Unfortunately, they had decided that the benches were more suitable for holding all their bags, so there was nowhere for me to sit. I considered interrupting their reverie, but decided it wouldn't be very Christian of me to disturb them, so I just placed an ancient Drasnian curse on them and plodded down the hill toward a small beach. I really didn't need any more sun, but at least there probably wouldn't be anyone praying there. The water was tempting, but I'm always paranoid about my belongings when swimming alone, and if my phone were taken I wouldn't be able to call anybody to come rescue me at the next crisis point, sure to come. So I skirted the coastline and arrived at a much larger beach, one with facilities and services. Opting to save money but complicate things by using the public changing rooms and showers, I made a discovery. [elision] This passage has been marked friends-only and you will not be able to view it without logging in. Nonetheless, I changed into my swimming trunks, stuffed everything else but my towel, sunglasses, and a disgusting sugary drink I never should have bought into my bag, and went barefoot a few buildings down to rent a locker. The locker was ridiculously large for a single person, which is good to know should I ever want to kidnap a family and torture them at this place. The proprietor informed me that they were closing at some specific time, about an hour from then. I had no timekeeping device outside of the locker, so I had to wing it, and erring heavily on the side of caution, I had a short beach diversion. Given that large swathes of my epidermis were turning bright red, it's probably for the best. Handling the shower, locker, and changing in the other direction was slightly more complicated, but I did it and then I headed inland, figuring I should have a substantial meal for once that day. On my way to find dinner is when things got really crazy. [elision] This passage has been marked friends-only and you will not be able to view it without logging in. I wandered over to a bench to eat my newly-acquired snack, and did not exercise enough caution, because the flimsy paper bag ripped and I got sauce all over my pants and a little on my shirt. Unfortunately I had used my napkin in that earlier episode, and I'm not sure how much it would have helped anyway. Thus I continued on, looking like a slobby retard. Then I made an obviously-poor decision and ended up having a subpar dinner. What happened after that is a story for another time.

24 June 2008

William Pitcock: E-mail autoresponders suck. Please stop using them.

I find it particularly annoying, when I write a mail to a list, like say, the NANOG list, and get emails in reply like:
From: EMAIL ELIDED
To: William Pitcock <EMAIL ELIDED>
Subject: NAME ELIDED is out of the office.
Date: Tue, 24 Jun 2008 12:02:35 -0400 (11:02 CDT) I will be out of the office starting 06/23/2008 and will not return until 06/26/2008. If this matter is urgent please contact NAME ELIDED at XXX-XXX-XXXX or XXX-XXX-XXXX
This really sucks, for multiple reasons. Mostly, that: So please, for the good of all of the Internet, stop using autoresponders, because they are hideously lame. Or, at least, put something like AUTO-REPLY in your autoresponder so we can just delete the spam you insist on sending us. Thanks lazyweb.

Russell Coker: Links June 2008

Paul Graham has recently published an essay titled How To Disagree [1]. One form that he didn’t mention is to claim that a disagreement is a matter of opinion. Describing a disagreement about an issue which can be proved as a matter of opinion is a commonly used method of avoiding the need to offer any facts or analysis. Sam Varghese published an article about the Debian OpenSSL issue and quoted me [2]. The Basic AI Drives [3] is an interesting papar about what might motivate an AI and how AIs might modify themselves to better achieve their goals. It also has some insights into addiction and other vulnerabilities in human motivation. It seems that BeOS [4] is not entirely dead. The Haiku OS project aims to develop an open source OS for desktop computing based on BeOS [5]. It’s not nearly usable for end-users yet, but they have vmware snapshots that can be used for development. On my Document Blog I have described how to debug POP problems with the telnet command [6]. Some users might read this and help me fix their email problems faster. I know that most users won’t be able to read this, but the number of people who can use it will surely be a lot greater than the number of people who can read the RFCs… Singularity tales is an amusing collection of short stories [7] about the Technological Singularity [8]. A summary of the banana situation [9]. Briefly describes how “banana republics” work and the fact that a new variety of the Panama disease is spreading through banana producing countries. Given the links between despotic regimes and banana production it’s surprising that no-one is trying to spread the disease faster. Maybe Panama disease could do for South America what the Boll weevil did for the south of the US [10]. Jeff Dean gives an interesting talk about the Google server architecture [11]. One thing I wonder about is whether they have experimented with increasing the chunk size over the years. It seems that the contiguous IO performance of disks has been steadily increasing while the seek performance has stayed much the same, and the dramatic increases in the amount of RAM you can get for any given amount of money over the last few years have been amazing. So it seems that now it’s possible to read larger chunks of data in the same amount of time and more easily store such large chunks in memory.

Share This

Matthew Garrett: Power management design philosphy

One of the traditional problems with power management has been balancing power savings against reduced functionality. The two traditional approaches to this have been to have sane defaults or to let the user tweak them, with a hybrid approach being to have sane defaults and an "advanced" box that lets users set their own parameters.

Both of these options suck.

The problem with sane defaults is that power management is an area where the "sane default" genuinely does differ between users. Usage patterns will determine the appropriate spindown period for a hard drive. AHCI's ALPM will disable hotswap detection, which is fine for home users but a pain if people want to rip out a failed RAID element. ALPM also introduces a small amount of latency (on the order of a seek in the most aggressive mode), which is entirely irrelevant for almost everyone but is a pain if you're trying to offer i/o latency guarantees. PCI-E link power management adds small quantities of latency. Letting the CPU run to full speed whenever it wants to is good for power management but potentially bad for thermal management. Screensaver timeouts vary depending on whether you're watching a movie or not.

The problem with leaving everything tweakable is that you're asking users to make choices about things but not giving them the information they need to make those choices. Whether you get a power saving from hard drive spindown depends on whether the drive is idle for long enough to save the power you'll spend spinning it back up. Get it wrong and you'll be putting your drive under extra load, reducing performance and consuming more power than you were to begin with. If you limit the maximum speed of your CPU to keep it cool, you'll be wasting power and performance even when it's well below the thermal limits. Disabling ALPM because you might want to change a RAID device at some stage means you're consuming power despite generally not needing to use the additional functionality.

But while these options both suck in different ways, they share one fundamental failure. They both imply static configuration. The correct power management policy depends on the way that a machine is being used at a specific instant in time, and no matter how many power management profiles you add there's no way for a user to switch between them fast enough to obtain a decent power/performance ratio.

Adaptive power management adopts a different strategy. Rather than define a static policy (or several static policies), the hardware is monitored to determine the best power management settings. An example of this is the ondemand CPUfreq governor. Rather than attempt to provide a static configuration, the processor frequency is automatically changed to meet demand while also keeping the overall power consumption as low as possible. This patch adds a thermal component to its policy, preventing the machine from overheating. Result? No requirement to provide tunables to userspace.

Something similar is achievable with hard-drive spindown. By simply monitoring how often reads or writes actually hit disk, it's possible to automatically adjust the spindown period to obtain decent power savings while reducing the number of spinups. This approach is imperfect - in an ideal world we'd know to spin the drive up before it's needed, but that simply isn't possible. However, the user is almost certainly unable to make a better guess. It's not realistic for a user to be able to predict whether or not a given page is still in cache or has been evicted. Nothing is lost by providing a dynamic policy, and another unnecessary tunable has vanished.

The difficult part of an adaptive policy comes when roles are switched in ways that don't correspond to changse in hardware status. One of the most common cases is that of screensaver settings when the user is watching a video. It's not practical for the kernel or a system daemon to predict this case, so we have two choices:.One of these choices involves the user having to explicitly state something that's obvious - playing a video automatically implies the desire to adopt a different power management policy, and so asking the user to manually adjust that is ridiculous.

The loss of hotswap notifications when ALPM is enabled? Simply reenable it if SMART starts complaining or the RAID set becomes degraded or a filesystem is unmounted or any of the other things that will have to be done before a drive can be safely removed. Latency an issue? Let userspace tell you what its latency requirements are. When we get down to it, the only settings that we can't infer are things like idle-time-to-suspend, screensaver settings and default brightnesses. Which, curiously, are about the only things that Mac OS exposes[1].

The problem we face is that we don't currently have interfaces to let applications tell us what their requirements are. This is theoretically straightforward, but getting it right is important. Beyond latency and screensaver issues, what constraints do applications want to impose? How can we expose an interface to those constraints? This is the sort of thing we'll be looking at at the Linux Plumbers Conference, so if you have ideas about what they'd need to look like then sign up. Or, even better, submit a paper and be part of the process that gets Linux's power consumption as close to the theoretical minimum as possible.

[1] My Vista system gives me 25 options which can be set for each of AC and battery.

Biella Coleman: Lost in Translation

I bet this rings familiar to many readers of this blog.

23 June 2008

Obey Arthur Liu: Daniel Burrows on cwidget-gtk

Sometimes people propose other solutions to have the Aptitude-gtk project done, like :
I d have thought that the most interesting way of doing aptitude-gtk would be to write a gtk backend for libcwidget. -Karellen
As this might be of interest to more people, I’ll repost Daniel Burrows‘ response :
Although I can see where you re coming from, a GTK+ backend to cwidget would have no significant upsides and significant downsides. The main upside to doing that from a practical point of view is that you would have only one piece of driver code for the GTK+ and curses interfaces. You wouldn t have to rewrite the entire interface from scratch to create a GUI frontend. But this is just an illusory benefit. cwidget is a terminal interface library and aptitude s curses frontend is a terminal-based UI. From a practical point of view, this means that aptitude regularly assumes that, for instance, to clear out a line it s sufficient to output $WIDTH space characters (ASCII 32) with the desired background style. A port of cwidget to GTK+ would have to introduce new, interface-independent abstractions for all operations like this and eliminate all direct use of terminal-based assumptions, at a significant cost in complexity for both cwidget and the aptitude frontend. In order to transition to these new conventions, I believe you would have to rewrite large portions of both cwidget *and* aptitude in order to make the interface abstraction layer work. (remember: the justification for doing this is to avoid having to rewrite aptitude!) Furthermore, while this approach avoids reimplementing aptitude s curses interface (sort of, except where it doesn t see above), it does so at the cost of requiring you to reimplement all of cwidget, for GTK+. I just did a quick line count (with wc -l ): in all of cwidget there are 23,390 lines of code, while aptitude s curses frontend has around 23,359 lines of code (some of which are actually shared with the other frontends but are in the curses directory due to the historical organization of the code tree). Not only is a rewrite of cwidget more involved, but the amount of code you d have to replace is comparable to the amount implicated in a rewrite of aptitude itself! Finally, even if these costs weren t enough to eliminate the idea, it s not at all clear to me that a GTK+ backend for cwidget is even desirable. Although aptitude has some superficial similarities to a GUI program, the process of designing an interface to run in a terminal (where the available input mechanisms are relatively restricted and all output is to a character-cell grid) is rather different from the process of designing a GUI. GUI programs can t rely on some simplifying assumptions that terminal programs can, and at the same time they have a lot more options available to them (just to name a few: graphics, multiple windows, differing text sizes and rotated text). Trying to force everything through an abstraction designed for terminal output would require you to either
  1. eschew those options,
  2. provide abstract GUI components that somehow fell back to terminal rendering in a terminal, or
  3. add ways for the frontend to extend the abstract widgets with custom functionality.
1. is not an option because I (and, I think, Arthur) want to design a good GTK+ package manager, not a good GTK+ imitation of a console package manager. 2. and 3. amount, in my strong opinion, to writing two separate interfaces but intertwining their implementations in a mass of maintenance-resistant and extension-resistant spaghetti code. At the end of the day, I think that having two separate frontends (sharing backend logic where appropriate, of course) is a far cleaner and more maintainable design, and also easier to create.
As I said earlier, I intend to go a little farther than just making a GTK+ “skin” over the curses interface. Think less like gvim and emacs and more like maybe ddd. I should also add that the cwidget library is used by no major programs except Aptitude so few others would benefit from a GTK+ backend. As the cwidget library homepage says:
cwidget was originally an internal user interface library for the aptitude package manager. This means, on the one hand, that cwidget is known to work in the real world for a real application; on the other hand, it means that the feature set of cwidget is coextensive with the requirements of aptitude.

Jeff Bailey: Flushing in the Inbox

Part of the joy of spending 3 months stoned is that my Inbox had managed to fill up quite nicely with all sorts of crap. In a desperate attempt to prune things, I've taken everything from LiveJournal and Facebook and archived them.

The LJ things are often bits where people have wished me well about the surgery and the recovery. But at over 80 conversations (some of which were multiple people), I'll just *never* get to them. I love you all for the well wishes, and I hope that I've managed to connect with all of you after them at some point.

I've also done the same with Facebook emails on the theory that Wall posts are probably stale by now (or already replied to), and the Inbox there will tell me what I haven't replied to (something like 77 posts)

I started the evening at 1350 messages, I'm now down to 497. Hopefully I'll get it down to zero over the course of this week.

Bed time!

22 June 2008

Meike Reichle: Open Source Census Part 2

Finally, here's the looong overdue second post on the Open Source Census. Very shortly after my initial post on my experiences with the Open Source Census I got an e-mail from one of their engineers adressing the points I had made in my post. So here are a few additions to my first post: Tolimar was also contacted by the developer, he explained to him how to exclude individual directories using the Ruby version (not sure if it's also possible in the Java version) and how reports can be submitted anonymously. So, the privacy protection issues can also be taken care of. I was very impressed by the quick and thorough reaction to both Tolimar's and my own post and hope this post will be of some help to the project and get them some more submissions. One last note: Meike is indeed a female name. So no, he won't post a response, but she will ;)

Christian Perrier: Console stuff not a secret, just neglected

Marc, from my experience, there is no 'secret' in the console-tools/kbd migration work. Just low involvment of all parties. I have watched the work on console handling stuff in Debian for a few years now, and my conclusion is just that nearly nobody cares about it now. I do maintain console-data that provides console fonts and keymaps. I took it over slowly from Alastair McKinstry (the console-tools maintainer mentioned in Marc's blog entry) because that involved maintaining localization of keymap names....that are used and visible in the installer. Ideas to switch the installer to console-setup and related tools are floating around. Just not achieved...because of lack of motivated manpower. Progress was made because of Anton Zinoviev and Colin Watson's work, but the project is currently hosed on a quite critical choice: About console-tools being installed by default: this is the consequence of it being Important while kbd is Extra. I think it should be up to kbd maintainers to push the change up. But this brings us back to the initial remark: indeed, now nobody cares about console handling, particularly when it comes at handling non-English environments. Those who still use the console in Linux environments apparently all do it with US keyboards and an English locale. For sure, kbd is more maintained than console-tools so, at least, we should switch to it. Funnily, our installer (even the graphical version) *still* relies on console keymaps during the installation process and, therefore, we rely on mostly unmaintained stuff here (people who think that I maintain console-data are plain wrong: I just keep it surviving...:-))

20 June 2008

Manoj Srivastava: Manoj: The Command Prompt

Or theming PS1 I am not fanatical about my command prompt. No sirree, not me. It is just that I spend half my life either staring at an Emacs window, or at an xterm command prompt; so even a marginal boost in productivity goes a long way. And I am often logged in over ssh to machines half a continent away, and am still comfortable enough on the Linux VT to spend time there, and I often do not have the GUI gew-gaws feeding me data. Hmm. Data. inpuuut. No. Must focus. With all these open xterms and Emacs terminal mode frames floating around, it is easy to lose track of where I am on each terminal, and what the working directory is. So I want my command prompt to help me keep track of where I am. If the terminal is an xterm, the title can be setup like 
 "user@host:../shotened/path/to/current/working/dir"
I want to know what machine, id (am I root?), and directory I am in. If I am deep down in the labyrinths of some work related directory tree, I want the path to be pruned, from the left, one component at a time. However, this does not help me on the console; so I also want the path to be in the command prompt; but it should not take up too much of the command line; and ideally, should just go away as I type a longish command. Gawd, I love zsh. I once did a analysis of the command history. The most often used command sequences were 
 cd some-place, ls
So. pwd and ls. I really really want to see the directory listing when I change directory to a new one. As one grows old, memorizing the directory contents for dozens of machines all the time taxes the gray cells a bit. So, I figure, why not let the command prompt handle all that? Having the current working dir always visible cuts down a heck of a lot on the pwd commands, and so all that s left is to insist that the command prompt thingy always run ls after a change directory. Simple enough. Saves on typing. And time. And this is not just some crazy talk. I want help with noticing whether the previous command exited with an error status (useful for commands that normally do not create an output). If I am logged in to a machine on battery power, I want to know that. I also like visual cues to the amount of power remaining (good for my laptop on the long flights). I don t want to have to know f the machine uses APM or ACPI, I just want my prompt color to change as the power fades. I want my command prompt to let me know if I am in a directory which is under version control, and if so, what branch I am on. (I occasionally have to come in contact with arch, bzr, git, subversion, svk, and mercurial). If I am in a version controlled project, where I am relative to the root of the checked out tree is often more important than the absolute path, so I want to see relative paths, not absolute paths. I want to know if there are uncommitted files in the working directory. Visually. I want to be reminded if I am in the middle of an ongoing 
 rebase -i .
This is not asking for too much, is it? Command Prompt So, here is a screen-shot of this in action: I start from my home directory, go to a directory not under version control, go to a project under git, then a different project with uncommitted files, and then finally to a subversion checkout. All with an angry fruit salad of colors warrantied to make Martin Krafft want to claw his eyes out biggrin.png . If you use zsh, then just grab hold of this, and then do: 
 autoload -U promptinit
 promptinit
 prompt manoj
This might be a bit of a hit on slow machines, but even my laptop is a core 2 duo, so I do not find it noticeable. Oh, and if you liked this article, you might also like Theming Emacs, and Theming XTerms.
Categorized under:

Edd Dumbill: Secure LDAP replication

Ever the sucker for punishment, I decided to pick three difficult things and stick them all together: LDAP, SSL and replication. Here's how to make it go on Debian and Ubuntu. The problem You want LDAP replication to happen over the internet, and you want it to happen securely. The caveat I'm not going to tell you how to set up your LDAP from scratch here: I'm assuming you've reached a solution you're happy with and want to replicate it. The solution We're going to set up a replicating slave LDAP server, which communicates with the master over the internet via an SSL-protected connection. Enabling replication First up, the master LDAP server needs to be configured to permit replication.  The key lines to add to your slapd.conf include: 
moduleload syncprov
index entryCSN,entryUUID eq
overlay syncprov
syncprov-checkpoint 100 10
syncprov-sessionlog 200
These load up the synchronization module, add indices which make sync go faster, and enable sync. For more detail see the OpenLDAP site. Next you need to add a replicator user to your LDAP database, give your replicator user access to passwords as well as general read access. To create the replicator user, I made this simple LDIF file and fed it to ldapadd. 
dn: cn=replicator,dc=mydomain,dc=com
objectClass: simpleSecurityObject
objectClass: organizationalRole
cn: replicator
description: LDAP replicator
userPassword: TOPSEKRIT
Once this user is in your LDAP database, you should give it read access to passwords (I assume you've already given read access to authenticated users.) I have this in my slapd.conf: 
access to attrs=userPassword,sambaNTPassword,sambaLMPassword
...
   by dn="cn=replicator,dc=mydomain,dc=com" read
To check that this works, try using ldapsearch to check that the passwords are returned: 
ldapsearch -x -D cn=replicator,dc=mydomain,dc=com \
  -W   grep -i password
Enter the replicator password when prompted, and you should see the encrypted passwords from your LDAP database. Securing access Now you've got replication enabled on the master, you will want to ensure it is available on the internet only via TLS or SSL. Here's what I added to slapd.conf to enable this: 
TLSCertificateFile      /etc/ssl/certs/ldapserver_crt.pem
TLSCertificateKeyFile   /etc/ssl/private/ldapserver_key.pem
TLSCACertificateFile    /etc/ssl/certs/myCA.pem
TLSVerifyClient         demand
As you will guess from the configuration, the first two lines set the SSL key and certificate the master uses (see "A little twist" below for an important note on key permissions.) The third line tells slapd where to find my site-local certificate authority (CA), and the fourth line says slapd must require any connecting client to have a valid SSL certificate signed by the site-local CA. This is important, as it provides a second layer of access control: a replicating client must connect using a certificate you signed, plus the replicator password. Before this enables TLS access, we must tell slapd which network interfaces to listen on. To do this, edit the SLAPD_SERVICES variable in /etc/default/slapd. Here's my configuration: 
SLAPD_SERVICES="ldap://127.0.0.1/ ldap://192.168.0.1/ ldaps:///"
This enables regular LDAP on the loopback and intranet network interfaces, and LDAP/SSL on all interfaces, including the public internet. So, with slapd restarted we are at this situation: connections are now possible from the internet, as long as they are made over SSL with a certificate signed by our site-local CA. (In fact, you can make much finer-grained access restrictions in your configuration than I have done. Using LDAPS rather than TLS over regular LDAP is a rather broad precaution. As explained on the OpenLDAP site, the ssf= parameter can be used to require a certain level of secure connectivity on a per-user or client basis.) Setting up the replicating server
Your slave server should have the same configuration as the master, except you can leave out the bits enabling replication. Firstly, you'll need add to slapd.conf the replication configuration: 
syncrepl rid=123
        provider=ldaps://ldapmaster.mydomain.com/
        type=refreshAndPersist
        searchbase="dc=mydomain,dc=com"
        filter="(objectClass=*)"
        scope=sub
        attrs="*"
        schemachecking=off
        bindmethod=simple
        binddn="cn=replicator,dc=mydomain,dc=com"
        credentials=TOPSEKRIT
Most of this I took as boilerplate from the OpenLDAP documentation. Items to note include: And here's the /etc/default/slapd configuration: 
SLAPD_SERVICES="ldap://127.0.0.1/"
The slave slapd exists only in this case to serve the local machine. Finally, there's the tricky bit! You need to configure slapd to connect to the master server using a certificate. I'll assume you've created and signed a key and certificate pair for your slave server (see my post Low-tech SSL certificate maintenance for more on this.) Awkwardly, the TLS configuration in slapd.conf is for the server only. Replication works as a client, and thus needs separate configuration. Furthermore, you cannot configure this globally on your machine, as the SSL certificate is a per-user only parameter (see man ldap.conf for more information on this.) Instead, we must set it in slapd's environment. Add these two lines to the end of /etc/default/slapd: 
export LDAPTLS_CERT=/etc/ssl/certs/slapd.crt
export LDAPTLS_KEY=/etc/ssl/private/slapd.key
This file is sourced as a shell script by slapd's init script. Amend the path to your certificate and keys as required. Use /etc/init.d/slapd restart and you should be good to go. Finally, we want the slave server to be certain it's talking to the real master. So we also configure client connections to verify the SSL certificate of the peer, in ldap.conf again: 
TLS_CACERT      /etc/ssl/certs/myCA.crt
TLS_REQCERT     demand
A little twist One gotcha to notice with both client and server is that slapd runs as the openldap user by default on Debian. Also by default SSL keys are readable only by the ssl-cert group. You'll need add the openldap user to this group, otherwise it won't be able to access /etc/ssl/private. Related articles on this site: Join the conversation about this post

MJ Ray: Firefox 3, day 3: first impressions

Previously, I wrote:
Seriously: the browser looks like a big improvement from Firefox 2, but there are so many niggles with this download day idea...
In reply to Open Sesame Did you download Firefox 3?, I answer "Yes". It was a major upgrade for me, requiring new versions of Cairo and GTK+2, and installation of DBus-GLib on my GoboLinux computer, which brought in new versions of Xorg and so required a recompile of my GNUstep desktop applications. Once that was done, Firefox compiled unattended. As noted by Adam Sampson in the comments on my last post, even after building from source, you still get all the obnoxious click-through EULA and when you type about:config into the address bar, you get a "no user-servicable parts" sort of notice, which really sucks. I notice that MozCorp don't call it "100% Open Source", preferring instead Firefox: 100% Organic Software (because we need another marketing campaign for free software, right?), so I expect I need to winkle out the restrictively-licensed parts again - GNUzilla, there's still demand for your good work! After day 3 with Firefox 3, what do I think of it? Well, it seems a lot faster and a lot less RAM-hungry, and I'm quite impressed that all of the fancier bits of Koha and Wordpress seem to be working nicely but while I'm not annoyed enough to switch browsers yet (unlike FF3 and Safari - DrBacchus' Journal), there are still a hell of a lot of niggles and interface bugs. Some of the problems may have been introduced in Firefox 2, but I didn't actually use that enough to notice. My day-to-day browsing for the last year or so has been on a customised Firefox 1.5. The FF3 user interface has some big steps backwards from FF1.5: in particular, I've lost the "force pages that try to open new windows into the same window" option (or whatever it was called... I can't find the FF1.5 manual online anymore); some keyboard shortcuts have changed - for no good reason that I can see (JavaScript has switched from Alt-E n Alt-S to Alt-E n Alt-J, for example); what on earth is the history drop down doing next to the "Go Forward" arrow?; and the button to close a tab is on each tab, so I need to be careful to miss it when trying to switch to a tab and my pointer makes a pointless detour to the top-right when I want to close a tab. It's not all bad on the interface. The new RSS feed and bookmark links in the location bar are much better than in previous versions. The bookmark tagging and auto-generated folders could be a great idea once I've used it for a while. I'm pretty annoyed that Firefox 3 seems to come with some spyware enabled as default. I usually have cookies either switched off or set to "ask me every time" so I was surprised to be offered a cookie from safebrowsing.google.com! I know it's for a noble goal, but what's this doing enabled without asking first? Untick the "tell me if the site I'm visiting is ..." options in Edit: Preferences: Security if you don't want details of your browsing to be sent to the USA. Another thing which really annoys me is that the Firefox support site requires javascript and seems unhappy with my cookie settings. Not cool. Other than that, the main problems with Firefox 3 are omissions rather than bugs. For example, Microformats [Alex Faaborg] support was one of the long-trumpeted new features in Firefox 3, but they're really not obviously included, as noted by others in posts like Firefox 3 is here - where's the microformats? And finally, searching mozilla.com for firefox returns 0 hits, which is a bit strange... are they ashamed of it?

Russell Coker: Letter Frequency in Account Names

It’s a common practice when hosting email or web space for large numbers of users to group the accounts by the first letter. This is due to performance problems on some filesystems with large directories and due to the fact that often a 16bit signed integer is used for the hard link count so that it is impossible to have more than 32767 subdirectories. I’ve just looked at a system I run (Bluebottle anti-spam email service [1]) which has about half a million accounts and counted the incidence of each first letter. It seems that S is the most common at almost 10% and M and A aren’t far behind. Most of the clients have English as their first language, naturally the distribution of letters would be different for other languages. Now if you were to have a server with less than 300,000 accounts then you could probably split them based on the first letter. If there were more than 300,000 accounts then you would face the risk of having there be too many account names starting with S. See the table below for the incidences of all the first letters. The two letter prefix MA comprised 3.01% of the accounts. So if faced with a limit of 32767 sub-directories then if you split by two letters then you might expect to have no problems until you approached 1,000,000 accounts. There were a number of other common two-letter prefixes which also had more than 1.5% of the total number of accounts. Next I looked at the three character prefixes and found that MAR comprised 1.06% of all accounts. This indicates that splitting on the first three characters will only save you from the 32767 limit if you have 3,000,000 users or less. Finally I observed that the four character prefix JOHN (which incidentally is my middle name) comprised 0.44% of the user base. That indicates that if you have more than 6,400,000 users then splitting them up among four character prefixes is not necessarily going to avoid the 32767 limit. It seems to me that the benefits of splitting accounts by the first characters is not nearly as great as you might expect. Having directories for each combination of the first two letters is practical I’ve seen directory names such as J/O/JOHN or JO/JOHN (or use J/O/HN or JO/HN if you want to save directory space). But it becomes inconvenient to have J/O/H/N and the form JOH/N will have as many as 17,576 subdirectories for the first three letters which may be bad for performance. This issue is only academic as far as most sys-admins won’t ever touch a system with more than a million users. But in terms of how you would provision so many users, in the past the limits of server hardware were approached long before these issues. For example in 2003 I was running some mail servers on 2RU rack mounted systems with four disks in a RAID-5 array (plus one hot-spare) - each server had approximately 200,000 mailboxes. The accounts were split based on the first two letters, but even if it had been split on only one letter it would probably have worked. Since then performance has improved in all aspects of hardware. Instead of a 2RU server having five 3.5″ disks it will have eight 2.5″ disks - and as a rule of thumb increasing the number of disks tends to increase performance. Also the CPU performance of servers has dramatically increased, instead of having two single-core 32bit CPUs in a 2RU server you will often have two quad-core 64bit CPUs - more than four times the CPU performance. 4RU machines can have 16 internal disks as well as four CPUs and therefore could probably serve mail for close to 1,000,000 users. While for reliability it’s not the best idea to have all the data for 1,000,000 users on internal disks on a single server (which could be the topic of an entire series of blog posts), I am noting that it’s conceivable to do so and provide adequate performance. Also of course if you use one of the storage devices that supports redundant operation (exporting data over NFS, iSCSI, or Fiber Channel) then if things are configured correctly then you can achieve considerably more performance and therefore have a greater incentive to have the data for a larger number of users in one filesystem. Hashing directory names is one possible way of alleviating these problems. But this would be a little inconvenient for sys-admin tasks as you would have to hash the account name to discover where it was stored. But I guess you could have a shell script or alias to do this. Here is the list of frequency of first letters in account names:


First LetterPercentage
a7.65
b5.86
c5.97
d5.93
e2.97
f2.85
g3.57
h3.19
i2.21
j6.09
k3.92
l3.91
m8.27
n3.15
o1.44
p4.82
q0.44
r5.04
s9.85
t5.2
u0.85
v1.9
w2.4
x0.63
y0.97
z0.95

Share This

Cameron Dale: DebTorrent 0.1.7: Long-Lived Torrents for Testing/Unstable

This new version of DebTorrent finalizes the work I previously uploaded to experimental. This new version is the first in unstable to use unique piece numbers to create long-lived torrents for testing and unstable. It also comes with a new version of the apt-transport-debtorrent package, which can now tell APT not to cache any package files downloaded with DebTorrent, which should save some disk space for some people. The locations of the long-lived torrent files and piece files that DebTorrent uses has also changed, as they are now hosted on my merkel page and should be more up to date. However, I was not able to get the ssh trigger for the sync from ftp-master setup, nor am I able to access the projectb database through pygresql. If anyone knows how I can get those working, please email me. Here is the changelog for debtorrent: And for apt-transport-debtorrent:

19 June 2008

Joey Hess: delayed gratification

I got a mail today from someone who wrote a game in the early 90's, and then forgot about it. Meanwhile, his game was one of the first packages I put in Debian in 1996, and it remained in the distribution for the intervening decade, accumulating various bugfixes and improvements from users. Finally, in 2008, he typed the name of his old game into google, and found the package, and mailed me
So thank you for maintaining that little bit of history as much as you did, it put a smile on my face.
Ditto!

Martin F. Krafft: IPv6 with Debian

Even though I ve dealt with IPv6 for almost a decade, have delivered presentations, and given multi-day courses on IPv6 security aspects, I ve never actually added IPv6 to my own server/home network infrastructure because it seemed that Linux and/or Debian just weren t ready for it. This seems to have changed (although there are still a number of problems) and in less than a day, I put a few of my machines online. In the following, I d like to share with you how I did it.

Kernel versions and stateful connection tracking Unfortunately, I have to start off with some bad news: even though Debian etch, our current stable release, which uses a Linux kernel version 2.6.18, speaks IPv6, I cannot recommend it for deployment, as the 2.6.18 kernel does not support proper stateful connection tracking for IPv6, and thus makes it impossible to firewall hosts in a sensible manner (I always add local packet filters to all my hosts, and if only to guard against the situation when a user installs a malicious programme to listen on a high port). Of course, it is possible to configure a packet filter statelessly in an acceptable manner once you know the use case, so do with this information what you wish; I prefer to stay general for now. For me, a remedy is almost around the corner: the 2.6.24 kernel seems to support stateful connection tracking for IPv6, and it s even available for etch as it will be included in the upcoming etch-and-a-half release. I simply ended up using the kernel packages pre-release, and so far have not had a problem with it. To do so, add the following line to your /etc/apt/sources.list, making sure to use a close archive mirror: 
deb http://ftp.xx.debian.org/debian etch-proposed-updates main
I then just upgraded the system and pulled in all proposed updates. As that may have let in software that won t be part of etch-and-a-half, or even lenny, you may want to pin the archive and only upgrade the kernel packages, by adding to /etc/apt/preferences (replacing amd64 with your architecture): 
Package: *
Pin: release a=proposed-updates
Pin-Priority: -1
Package: linux-image-2.6.24-etchnhalf.1-amd64
Pin: release a=proposed-updates
Pin-Priority: 600
Alternatively, you could use the 2.6.24 linux kernel packages on backports.org.

Xen and IPv6 One drawback of switching to 2.6.24 is that you cannot run a dom0 on that machine any longer, so by practical extension, you cannot connect it to the IPv6 network with a packet filter in place. Supposedly, running 2.6.24 instances on a 2.6.18 dom0 is reported to work, however.

Configuring the packet filter The first thing I did was to configure the packet filter on each host appropriately. Unfortunately, this is harder than it should be, because to quote one of the netfilter developers when ip6tables was conceived, someone had a big bad brainfart : rather than adding IPv6 rules to your existing iptables ruleset, you have to create a new ruleset, duplicate all chains, networks, hosts, and individual rules, and maintain the two in parallel. Even though there are efforts of unification on the way, I speculate it ll take another couple of years until PF_INET6 will be fused into PF_INET and one will be able to do sensible cross-address-family packet filtering with Linux. Since I ve recently started to look (again) at pyroman, maybe the most logical way forward would be to extend it to write both, IPv4 and IPv6 rulesets from its knowledge about the hosts and networks you configured. Anyway, we want to get stuff working now! Thus, let s configure ourselves a packet filter. (Almost) all IPv6-related filtering must be configured via ip6tables (read on further down about IPv6 in IPv4 tunneling, the reason I said almost ). The following is a simple default ruleset to start with, which I put into /etc/network/ip6tables to load with ip6tables-restore: 
*filter
:INPUT REJECT [0:0]
:FORWARD REJECT [0:0]
:OUTPUT ACCEPT [0:0]
:in-new - [0:0]
### INPUT chain
# allow all loopback traffic
-A INPUT -i lo -j ACCEPT
# RT0 processing is disabled since 2.6.20.9
#-A INPUT -m rt --rt-type 0 -j REJECT
# allow all ICMP traffic
-A INPUT -p icmpv6 -j ACCEPT
# packets belonging to an establish connection or related to one can pass
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
# packets that are out-of-sequence are silently dropped
-A INPUT -m state --state INVALID -j DROP
# new connections unknown to the kernel are handled in a separate chain
-A INPUT -m state --state NEW -j in-new
# pass SYN packets for SSH
-A in-new -p tcp -m tcp --dport 22 --syn -j ACCEPT
# log everything else
-A INPUT -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[INPUT6]: "
### OUTPUT chain
# RT0 processing is disabled since 2.6.20.9
#-A OUTPUT -m rt --rt-type 0 -j REJECT
# allow outgoing traffic, explicitly (despite chain policy)
-A OUTPUT -j ACCEPT
### FORWARD chain
# RT0 processing is disabled since 2.6.20.9
#-A FORWARD -m rt --rt-type 0 -j REJECT
# disallow forwarded traffic, explicitly (despite chain policy)
-A FORWARD -j REJECT
COMMIT
Note that this recipe is pretty much unusable on pre-2.6.20 kernels due to their broken implementation of stateful connection tracking. The ruleset should be fairly obvious, but you might wonder about my use of REJECT and allowing all ICMP after all, you ve heard for the past 30 years that ICMP is a bad hacker protocol , and Internet security is no domain for being nice to people, so to prevent any information disclosure, you should DROP connections, not let people know that they re simply not allowed. Well, to hell with all that! I don t see a single reason or attack vector that is foiled by DROP or disallowing ICMP. In fact, it s just security by obscurity, and might inconvenient at the same time. ICMP is also much more important with IPv6 than with IPv4 (it replaces ARP, for instance), and it s actually useful to be able to ping hosts, or get back informational messages on why something failed. Finally, rejecting traffic rather than dropping it doesn t suggest to a hacker that something s hidden here. Then there is RFC 4890, which almost made me puke. This document is part of the reason why I say: let s fix problems in the kernel, rather than shielding them with unreadable and unmanageable rulesets!

Getting connected If you already have an IPv6 address, you re basically ready to go, but may want to read further down on how to connect your local network to the IPv6 Internet as well. If you are searching for a provider, have a look at the list of providers with native IPv6 connectivity over at sixxs.net. If you are reading up to here, I assume you are connected to the Net with IPv4. There are two ways for you to move towards IPv6: 6to4 or by way of a tunnel provider. A Kiwi website explains how to setting up 6to4 connectivity, and thus I will concentrate only on the tunnel approach. Even though everyone can set up 6to4 in a breeze without any accounts or waiting, there are a number of security considerations, it s pretty ugly to debug (due in part to asymmetric routing), and makes your life unnecessarily difficult when all you have is a dynamic IP that changes from time to time. If you are stuck behind a NAT gateway, you cannot use 6to4 either. Thus, I prefer the tunnel approach. With the tunnel approach, IPv6 packets are wrapped up in IPv4 packets on your host, and sent to the IPv4 address of your tunnel provider, who has native IPv6 connectivity. The tunnel provider unwraps your packet and shoves the contained IPv6 packet onto the backbone. The IPv6 address you used as source address is routed to the tunnel provider, so any replies arrive at their machines, where they re again wrapped into IPv4 packets and sent to your (publicly-accessible) IPv4 address. Those IPv4 packets specify payload type 41 ( ipv6 ), which is why we need those -p ipv6 -j ACCEPT rules in the iptables ruleset. There are a few tunnel providers out there. I chose SixXS and have not regretted my choice. I shall thus assume that you do the same: sign up for an account right now, so that you have it by the time you finished reading this document! SixXS works on a credit system: tunnels and subnets cost credits, which you can accumulate by maintaining your tunnels properly. This ensures that everyone can play around, but to do more advanced stuff, you need to first display competence with the basic concepts. Your first step with SixXS will be to request a tunnel. SixXS offers three types of tunnels:
  • static tunnels, for those with static IPs,
  • heartbeat tunnels, for those with dynamic IPs, and
  • AYIYA tunnels, for those behind NAT gateways.
Each of these tunnels have advantages and disadvantages, as everything does: the first two types of tunnels use IP protocol 41 packets to encapsulate the IPv6 packets. As such, there are security considerations involving the impersonation by spoofing, and all upstream firewalls must let protocol 41 pass. AYIYA addresses these problems by using signed packets, but that solution comes with extra computation overhead and smaller MTUs. I suggest to use the first type of tunnel that fits your situation. Debian s aiccu package can take care of heartbeat and AYIYA tunnels for you, and it can even set up static ones. During registration, you will also need to choose a PoP , which stands for Point of Presence . If your country only has a single PoP, that s the one you will end up using (unless you have a good reason for another one), but if there are more options, I strongly suggest that you go through the list of PoPs and select the one with the best roundtrip time and lowest latency from your location! Note that you must answer ping requests (ICMP echo-request) from the PoP you chose, or else the tunnel will not be created. Once your tunnel request gets approved, you ll get a /64 prefix, in which you only use two addresses: the PoP will configure the :1 address and you need to configure your host to use the :2 address on the tunnel interface. You ll also be told the IPv4 address of your PoP endpoint . Joey Hess taught me that aiccu can set up the interface for you, using the data it queries from the SixXS registration (TIC) server. I tried it, and it works. However, I prefer the pure ifupdown approach, as it makes things explicit and allows me to use the hooks for stuff like loading the packet filter. So in my /etc/network/interfaces, you can find: 
auto sixxs
iface sixxs inet6 v4tunnel
  endpoint 194.1.163.40
  address 2001:41e0:ff00:3b::2
  netmask 64
  gateway 2001:41e0:ff00:3b::1
  ttl 64
  pre-up ip6tables-restore < /etc/network/ip6tables
  up ip link set mtu 1480 dev $IFACE
  up invoke-rc.d aiccu start
  down invoke-rc.d aiccu stop
Make sure to read about MTU values of the tunnel and adjust the 1480 value in the above to your tunnel settings and ISP connectivity. Also set ipv6_interface sixxs in /etc/aiccu.conf, if you are using aiccu, or else aiccu will bring up a duplicate/additional interface. If you tell it to use the same interface, it will actually execute all the same commands (which will fail), but won t report any errors. A future version will have a switch to prevent it from configuring the interface. Unfortunately, this will probably not work. The reason is that your regular IP packet filter (iptables, without the 6) doesn t let those encapsulating IPv4 packets pass, unless we tell it to; we probably want to do this early on in the chain, and also limit it to our tunnel peer, so: 
iptables -I INPUT -p ipv6 -s 194.1.163.40/32 -j ACCEPT
For AYIYA, you need to open port 5072, either for UDP, TCP, or SCTP, depending on how you configured it. Also have a look at this FAQ entry on what a firewall needs to pass. If it still doesn t work, you have an upstream packet filter that needs some of those holes poked. Good luck. In most situations, the FORWARD chain does not get such a rule, since the tunnel terminates at the gateway, which routes to a native IPv6 network, or another tunnel. Allowing tunnels through a gateway is almost always a bad thing, as it would allow undetected and untraceable traffic from compromised boxes in the local network. The OUTPUT chain also does not need such a rule, if you have configured stateful filtering properly. Now bring up the interface and verify the connection: 
# ifup sixxs
# ping6 -nc1 2001:41e0:ff00:3b::1
PING 2001:41e0:ff00:3b::1(2001:41e0:ff00:3b::1) 56 data bytes
64 bytes from 2001:41e0:ff00:3b::1: icmp_seq=1 ttl=64 time=74.0 ms
[...]
# ping6 -nc1 ipv6.aerasec.de
PING ipv6.aerasec.de(2001:a60:9002:1::184:1) 56 data bytes
64 bytes from 2001:a60:9002:1::184:1: icmp_seq=1 ttl=55 time=91.5 ms
[...]
Welcome to the Internet of the future!

Setting up an IPv6-capable gateway Your IPv6 connection works, but it s limited to a single address, and you do not get to specify the reverse DNS PTR record for it. Since the concept of NAT is mostly absent from IPv6 (thanks! thanks! thanks!), you will not be able to connect any other hosts to the IPv6 network. If your local network has a few hosts behind a gateway, you will need to request a subnet from SixXS and configure your gateway (which has the tunnel connection) appropriately. Don t worry, this is not really very difficult. First, request a subnet for your tunnel from your PoP via your SixXS homepage. Once approved, you will get a /48 prefix for your own use: 2^80 1.2 heptillion addresses which are yours to assign to every dust particle in your office or home, if you so desire. The way I set it up is to add the first of these addresses to your internal interface on the gateway, by adding the following two lines to the interface s stanza in /etc/network/interfaces; you will need the iproute package installed (which you should be using for everything network-related anyway): 
up ip -6 addr add 2001:41e0:ff12::1/64 dev $IFACE
down ip -6 addr del 2001:41e0:ff12::1/64 dev $IFACE
Instead of bringing the interface down and up, just run ip -6 addr add 2001:41e0:ff12::1/64 dev eth0. Note the use of the /64 prefix instead of the /48 that got assigned, leaving only 20 pentillion addresses. Oh no! The reason for this is buried in the specs: basically, /48 is a site prefix, but individual networks should not be larger than /64, which is the prefix length of links in the IPv6 domain. Now is also a good time to enable IPv6 forwarding, e.g. like so: 
# echo net.ipv6.conf.all.forwarding = 1 >> /etc/sysctl.conf
# sysctl -p /etc/sysctl.conf
Obviously, you will also need to change the policy on the ip6tables FORWARD chain. For now, let s just set it to accept. You should later create a proper ruleset, though! 
# ip6tables -I FORWARD -j ACCEPT

Bringing IPv6 to your local network The final step is to spread the love to your local network. Refrain from selecting addresses from your subnet and assigning them to the local hosts, or wondering how to configure the DHCP server, because IPv6 does it differently: your gateway will advertise its routes (which includes a default route) to your network, and each host will pick an address based on its MAC address (unless it already has an EUI-64 address assigned. This all happens automagically, at least with current Debian and Windows machines. On the gateway, you need to install radvd and simply tell it which prefix to use on which interface. My /etc/radvd looks like this, and I won t explain it: 
interface eth0
 
  AdvSendAdvert on;
  prefix 2001:41e0:ff12::/64
   
   ;
 ;
Note again how we advertise a /64 network rather than the /48 we own . You cannot advertise smaller networks if you want automatic configuration to work, and you should not use networks larger than /64 in any case. If 2^64 addresses are not enough for you, I trust you ll be able to figure out how to advertise another of your 65536 /64 prefixes in the /48 subnet to appropriate hosts. Restart radvd and run over to another host to witness how it automagically gets connected to the IPv6 network by scanning /var/log/kern.log and watching the output of ip -6 addr and ip -6 route. Try ping6ing from there! Find the dancing turtle! It should all work. If you don t like the automagic aspect of this, look into stateful configuration, using DHCPv6, as provided by dibbler-server and ?wide-dhcpv6-server.

Resolving names Take note of the IPv6 address of each host. There s a way to determine it given the host s MAC address, but this is easier (ipv6calc is also useful). You might want to let your local DNS server know by adding AAAA records in parallel to the existing A records, and possibly even adding PTR entries. If you re serious about IPv6, you can tell SixXS to delegate reverse lookups for the IPv6 addresses to your DNS servers, but you ought to refrain from polluting the DNS namespace. Note that bind9-host provides an improved host tool, which fetches all kinds of information about names, not just the one single information configured as default: 
% host pulse.madduck.net
pulse.madduck.net has address 130.60.75.74
pulse.madduck.net has IPv6 address 2001:41e0:ff1a::1
pulse.madduck.net mail is handled by 99 b.mx.madduck.net.
pulse.madduck.net mail is handled by 10 a.mx.madduck.net.
% host 2001:41e0:ff1a::1
1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.a.1.f.f.0.e.1.4.1.0.0.2.ip6.arpa
domain name pointer pulse.madduck.net.
Oh, and if you re really that curious about how IPv6 addresses are computed from MAC addresses, read RFC 2464. Basically, given a prefix 2001:41e0:ff1a:: and a MAC address aa:bb:cc:dd:ee:ff, the resulting IPv6 address is obtained by:
  1. inserting ff:fe into the middle of the MAC address to yield aa:bb:cc:ff:fe:dd:ee:ff;
  2. flipping the second lowest bit of the first octet to yield a8:bb:cc:ff:fe:dd:ee:ff;
  3. removing the odd colons to yield a8bb:ccff:fedd:eeff, the EUI-64;
  4. concatenating the prefix and this result to get 2001:41e0:ff1a::a8bb:ccff:fedd:eeff.
If you find your (Windows) IPv6 addresses changing all the time, you might be faced by privacy features .

Remaining issues Even though my IPv6 connectivity works, I have two remaining issues.

Sending larger amounts of data to the network I am experiencing a curious issue where outgoing ssh IPv6 connections time out and outgoing data transfers hiccup. I have yet to find out what s going on.

Mapping names to laptops Laptops generally have two interfaces, one with a cable, and the other wireless. Both of these interfaces will have separate MAC addresses, and by extension, the laptop will have different IPv6 addresses depending on how it is connected to the local network. I want to be able to connect to laptops without knowing the medium they use to connect to the network. Unfortunately, there seems to be no feasible way. The solutions I see are:
  • override the MAC address of one interface with that of the other, which is going to cause bgi problems in the case when the laptop (accidentally) gets connected to the same network twice;
  • add both IPv6 addresses as AAAA records to the laptop s DNS name, which will cause random delays when connecting as the resolver may return the currently inactive address first;
  • set up mobile IPv6, e.g. by following this Mobile IPv6 how-to, which would allow accessing the laptop uniformly, no matter where in the world it is. Unfortunately, Debian s support for Mobile IPv6 is severly lacking at time of writing. Also, Yves-Alexis Perez notes that this how-to is horribly outdated and promised to tend to it Real Soon Now .
The second solution works for me for now, but I am interested in the third. In response to this document, Andreas Henriksson has suggested the replace the stateless configuration (radvd) with stateful configuration, using DHCPv6. I have yet to investigate this option. Jeroen Massar suggests to unite cable and wireless into a bridged interface, which seems like a very good idea.

Credits Thanks to Bernhard Schmidt, William Boughton, and Jeroen Massar, and everyone on #ipv6/irc.freenode.org for their help over the past few weeks, and all those who fed back comments in response to this document!

Jeff Bailey: Medical Update ; iTunes

This past week has been a really good ride on the medical front. As of yesterday, I've dropped the 1g of Tylenol I was taking first thing in the morning. Pills-wise, that leaves me with one Ultrum (Tramadol) at night before bed. Sleeping on my surgery site without pills still causes me to wake up in the middle of the night. Next week's goal is to drop the prescription drugs completely, except for an occasional one on the weekend when I'm doing a lot with Leif. As far as I know, it's okay to take Tylenol and have an occasional glass of wine - I'm going to check with the pharmacist - but there's a winery tour that I'm signed up for late next week...

I'm also back to Physical Therapy and have appointments booked for twice a week over the next few months. My PT doesn't seem to think that I'll need to stay that long, which would be awesome, but my hopes aren't high on that. We keep discovering new things like "Hey, if you had triceps, you could do this exercise that I want you to. But you don't, so do tricep pull downs for now." From not doing a whole lot, I find I have to ask other people to do things like open jars, or open heavy doors still. We'll see how it goes. My lifting/bending/twisting restrictions won't be lifted until August and then the use-more-than-4kg-weights exercises can finally start.

I'm biking around the campus at work a lot now. I'm getting braver with going out on the streets, and think that I'll probably be ready to try taking my road bike for a trip between home and work early next week. I'd originally hoped for this week, but I'm just not there yet.

In other news, I'm hoping that someone can point me to a freer replacement for this: I've wanted Angie to see the movie Wargames for a bit, so I plugged the Windows laptop into the TV and the stereo, rented it off of iTunes, made it full screen and we enjoyed the movie. I'm quite fussy about DRM when it comes to music or movies that I've purchased, but I'm less fussy where I've contracted to watch something for a day. Netflix has an on-demand movie service like iTunes as well, but neither of them seem to run under Linux or Wine. I suspect I'll need to find the person who was hacking wine to work with iTunes and see if (s)he likes occasional shipments of beer. =)

Manoj Srivastava: Manoj: Theming XTerms

Or configuring LS COLORS After successfully Theming Emacs, I decoded to turn my efforts to providing more copacetic colors in other aspects of my working environment. I ll write more about my effort to write my own color theme for fvwm when it is closer to being done. For now, I ll concentrate on ANSI Colors in XTerms, and how one may theme those. The old Tektronics (or ANSI) terminals had 8 colors, namely, black, red, green, yellow, blue, magenta, cyan, and white. These colors had a normal, and a bright mode. The default set of colors is OK, but is not optimal, since they have to work on both dark and light backgrounds; however, I use mostly dark background colors. ls colors Given that, I set to design my own colors, using fully laturated colors, with a base luminosity of 70, and equally spaced in chroma and used the same saturation and chroma, but a luminosity of 80, for the bright set. This results in a pleasant pastely set of colors, which are e4asy on the eyes, when used on very dark backgrounds. The configuration can be seen in XTerm. I set the colors for rxvt as well as xterm. Now, the most noticeable effect of these changes is if using the color option for ls, therefore the next thing to do was to configure the colors used by ls using a utility dedicated for that task, namely, dircolors. By trial and error, I modified the default configuration provided by dircolors and ended up with dir colors, with the results as you see.
Categorized under:

Next.

Previous.