Search Results: "ana"

16 December 2009

Raphael Geissert: what could you tell about spammers after receiving one billion messages?

This is a question that Project Honey Pot's report answers.

They have released a report of the analysis of not only spam messages, but also of the harvesters behind spam.
Available with a simple overview the complete report provides some rather interesting details.

Do spammers take holidays?
The volume of spam drops approximately 21% on Christmas Day and 32% on New Year's Day. Saturday is the quietest day of the week.


And for those sending mails to public mailing lists, the BTS, etc:
How much harm does a harvester visiting my site do?
You can expect to receive 869 spam email messages every time a harvester picks up your email address from a website online.


The interesting bits are really worth the five-ten minutes it takes to read the complete report, enjoy it (and yay, it is CC-BY).

14 December 2009

Russell Coker: Nagios and SSL in Debian

I was doing some work on NRPE (the Nagios Remote Plugin Executor) and I noticed bug report #547092 [1] which concerns the fact that the default configuration uses the same SSL certificate for all Debian servers and provides a patch to fix the problem. After building the patched package I followed the advice of the DebianAdministration.org article on creating self-signed SSL certificates [2]. cert_file=/etc/ssl/certs/FOO-cert.pem
privatekey_file=/etc/ssl/private/FOO-key.pem
cacert_file=/etc/ssl/certs/cacert.pem Then I added the above lines to /etc/nagios/nrpe.cfg to instruct the nrpe to use the certificates. For the Nagios server I had the problem that most of the systems I monitor run old versions of NRPE while only a few are recent Debian systems that allow me to easily install a new SSL checking nrpe. So I installed the following script as /usr/lib/nagios/plugins/check_nrpe to run either the old or the new check_nrpe: #!/bin/sh -e
if echo $2 egrep -q server0\ server2\ mail ; then
/usr/local/sbin/check_nrpe -C /etc/cert/cert.pem -k /etc/cert/key.pem -r /etc/cert/cacert.pem $*
else
/usr/lib/nagios/plugins/check_nrpe.orig $*
fi The reason I started working on Nagios was to try and solve bug #560002 [3] which I filed. The bug concerns the fact that applications such as mailq which are run as part of Nagios checks were inheriting a TCP socket file handle from the nrpe. SE Linux prevents such file handles from being inherited, but it does mean that I get audit messages (and this is not a good case for a dontaudit rule). Update:
One thing I forgot to mention is that the SSL key checking requires that the server common name used in the SSL certificate of the nrpe system matches the name that is used by the check_nrpe program. So if you check by IP address then you need to use the IP address in the certificate name which is rather ugly. So I have moved to putting the hostname of each server in /etc/hosts on the NAGIOS system and using the hostname in the SSL certificate. This required using $HOSTNAME$ instead of $HOSTADDRESS$ in the NAGIOS configuration (thanks to John Slee for a tip in that regard). Update2:
I removed some printf debugging from the script. It seems that I included a pre-production version of the script in the first version of this blog post.

Phil Hands: arduino for the java challenged

Having excitedly bought an Arduino from the nice folks at tuxbrain.com while they were at DebConf9, I was then a little deflated to discover (while attending the UKUUG's Arduino Workshop in August), that the IDE is written in Java. While I can see that the authors might perceive this as the easiest way to provide cross-platform pointy-clicky-ness, JREs have a half-life of about an hour on my systems.Anyway, I fought with Java to show willing, until I'd determined that the IDE didn't seem to be happy running under xmonad (rendering menus so that they were unclickable) and gave up -- spending the rest of the tutorial proving to myself that I could compile and upload C code for the little beastie, but failing to compile Arduino sketches on the command line.Ever since, I've been meaning to work out the required CLI incantations, and finally made time over the weekend. The incantations turn out to be rather simple, if not very obvious, so I thought the world could benefit from a package that encapsulates the required knowledge.I've pared the source tree down to the c++ code and some examples and tweaked the Makefile so that it'll do it's work out of tree, referring to the files that the package installs for you, which allows you to apt-get install aurduino-core and then cut&paste a few lines from the contained README.Debian in order to get an LED blinking for the first time.The resulting git repo is now on Alioth. The binary-indep package it produces I've called arduino-core -- hopefully I've left enough room for any fan of the Java IDE to shove that back into the upstream branch and add packages for the full IDE, and maybe throwing in another package for the GTK IDE that's out there.I'm very mildly concerned by the fact that Arduino is a trademark of the Arduino Team, but that trademark is for the circuits, not the software, so this is probably irrelevant. If you think I should care enough to rename the package to freeduino, say, please get in touch.Anyway, if you have an Arduino (or similar) please grab a copy and test it.

13 December 2009

Russell Coker: Links December 2009

Dan Gilbert gave an insightful TED talk about our mistaken expectations of happiness [1]. Don Marti has an insightful post about net neutrality and public property [2]. When net access requires access to public property then it should be sold in a neutral manner. Rachel Pike gave an interesting TED talk about the scientific research behind a climate headline [3]. The people who claim to be skeptical of the science should watch this. Mark Peters wrote an interesting article A Happy Writer Is a Lousy Writer about the correlation between emotional state and work quality [4]. Apparently watching a film about cancer will make people more careful and focussed on details. CERIAS has an interesting short article about Firefox security as well as some philosophy on why web browser security generally sucks [5]. Cory Doctorow writes in the Guardian about Peter Mandelson s new stupidity in trying to legislate against file sharing [6]. This is going to seriously damage the economy of every country that implements it. Charles Stross has been blogging a series of non-fiction essays about space colonisation, in The Myth of the Starship he describes how most ideas of space travel are bad and how the word ship is always going to be unsuitable [7]. Brent T. White is an associate professor of law at the University of Arizona who has written an interesting paper about mortgages [8]. He says that anyone who is underwater (IE owing more than the value of their house) should walk away. The credit damage from abandoning a bad mortgage apparently isn t that bad, and there is the possibility of negotiating with the bank to reduce the value of the loan to match the value of the house. Mako is working on a project to allow prisoners to blog [9]. It s basically a snail-mail to web gateway as the prisonsers are not allowed Internet access. PracticalEthicsnews.com has an article about the special status that homeopathy is given [10]. It also notes that homeopathic medicines include arsenic and mercury. Such quackery should be outlawed, a life sentence for homeopathy would be appropriate IMHO. Cory Doctorow wrote an interesting essay about why he is not selling one of his books in audio form (he s giving it away) [11]. He concludes by noting that he wants no license agreement except don t violate copyright law . The fact that he can t get anyone to sell an audio book under such terms is a good demonstration of how broken the marketplace is. Thulasiraj Ravilla gave an inspiring TED talk about the Aravind Eye Care System a program to bring the efficiency of McDonalds to eye surgery [12]. Hopefully that program can spawn similar programs for other branches of medicine and spread to more countries. In many ways they are providing better service (both in quality and speed) than people in first-world countries who pay a lot of money can expect to receive. Scott Kim gave an interesting TED talk about his work designing puzzles [13]. He is also a big fan of social networking, unfortunately (for people like me who don t like social networking) his web site ShuffleBrain.com relies on Facebook. Gordon Brown (UK Prime Minister) gave an inspiring TED talk about global ethic vs the national interest [14], with a particular focus on the global effort required to tackle the climate change problem. Now if only we could get Kevin Rudd to listen to that. Brough has written an interesting analysis of the AT&T network problems that are blamed on the iPhone [15]. His essential claim is that the problem is due to overly large buffers which don t cause TCP implementations to throttle the throughput. This seems similar to my observations of the Three network in Australia where ping times of 8 seconds or more will periodically occur. One particularly nasty corner case with this is when using a local DNS server I can have a DNS packet storm where basic requests time out while BIND uses a significant portion of available bandwidth (including ICMP messages from receiving ports that BIND has closed). To alleviate this I am now using the Google public DNS service [16] (the Three DNS servers never worked properly).

Debian News: Debian is moving sieve to its IANA allocated port

The IANA port allocated for ManageSieve is 4190/tcp, and the old port used by timsieved and other managesieve software in many distros (2000/tcp) is allocated for Cisco SCCP usage, according to the IANA registry.

Starting with the version 4.38 of the Debian netbase package, the sieve service will be moved from port 2000 to port 4190 in the /etc/services file. This affects to those using Debian Squeeze, and Debian Sid. Debian Etch and Debian Lenny users are NOT affected.

Any installs which used the sieve service name instead of a numeric port number will switch to the new port number as soon as the services are restarted/reloaded, and in some cases, immediately after /etc/services is updated.

This will affect Cyrus IMAP. This may also affect other sieve-enabled software such as DoveCot.

In order to avoid downtime problems, mail cluster administrators using Debian are urged to verify their Cyrus (and probably also DoveCot) installs, and take measures to avoid services moving from port 2000/tcp to port 4190/tcp by surprise in either servers or clients.

It is worth noting that:

  1. /etc/services will only be automatically updated if you never made any modifications to it. Otherwise, you will be presented with a prompt by ucf or dpkg asking you about the changes.
  2. You can edit /etc/services and change the sieve port back to 2000 if you want (this is not recommended, though).
  3. You can edit /etc/cyrus.conf and any other relevant config files for your mail/webmail cluster (e.g. on the sieve web frontends) ahead of time to force them all to a static port number.
  4. You can configure cyrus master to listen on *both* ports (2000 and 4190) at the same time, and thus avoid the problem entirely. This also allows for a much more smooth migration from port 2000 to port 4190.

Contributed by: Henrique de Moraes Holschuh

Clint Adams: Sanjana Kumar goes to the movies

I have an imaginary automobile. It is the finest hybrid of German and Japanese engineering. It has a peppy little engine and a turning radius so small it would make a baby camel cry, and a six-speed manual transmission, and it is powered by the urine of vegan bears. My imaginary father asked me why I would possibly want to buy an imaginary car with a manual transmission. Obviously, whichever marketing campaign brainwashed him in the first place wasn't strong enough on the memetics, because after borrowing my imaginary car, he shut the hell up. You see, it's considerably more fun to drive than an automatic. There are other benefits as well, of course. The primary one is that the transmission does what I want. An automatic transmission is configured for maximum comfort , so it wastes a lot of power when shifting. With a manual, you can slip from gear to gear with minimal clutch time, or in some cases, if you're feeling sassy, without using the clutch at all. You can control your engine speed so that your shift will be just as smooth as an automatic without the egregiously long clutch disengagement or shift overlap. You also can see the road in front of you, so you're always going to be smarter than an automatic transmission, unless you don't understand how slopes work or what collisions with other cars mean to your well-being. Now some NRA member is going to point out that you can get a shift kit and tweak your automatic to be less awful. While spending money on this type of thing sounds like a great idea, a shift kit is not going to bring an automatic into the same league as my car. I can still start the car when the battery's dead. I can get higher fuel efficiency (vegan bears don't exactly urinate for free). I can shift into fifth gear, and coming from a full stop, drive up a hill without stalling (this is particularly useful when a squirrel has gnawed the teeth off of all your other gears). Furthermore, shift kit or not, an automatic is absolutely no fun. Someday, when transmissions have advanced into the realm of liquid hyperspace buckminsterfullerene cones that commune with the unicorns, and they'll have cameras and radar feeding the car's computer with useful information about its surroundings so it can make intelligent shift decisions faster than a human. Then I'll have to choose between saner and more fun. That era has not yet arrived. This is a lousy metaphor for debhelper.

12 December 2009

David Paleino: wicd_1.6.2.2-5~exp1_i386.changes is NEW

Why is wicd NEW? Simple, I managed to split the files into separate packages. As soon as it clears NEW, those who are using experimental will be able to install a cli-only wicd, i.e. no more unnecessary throwing in of GTK+ and X Smile I will make a call for testing as soon as this package clears the queue. Hope I didn't make too much mistakes Wink

Russell Coker: US Border Security

Making Light has a post about the Canadian sci-fi author Peter Watts who was beaten and jailed overnight without access to a lawyer by US border guards because he asked what they were doing [1]. Apparently one is supposed to cringe in fear whenever questioned by authority in the US, so much for land of the free . In 12 hours time I will hopefully have got past the security police and be boarding a flight out of the US and it will be a great relief. On a practical note, if al Quaeda was a serious threat, if anyone really cared to stop the importation of drugs, of if any of the other reasons for having security on borders were taken seriously then this sort of thing really wouldn t help to achieve such goals. One of the purposes of a justice system is to encourage cooperation from the population. If a suspect will be treated fairly and considered to be innocent until proven guilty then there is no reason not to report suspicious people to the police. If however a suspect will be summarily beaten and denied access to a lawyer then no decent person can make a police report unless they are certain of the suspect s guilt beyond all reasonable doubt. When Barack Obama was elected the majority of the world s population was greatly relieved, we expected some significant changes in the way things are run. It doesn t seem that he is living up to our expectations.

Russ Allbery: Peter Watts attacked by US border guards

Signal boost. While returning from a trip to the US (in other words, going to Canada) on December 8th, Canadian SF author Peter Watts was attacked, beaten, pepper-sprayed, and imprisoned without access to legal counsel by the US Border Patrol. See also his post on the subject. The government's case has obviously not yet been presented. However, given the past behavior of the US Department of Homeland Security, I find Watts's description of the event entirely credible. This is an outrage. This is the kind of thing that happens in third-world dictatorships. See also Making Light and Boing Boing. John Scalzi's post has information on how to donate money to Watts to get through this legal fight. I will be giving him money. I also plan on calling my US Representative and Senators when I'm no longer at work, as this sort of treatment of a Canadian citizen warrants a Congressional investigation and heads rolling, not only at the agent level but at the Homeland Security management level. Even apart from basic questions of human rights, Homeland Security is going to systematically destroy the tourist industry in the United States if this sort of behavior is not stomped on immediately and severely. There are border patrol guards who are probably still on duty today who are common criminals and belong in prison for assault. ETA: The government's side of the story. This is pretty much what I expected, and with the data available so far, I feel confirmed in my initial reaction. This level of force is completely disproportional and flatly unacceptable short of Watts pulling a weapon on the agents, which they are not claiming that he did and which therefore he clearly didn't do. I'm highly skeptical of this supposed "choking" of a police officer after they pepper-sprayed him. This reads like the standard cover-your-ass charges filed by the police after they beat someone up. Once they've provoked the struggle by grabbing you and have done something as painful and disorienting as pepper-spraying you, chances are you're going to physically react in ways that they can conveniently misinterpret and use as the basis for charges. People elsewhere have pointed out that contacting Congressional representatives is probably better done Monday morning since nothing will happen during the weekend.

11 December 2009

John Goerzen: Graduating and what now?

I guess I ve never been one to do things a certain way just because that s how they re normally done. I took my first college class pretty young, back in 1995, just after my freshman year in high school. By the time of my junior year in high school, I was well on my way through the computer science curriculum at Wichita State, and would probably have graduated from WSU at about age 20 had I not moved to Dallas, and then Indianapolis, for jobs. I did so because I (correctly) thought that in my line of work, experience was more important than a degree. However, playing into that was the incorrect notion that an education was useful primarily as a means to a career a notion that colleges and universities unfortunately have been promoting themselves. After moving back to Kansas, I started work to finish my degree I had effectively one year left had I been a full-time student. But it was difficult doing so, living almost an hour from Wichita, having a full-time job, and then when Jacob was on the way, I stopped classes entirely. Having online classes available to help me finish out the degree has been great. A year ago, I re-thought the value of education to me, concluding that there is a lot of value in education for its own sake. Looking at education as little more than a means to a career is the wrong way to approach it. Since I have had that re-evaluation, I ve adjusted my path even at this late point, and have enjoyed my classes more than I ever had before because of it. Graduating! So anyhow, here I am, graduating in a few days, 14 years after I started. I ve had some good news this semester, too: I ll pass the requirements to be a graduate of the WSU honors program, and also to graduate magna cum laude. So it may have taken me 14 years to do finish college, but at least I ve done it well . It feels weird to be about to be done with college, after so many years of being almost done, except for And a question I ve been thinking about a lot is: What next? Obviously I ll start by taking a break from college stuff. I want to be able to spend more time with Terah, Oliver, and Jacob, and there are things around the house that need attention. But after that? I figure I have a few options: No more college, but some reading. I do enjoy reading, and especially reading older works that are nicely out of copyright and thus free. I ve read few of the classics, but I enjoy reading them, and want to be better informed about them too. That could be fun. But sometimes it s hard to know how to place things in context. For instance, I don t think I could have gotten as much out of reading some famous philosophers as I did by doing so in a class that put them all in context and discussed criticisms of different positions. I also wouldn t necessarily have the ability to discuss and debate ideas with my classmates. Take the occasional undergraduate class in a topic that interests me. I m especially interested in history and literature, and haven t had much exposure at the college level in either. Various online courses are available from universities in Kansas on those topics. I could take a course or two each year and become a better-informed, more well-rounded person, and study an interesting topic at the same time. Maybe this would eventually lead to a degree, and maybe not. It would be fine either way. Graduate studies. It s a lot of work, and I m not sure it would be all that interesting to me. I ve looked some at curricula. I don t think that I m interested in computer science as a graduate program; I enjoy coding more than theorizing and analysis. But I also don t think that a graduate MIS or some such degree is for me; most of them appear to either focus on Windows or cover things that I ve already known how to do for years, and thus I wonder just what the point of them is (in addition to questioning how rigorous the curriculum is). I could also, down the road, pursue a graduate degree in something like English, philosophy, or history, but it seems a natural path to that would be to take more undergraduate classes in those topics first. So for now, I ll sit tight. I ll see how I feel by summer, how pressed for time I feel at home, and if I still feel the desire to take more classes. If so, I suspect I ll try to take some undergraduate classes in whatever sounds interesting to me at the time. In a way, that is quite freeing take a class on whatever topic I like, without a defined degree in mind. I ve never been free to do that before, and I like the idea of it.

10 December 2009

Matthew Garrett: Nook update

My nook arrived today, along with an email asking for my shipping address in order to be sent a CD with the source on. So that's progress. The nook itself is an interesting device - it comes in impressively well produced packaging, which looks easily as attractive as any Apple product I've laid hands on lately. Except that it then includes a double-sided sheet of instructions in the outer packaging to tell you how to get the damn thing out.

And so far, that seems like a pretty good summary of the device. There's a huge quantity of form here, but the function is lacking. The initial registration was made infuriating by the lag between hitting a key on the keyboard[1] and anything happening. I'm not talking about the understandable lag due to the latency of updates on e-ink screens - I'm talking about the seemingly non-deterministic time between tapping the screen and it indicating that I've pressed a key. The coverflow feature for books is better than selecting from a menu of items (e-ink lends itself badly to interactive displays), but slow and jerky. Worse, it's limited to B&N content. Anything you obtain elsewhere and then copy onto the device (which presents as USB mass storage) ends up in a separate menu without any coverflow. And, even more infuriatingly, you can't catagorise the files you copy on there. It's just one big list, sorted alphabetically by author (surname) and then title.

Once you're in a book, things aren't bad. It reformats the text every time you enter a book (no caching), but that takes much less time than my Sony did. The default font is very readable, even at small sizes. But there's clearly something horribly wrong - various epub files I have take up to 4 seconds to perform a page turn, which is way longer than the second or so my Sony took. There's no way to skip to a given page number, which seems like an insane oversight. And, though it's a minor point, the next/previous page buttons are the opposite way around to the Kindle or Sonys, and it's taking a while to get used to that.

It's a promising device. The hardware's clearly capable and the software is mostly there, with the features I'm really missing being ones that shouldn't be hard to implement[2]. But those features are pretty glaring, and right now they make this less functional than the Sony, let alone the Kindle. I'm also kind of surprised that it doesn't ship with any kind of cover at all. There's ample opportunity for physical trauma to turn one of these into a paperweight.

Side note: My nook managed to include the QA checklist slip, presumably by accident. The nook's internal manufacturing designation appears to be "X2", and mine was built on 2009/11/25. Which would seem optimistic for an intended shipping date in the US of 2009/11/30, which does support the idea that the shipping delay was due to some kind of delay in the hardware production.

[1] Presented on the LCD panel
[2] Of course, this being closed-source, I can't do so myself. Sigh.

John Goerzen: Being a Dad at Thanksgiving

Over Thanksgiving this year, we took a train trip to visit Terah s family in Indiana. Here s the story of the trip. The Train to Indiana Our last train trip (to Chicago) was in May, before Oliver was born. Jacob always has enjoyed the train, but this time wow. We told Jacob about the trip the day before we were to leave. Several times in the day, he d get a sly smile, and ask, What are we going to do? Terah or I would spell out the day s activities, and when we d get close to the part where I d wake him up at night and we d go to the train station, he d have a big smile. The big hour (2AM) arrived and Jacob woke up while Terah was putting his shoes on. I was there a moment later. Normally Jacob is terribly grumpy after waking up, even more so if somebody else woke him up. That day, I saw a very groggy-looking boy. I asked him, Jacob, are you ready to go to the train station? He answered, YEAH! while rubbing his eyes. Dad, let s go! We had Amtrak s family bedroom for the first time this trip, and it worked out great. Jacob actually slept, despite his extreme excitement. He d been talking about eating in the diner car for quite some time, and was really excited when breakfast time rolled around. We sat down, and he pressed his nose up against the window right away. He counted the freight trains the he saw, pointed out train tracks and crossing guard lights , noticed barns and trucks, and gave us frequent updates on how fast we were going. His voice was high-pitched from excitement, and sometimes we d have a running commentary we could barely interrupt, and other times he d sit there silently just soaking it all in. But he didn t eat. He didn t want to turn his head away from the window for a second to take a bite. I kept reminding him to take a bite, and finally, fearing he d be really hungry as soon as we went back to our room, fed him a few bites for the first time in ages. I don t believe he realized that happened. He just opened his mouth by reflex and once I had to tell him to close his mouth around the fork that was in it. It was much the same story for lunch, and for dinner on the train though less so for dinner since it was dark outside. At dinner, a man sitting across the aisle from us said, If you weren t getting your meal free [since we're in the sleeper], I d buy it for you. Your boys are amazingly well-behaved! I thanked him politely Jacob and Oliver both were doing well but didn t really think it was that unusual. Oliver and Jacob were both big hits on the train. They got smiles from so many people on board and in the stations. Several of the dining car staff seemed to linger at our table longer than elsewhere, looking at them (and especially Oliver). Oliver slept well on the train: img_2240.jpg And here are a couple of typical Jacob photos: img_2244.jpg img_2241.jpg In Indiana I ve got to start this out with one of the family gatherings at Terah s aunt s house. They have a large kids area in their basement, and a swingset with a slide outside. Jacob loved all of this, and spent hours playing with me. Sometimes if I d go upstairs with the adults, I d hear Jacob s voice from the basement a few minutes later: DAAAAAD!!!! Shall you come back downstairs? DAAAD! And, of course, it s hard to resist a 3-year-old that wants to play with me, so I inevitably would. We played air hockey Jacob shrieked with delight whenever anybody scored a goal in any way. At one point, he stood up, looked around, and said, Hmmm. Where are all the trains? Apparently he expected every home to be stocked with toy trains, and this one wasn t. A few seconds later, he was all excited. He ran over to a toy semi, and said, I will use the semi train! Outside the short slide had a ladder going up to it. I held Jacob s hands while he climbed up the first few times, but I knew he could do it himself. I started helping him less and less, and eventually refused to help him at all. He was rather frustrated with that for a moment, but a couple times later, climbed up and said, Dad, I got up here all by myself! A few times later, it looked like he was having trouble, so I tried to help, and got shooed away with, No, dad! I can do it. The best moment of the weekend came during the drive back to Terah s mom s house where we were staying. Terah was driving, and I heard Jacob say, Shall you come closer? Not sure how to do that from the front seat of the car, I reached my hand back there and he held it. He held one or the other of my hands for 20 minutes until he fell asleep in the car. It s funny what feeling needed by a 3-year-old makes a person feel, but that was surely a happy day. Jacob got to see all of his Indiana grandparents on this trip, and loved playing with them. He especially liked using a perfect red train (his words!) he drug home from the nursery at church on Sunday, and a child-sized tent in the basement. I m not sure how many times I heard Gampa Mike! Shall we go to the basement? that weekend. One point he wanted me to come down to see it and play with him, so I did. Then he decided we all needed to be down there, and told me I ll go pick up mom. I ll be right back. Stay downstairs, dad. Fearing that Terah wouldn t know what to do when Jacob attempted to pick her up , I tried to discreetly follow at a distance. That was no match for a 3-year-old. He looked back at me from the top of the stairs, pointed an accusing finger at me, lowered his voice, and said, Dad. Stay. Terah and I spent a day in South Bend and stayed at the Oliver Inn while Jacob and Oliver got some time to themselves with their grandparents. We had dinner at the beautiful Tippecanoe Place, just next door. The Trip Home We had a few hours in downtown Chicago on our way home. We went and ate at a small Indian restaurant. Jacob kept alternating between three things: This food is very spicy! I like spicy food. I need another drink! A little while later, Terah decided we should have some dessert. So she and Oliver stayed in the waiting area while Jacob and I went asporing (exploring) to find something. We walked a block or two to the nearest Dunkin Donuts, then to a Starbucks, and then back. Jacob carried the sack of donuts, guarding them so carefully that I was afraid he might smash them. (They were cream-filled so that wouldn t have been good.) He had a lot of fun exploring Chicago Union Station, and also enjoyed walking in Chicago (though not quite as much; it was all a little too loud for him.) In the train station, he loved the escalators. Up and down we went, whenever we wouldn t get in people s way. I had to eventually drag him away from them as I was getting a little self-conscious at all the people standing around watching us repeatedly go up and down the escalators. Jacob has found a 3-year-old way to express his feelings: he ll pretend to be a kitty, and say the kitty is sad or something like it. All day he had been saying the kitty is hungry. He hadn t had a big breakfast, but he had plenty of opportunity to eat at lunch, and a snack. We were a bit confused, but I think we finally figured it out on the train back to Kansas. We didn t have time to eat breakfast in the diner on the train from South Bend to Chicago, so Jacob missed out on the dining car then, and I think he was disappointed or confused about that. Once we had dinner in the dining car, he was happier. Since we ve been back home, he s been talking about our next train trip. He says which suitcases we ll take, that we ll eat in the diner, that it will be a night train we take, and generally is still very excited about it. It was a fun trip.

Andreas Schuldei: Dear Lazyweb: Datacenter in the US?

I look for a data center in the US for some servers with ~40kW cooling power. I plan to use those nice energy efficient double sided racks from SGI (formerly Rackable), so the place should allow that i bring my own racks. I don't need super-low latency to Europe, but rather to the USA and Canada. And it needs to be highly reliable, preferably tier 3 or better!

However, what is most important to me (living in Europe) is that the remote hands actually have a clue. At one place in London i got really lucky: They are able to patch up new servers, can install spare parts and mail broken stuff back to IBM, know about tcp/ip, link lights, switches and routers, debug bad hardware (where LOM is not good enough) and are generally really friendly and quick. As icing on the cake they can even spell debian! Good pricing would be nice, too, but less important then the clueful people.

Should you know such a place, please mail me!

9 December 2009

Asheesh Laroia: A big machine that nothing can stop

A couple of weeks ago, Lucas Nussbaum wrote about his experience at the Ubuntu Developer Summit. Two things stuck out at me: In the summer of 2004, my laptop (an iBook G4) ran Debian GNU/Linux on PowerPC. I tried out Ubuntu that autumn, and was very impressed. It's been five years since then. My desktop still runs Debian, and I'm proud to be a "two-distro" community member. At the same time, this juggernaut nature raises concern. When Ubuntu releases ship with significant flaws, I quietly sigh and wonder, Are we doing good service to the people coming to GNU/Linux for the first time and seeing Ubuntu? This bugginess has bit me a few times (even for upgrades between one release and the next), and it pushed a friend of mine to switch a lot of his computing to DragonflyBSD. Lucas points out that Ubuntu's done a fantastic job of becoming visible. A look at Google Trends shows that people search for "Ubuntu" about as much as they search for "Linux" at all. Richard Stallman complains when people refer to GNU/Linux as just "Linux." But today, the most popular name for an operating system based on GNU is probably "Ubuntu." For Stallman, having his hard work associated with someone else's "Linux" project must be frustrating. As a Debian contributor, it would be easy to succumb to the same feeling with regard to Ubuntu. Lucas is well aware of that. He asks, Debian does have users! I was feeling a bit disheartened about my maintenance work on alpine lately, when out of the blue I received an email from a fellow at MIT asking me if I would accept some help with maintenance from him. Just seeing the note was a relief; it's very nice to be reminded that I have users to take care of. Debian is not as popular as Ubuntu on the desktop (or laptop), it's true. But technically sharp people do frequently still choose it. For example, the FreeNAS project recently announced a switch from FreeBSD to Debian (and a new name as coreNAS). But we could end up a "package supermarket". To stave this off, we must do good releases and make sure people know about them. Doing all that work takes us all time. We must recruit new contributors, help them find things to work on, and make sure they feel welcome. And sometimes-sloppy maintainers like me ought to fix our packages, and get help from others where necessary. I promise to work toward that.

Joachim Breitner: Packaged unicode-screensaver properly

Three years ago, while I was in Ghana, I wrote a small screensaver that displays a randomly picked unicode character full-screen. I like it because there is a huge bunch of very weird characters and signs in the unicode standard. It also prints the unicode codepoint and character name.
Now I finally got around to properly package it: Created a small homepage, a source tarball with (hopefully) proper autotool files, all managed in a git repository (also on gitorious), complete with manpage, README and other additional files. I m also going to upload the package to Debian, so you can just do apt-get install unicode-screensaver as soon as it gets accepted.It can probably not enter the official xscreensaver distribution as it adds dependencies on the fontconfig and freetype libraries.

Stephan Peijnik: ujail: use cases, FAQs, part 1 & proof of concept, part 2

As I ran out of time whilst writing the "introducing ujail" post on monday I would like to further elaborate on the idea, giving you some examples of possible use cases and then having a look at FAQs regarding ujail. Additionally I have created a second proof of concept that should be a lot faster, see below for more details.

Use cases of ujail

Monday's post was rather technical, so let's have a look at possible use cases today.

The main reason for both having the idea of ujail and starting working on it is my web server. I am running quite a few (S)CGI scripts there and, even though running them as different users, on a per-vhost basis, I have the impression of the whole thing being a bit insecure.

Okay, PHP does provide its famous open_basedir feature, but I am also running some Python applications which I simply cannot restrict easily. My first ideas involved adding something similar to open_basedir to Python, followed by the idea of replacing some C library functions, like fopen and friends on startup time.

Whilst the adding open_basedir to Python would have involved changing a lot of Python's internals I soon discarded the library patching idea as those could be worked around by injected code directly invoking syscalls. It didn't take long for me to notice that I have to dig deeper. The idea of ujail was born and after coming up with the proof of concept this seems to be a viable solution.

Now ujail is not only about protecting a web server from its web applications, but could do a lot more, for example:

I am sure you can come up with even more use-cases. What should be noted is that emulating a system call does not mean that one necessarily needs to emulate the whole filesystem. What can be done, for example, is patching through access to common files (libraries, executables, etc.) whilst maintaining a virtual filesystem for data that will eventually be modified. A copy-on-write approach is possible too, for example. There are multiple methods with which the multiple filesystem could be implemented, the most common would probably be using a state directory.

FAQs

There have been some questions about ujail in comments to my first post which I would like to answer. Also, I have been thinking about things that are different about ujail compared to other virtualization techniques. Feel free to add additional questions either in a comment or drop me an email: debian at sp dot or dot at.

  1. Could you change the license of ujail to ... ?

    Not likely to happen. The proof of concept's license is GPLv3 and the actual code's license will be too. However, ujail is a userspace application that does not need any modifications to the kernel so there should be no problems with porting ujail from GNU/Linux to any other system.

  2. Does ujail work on operating systems other than GNU/Linux?

    Not yet. If it's technically possible to implement the technique on other operating systems I would be happy to accept patches.

  3. Do I need to patch my kernel for ujail to work?

    No, ujail is running in userspace. The only thing it needs is Linux with support for PTRACE_SYSEMU.

  4. How is this approach different from using LD_PRELOAD?

    With LD_PRELOAD one can replace library functions, but malicious code could still directly invoke syscalls, working around this protection completely. Also, statically linked binaries cannot be restricted with LD_PRELOAD.

  5. How is this approach different from user-mode-linux?

    User-mode-linux (UML) works by emulating a full kernel in userspace and allows you to virtualize a whole Linux instance (including a new init process, etc). ujail is about providing a way of restricting a single process (and its childs) inside a running system in terms of access to syscalls and the partial emulation of those.

  6. How is this approach different from linux-vserver?

    Linux-vserver is a kernel patch and runs in kernel space, as opposed to ujail, which works in userspace.
    Also, linux-vserver works similarly to user-mode-linux, providing a fully virtualized Linux instance.

  7. Does the account running ujail need any special privileges?

    No, the only restrictions that apply are those of ptrace.

  8. Where is the code?

    Right now ujail is in a planning phase, and only the proof of concept code has been written and published. The actual ujail code is yet to be written and the code will be hosted on launchpad.net.

Proof of concept, part 2

An anonymous person (who were you stranger?) added a comment to my first post, suggesting "Also, why patch the process rather than just modifying its state and trapping into the kernel?". I have had a look at this approach earlier, but it didn't work out. However, I decided to give it yet another try and created a second proof of concept. That code does not require patching any code, but only modifies the instruction pointer (eip) and the first register (eax). This should be a lot faster than patching the code.

Technically the new main loop works by calling PTRACE_SYSEMU and waiting for a notification. It then saves the instruction pointer and switches to PTRACE_SYSCALL. As before it waits for the emulated syscall to exit and at this point sets eax from orig_eax and decreases the value of the instruction pointer by the size of the "int $0x80" instruction. Another call to PTRACE_SYSCALL resumes the process. The next event is the process actually entering the real syscall and yet another one leaving the syscall again. These are resumed by PTRACE_SYSCALL and PTRACE_SYSEMU respectively. So, comparing this with the first approach we are only modifying two registers now, instead of writing to the TEXT area of the running process.

Thanks should go to the anonymous commenter for making me give this approach another try.

Questions? Criticism? More ideas? Want to contribute?

Coming to an end I would yet again like to let you know that I am open for questions, criticism, more ideas and contributions in general. So if you are interested in this topic come join the discussion by either dropping me an email, writing a comment to this post or replying to this post on your own blog.

8 December 2009

Christian Perrier: Bug #560000

Mika Tiainen reported the Debian bug #560000 on Tuesday December 8th, against the ferm package. Bug #550000 was reported as of October 7th 2009. So, again, no matter what, we're still keeping up with the very regular "10,000 bugs in 2 months" pace This time, I would like to highlight a specific bug that missed to be #560000 by very few hours: namely "#559980: aptitude: Totally broken on GNU/kFreeBSD". Please read that bug report and take time to send Cyril "KiBi" Brulebois a nice "Chapeau" ("hat off") for the time he invested in analyzing this bug and providing a good workaround. Update: KiBi told me that his workaround is broken. Still, the "hat off" still stands as what impressed me is more the nice investigation work than the workaround itself. Of course, admittedly, Cyril is not alone doing such work but it never hurts telling good people they are doing good work. To all Planet readers: <Teletubbies mode ON>from time to time, please remember putting more light on those people doing great and sometimes obscure work in our project</Teletubbies>. After all, blogs are not only about ranting, right? :)

7 December 2009

Martin Meredith: Everything but the Kitchen Sink

Transforming data is hard. When I joined my current company, there were stupendous amounts of Perl/PHP/Bash/<insert random programming language here> scripts that would run on a cron job and do magic things to our data. They d create reports, they d tell the purchasers when we were running out of stock, they d synchronise data between our Frontend and backend databases, they d collect, they d collate, they d do everything and anything. Except, with all these scripts, in all these random languages, written by a multitude of previous developers (at different skill levels), they weren t particularly maintainable (and sometimes, they weren t particularly readable or understandable either imagine a 6000 line perl script that pretty much ran different permutations of the same data over and over again) Enter Pentaho, and specifically it s Kettle project. (since renamed Pentaho Data Integration ), a tool that lets you manipulate your data in pretty much any way you can imagine, in the simplest and easiest way imaginable.

That s right, it s a GUI for data manipulation.

I know a lot of you are probably sceptical right now. The first time I ever saw this was when a previous boss of mine put it forward as a potential solution for one of our problems (getting our orders from the front end database down to the office/warehouse). I saw it, and I thought GUI? Nah, that s not how real programmers do things! , so after the development team put forward another proposal to solve this, and it got accepted, I thought I d never see the thing again.

That was until my current boss started playing with it, trying to work out what it was doing so that he could get these evil GUI based scripts into something manageable, like nice, pretty code. Thing s, when my boss plays with things that he doesn t know about, he tends to read up, research, and, 9 times out of 10, change his mind.

We wiped the previous server (it was rather noisy! We re glad it s no longer switched on!) and set up a new server to house our BI platform . Starting off with a few scripts, my boss learnt to love this tool, and then, as I m his 2nd in command (aka general lackey) started making me learn how to use it.

Again, I was sceptical, I didn t want to learn, and I put up resistance, but my boss was going away for nearly a month, and by this time, a few of our key business processes relied on Kettle, so, grudgingly, I sat down, and started to learn.

You may be wondering now, why I started off this story talking about all those magical and wonderful scripts that no one seemed to know the inner workings of. These scripts, as I ve already mentioned were unwieldy, and at times, god-damned awful. The plan was to move them to the BI system (as my boss had been doing already).

I like to think of Kettle as a bridge between the process-flow diagram, and the code. I started converting these scripts, and I was astonished by the fact that most of the conversions I was doing was converting a long perl script into 3 or 4 Integration steps

I m totally besotted with this program now. Any time I have to do data manipulation, I turn to it. I can t describe how (once you ve got used to it s quirks) easy it is to use, how simple it is, and how much it just makes sense. Best of all, most of those evil scripts are gone now, and replaced with pretty diagrams that do the work for you.

If you have to play with large data sets on a regular basis, I urge you to try it out. You can buy me a beer for reccommending it next time you see me at $conference.

Cyril Brulebois: RBCN

I really liked doing stuff like Zack s RCBW in the past, but nowadays I m busy with some buildds, reporting RC bugs because some maintainers don t test their uploads, as well as posting patches to fix FTBFS on GNU/kFreeBSD. But from time to time, it s also fun to close RC bugs. Here s a new concept (that I probably won t reproduce as regularly as Zack and his minions, though): RCBN for Release-Critical Bugs of the Night. Those are now closed: Many thanks to Peter Green, to Ruben Molina, and to Ilya Barygin for their patches. Special thanks to Richard Darst for orpie s patches: he even managed to get rid of embedded code copies while addressing the FTBFS. Nice! Unfortunately, in the same time, I had to open some FTBFS/RC bugs: So, what s next? Get back to work!

6 December 2009

Steve Kemp: I feel a hate crime coming on.

Recently I've been spidering the internet, merrily downloading content for the past few days. The intention behind the spidering is to record, in a database, the following pieces of information for each image it stumbles across: I was motivated by seeing an image upon a website and thinking "Hang on I've seen that before - but where?". Thus far I've got details of about 30,000 images and I can now find duplicates or answer the question "Does this image appear on the internet and if so where?". Obviously this is going to be foiled trivially via rotations, cropping, or even resizing. But I'm going to let the spider run for the next few days at least to see what interesting things the data can be used for. In other news I'm a little behind schedule but I'm going to be moving from Xen to KVM over the next week or ten days. My current plan is to setup the new host on Monday, move myself there that same day. Once that's been demonstrated to work I can move the other users over one by one, probably one a day. That will allow a little bit of freedom for people to choose their downtime window, and will ensure that its not an all-or-nothing thing. The new management system is pretty good, but I have the advantage here in that I've worked upon about four systems for driving KVM hosting. The system allows people to enable/disable VNC access, use the serial console, and either use one of a number of pre-cooked kernels or upload their own. (Hmmm security you say?) ObFilm: Chasing Amy

Next.

Previous.