Search Results: "zed"

10 January 2008

Christine Spang: MIT Athena: Not Dead Yet

(Disclaimer: I work for MIT Information Services & Technology blah blah blah I am a student blah blah these are only my words and do not represent the opinion(s) of my employer.) Way back in the day in 1983, MIT started something it called Project Athena, which had the goal of investigating how computers could be used to enhance education at MIT. This project became the origin of such pieces of software as the X Window System and Kerberos, as well as a campus-wide network of workstations all running similar software. In 1991, Project Athena ended, but the Athena system continued on, taken over by MIT Information Systems. It’s 2008, and Athena still exists at MIT—walk into a computing cluster, and chances are all the machines will be running Athena software. (Unless that particular cluster has been taken over be Athena’s evil sibling, often referenced as “WinAthena”, which often seems to be maintained by no one, and whose existence is sometimes denied by those who maintain and support Real Athena. Not many machines have suffered this agonizing fate, luckily.) Unfortunately, as the years have progressed recently, fewer and fewer resources have been put into the development of Athena. In many ways it is no longer on the cutting edge of things as it has been in the past, though it is certainly still useful. Some students these days, however, even think of Athena as only “their MIT email” or maybe use it occasionally to print something in a cluster on the way to class. I’ve seen people grow frustrated and storm out of a cluster due to things like instructions for using a flash drive that involve more steps than just plugging it in. While on the one hand it’s easy to laugh at people for a lack of patience or being open to a computing system that is not either Windows or Mac and perhaps a bit more complicated on the surface, on the other hand, some of these things really should just work by now, and on other UNIX-based systems they do—but Athena has lagged behind. When you log into an Athena workstation for the first time, you are confronted with what looks like this: Clean MIT Athena login That’s GNOME 2.8, and it looks kind of like the GNOME 2.8 default but uglier. When you look under the hood, there are pieces of the system that obviously originated in a time where nothing better existed and so something new had to be created—the “xlogin” login system doesn’t support PAM, and the login path involves a hairy maze of scripts that perform various tasks. Under the hood, the base OS is currently RHEL 4 or Solaris 10. I still can’t remember what the right options are to feed RPM whenever I need some information that I think I should be able to get from it. These things were once necessary but today just look crufty. Some people have tossed around the idea that everyone has a laptop these days and thus clusters are uselessly expensive, but it’s been met with a lot of resistance. I’m all for progress, but I use clusters all the time! They’re one thing that makes it an easy choice for me to have a really portable laptop, because when I really need computing power or screen space on campus away from my room, there is always a cluster nearby. They’re also nice isolated places that can be associated with “work mode”, which is good for productivity. Beyond all this, Athena provides a common platform that all technical classes can depend on, from providing standard and specialized academic software to standing as a baseline of “make sure your code works on this before handing it in because that is how it will be graded”. At any rate, things are finally happening on this front. Athena can’t stay RHEL 4 forever—while AFS allows non-system software to be easily updated for all machines without requiring any OS release, hardware moves on and compatibility is lost, and the rest of the world is coming up with new awesome things that Athena machines are missing out on. A student group that I am vaguely involved with, SIPB, took the early initiative with this and created DebAthena, which takes stock Debian and Ubuntu and allows it to closely resemble an Athena environment. We even run a campus Linux dialup using the packages. (I won’t claim to have done much work on DebAthena myself because, well, I didn’t.) Amazingly, the Athena release team has decided to work with the SIPB on the next release of Athena. So, what I really just wasted over 600 words prepping for is to say that Athena 10 will be based on Ubuntu. This is only relatively new news, but I am lame about finishing blog posts. There are a variety of reasons why Ubuntu was chosen over Debian. Some of them are lame and some of them are not. Even so, I am excited about this prospect—and it should be possible to peripherally support Debian in such a way that was not possible with Athena 9.4. I really like Athena. Even coming to MIT with a background in Linux there has been a steep learning curve, though, and I think that things could be better. Lots of people here have their first encounter with a ‘nix system in the form of Athena, and we should strive to make that experience as good as possible. I’m kind of skeptical on the expected release date of this upcoming summer, but as long as it happens soon it will be a big step forward. In some ways it’s kind of sad that I’m excited about a big part of a research institution catching up with others—but inertia is a strong force, and dealing with thousands of workstations is no trivial task. I kind of long for the exciting days of creating new big pieces of free software like X. I bet I imagine those days as being better than they actually were.

Alexander Schmehl: About Apache, Basic Authentication, Location and regular expressions

The other day, I had an interesting problem, regarding Apache, the LocationMatch directive and regular expressions. I'll put it here, so I'll be able to find it again... Actually it was a quite simple problem: A simple web page, where all web pages are protected with a password. No big Problem: Use a <location /> and AuthType basic, problem solved.... or isn't it? Some documents might be needed to be available without the authentication... for example robots.txt... or the funky custom error document, which explains how to get the authentication data, in case the authentication fails. Looking at the documentation, found a solution: LocationMatch exists, where you specify a regular expression, which must match, for the following configuration to apply. What isn't mentioned in the documentation, that the obvious solution... something like <LocationMatch !"^/(robots.txt my401handler.html"> doesn't work. See apache bug 10932. Needed quite some time, to find this out... Luckily Meike found the bug report above, which has a workaround for that problem: Use <LocationMatch "^/(?!robots.txt my401handler.html)"> Oh... and if your customized error handler references some pictures or style sheets, don't forget to add them to the regular expression above, too.

7 January 2008

David Nusinow: Finally

A few years back I asked Branden Robinson for access to the X Strike Force SVN repository in order to improve the X server's autodetection, with the goal of stealing what I could from knoppix. At the time, users were constantly wandering in to #debian saying that they had used knoppix to create an XF86Config-4 and then they used that file on their Debian installations. They were also constantly whining about how Debian wasn't doing as good a job. So I decided to do something about it, and took a look at what knoppix was actually doing so much better than us, and I was surprised to find that they pretty much just wrote a skeleton configuration file and let the X server fill in the details. I had no idea the server was capable of such things. So Branden graciously gave me SVN access and I began comparing the knoppix method to the script called by "dpkg-reconfigure xserver-xfree86" and realized that we probably couldn't adopt the same method because we had tons of checks for portability. So I put the problem on the backburner for few weeks and worked on something else for a while.

Well, somewhere in between then and now I got sucked in to transitioning Debian to X.org (using the aforementioned SVN access) and then working on all the things that went along with maintaining X in Debian, some well and others less so. Between transitioning to Xorg, and then transitioning to a modular Xorg, even with the ability to steal Ubuntu's packages it still took about two full years with me being a rookie and all that. There were tons of people who worked on this with me, but it was just a damn big job. Eventually though, etch released with modular X packages, and we were running at a pretty good pace with upstream, so it was time to revisit the problem of configuration again after the two year detour.

The problem of configuration had been reframed for me by having looked at what knoppix was doing and discussions with upstream. Upstream was starting to come to the conclusion that we shouldn't have a config file at all, and that the server should be smart enough to do everything by itself. It was already partially there, it just needed a push in the right direction. This was a major shift in how I thought of it. Coming from a Debian background, where the answer is to always just regenerate or edit your config file, having the server work things out for you was a totally alien idea. But I have my deepest roots in the Macintosh world, so immediately fell in love with it. The problem was that the server had a whole body of code to use if you had no config file at all, and another, with far less automagic goodness, if you had a config file, even if it was a 0 byte file. The goal became to have the server work really well with a minimal config file, so you could override what you don't like in the defaults and let the server figure things out for itself at boot. The way forward was to translate as much of the logic in our configure script in to the X server itself as possible.

Ubuntu had put in a lot of work in to the configuration setup that we automatically benefitted from, so given that most users were happy with things as they were, I was able to carve away at the problem without disrupting anything. And there was a lot to carve out, as the script that runs the config is an absolute mess that was slated by Branden for a rewrite all those years ago. Early on I picked off the low hanging fruits like the font path and modules. Similarly, Redhat's Adam Jackson had also been working on this problem, and he killed off the ServerLayout section as well as putting in lots of critical fixes all over the place elsewhere. More recently, I've gone and cut explicit modesetting out of the configure script. Hardcoding this information is generally a bad idea in the randr 1.2 world, and most of the drivers will do as good or better job of figuring out the modes than our config script would. This let us jettison using the xresprobe program to ask the monitor for the settings to use. This cut out a lot of the code that we had to deal with, simplifying the configure script and letting us all benefit from upstream's work. This leaves a big gaping hole in user configuration, which is something I'm looking to address in a few weeks, but for now there are workarounds like editing your xorg.conf manually to make things work.

Finally, yesterday I was able to upload a version of the script that no longer uses the discover program to figure out what driver to load. I've patched the server to do this at runtime. If there's no driver listed it simply scans the PCI bus, picks out your primary video card, and loads the first driver that claims to support that PCI ID. This let us jettison the last external dependency that the configure script had, so now we have a relatively small chunk of shell script with no external C code and a simplified setup. At this point we're now shipping a more skeletal config file than knoppix ever did simply because it wasn't possible to ship something like this before and have the server work at all. This is a huge milestone for me because finally, after over three years, the problem I originally set out to work on with X is done. There's still bugs to uncover and fix with all of this, but I'm convinced that what we have now is superior to our old method. Eventually, with any luck, xorg.conf will just fade away for most people. There's a lot to work on before that happens though, but I'm happy to have finally gotten here.

Andrew Pollock: [life] Post-op, Day 4

Sarah slept better overnight, but still not great. She was certainly more lively this morning when I got in, than yesterday. She's resigned to the fact that you don't come to a hospital to get rest, which is kind of ironic. She did need to get put back on the oxygen overnight because her O2 saturation dropped. She was off it again by the time I got in. Today's big achievement was getting the external pacing wires removed. They were put in during the procedure as a precaution, I think because if her heart played up, they didn't want to have to use defibrillator paddles, and because they'd cracked her chest open, manual CPR wasn't an option. The wires could be attached to an external pacemaker if necessary (but it never was). I watched the removal quite closely, as I was pretty curious. Basically the physician's assistant just came in and yanked them out. Judging from the look on Sarah's face, it was an interesting sensation. We were both amazed at how long the wires were. There was probably 30 centimetres of wire under the skin. The last bit was loosely spiralled. I was just really surprised, because the PA's hand started off at Sarah's chest, and ended up at the PA's shoulder. It was just a lot longer than we both expected. All the extra length was because the wires wrap around the heart. The wires came out at about 2:30pm. Sarah had another chest x-ray and a CT scan today. The CT scan looked good. The x-ray showed some fluid in her lungs, and her lung capacity is still sub-par. It seems when she's fully horizontal, her breathing isn't so crash hot, but it's fine when she's sitting up or walking around. All Sarah's got left is one peripheral line now. There was some talk of her possibly going home on Wednesday. She has an echocardiogram tomorrow, and I guess it'll all hinge on how her lungs are going.

3 January 2008

Martin-&#201;ric Racine: Hunting for a new laptop (part 2)

Earlier, I had reported on my quest for a new laptop. To recap, I had narrowed down my choice to an LG model whose 12.1" 1280x800 WXGA display and ATI Mobility Radeon X1250 graphic chip failed to be recognized by the Ubuntu CD. Back then, my initial tests were performed using a Feisty CD. Since then, I have returned with a Gutsy CD and got equally disappointing results: E: screen found, but none offers a usable configuration. Some people suggested buying a laptop with an all-Intel chipset. While Intel chipsets are indeed the best-supported in the business, largely because Intel invests its own resources into coding Linux drivers for all of its products, products based on Intel chipsets tend to be double the price of those based on VIA motherboards with ATI graphic chips. Seeing how quickly hardware devaluates on the market, it feels rather unjustified to pay such a fortune for an Intel-based product, only to see its market value cut in half less than a year later. Here's a tip for AMD/ATI marketroids: invest as much resources into coding GPL kernel and X.org drivers for all of your products as you did for the OLPC project and always do so a good 12 months before entry to market. Otherwise, prepare for bankruptcy, because companies like Intel who don't shun the Linux market will eat you alive. Me goes back laptop hunting...

Ond&#345;ej &#268;ert&iacute;k: SymPy/sympycore (pure Python) up to 5x faster than Maxima (future of Sage.calculus?)

According to this test, sympycore is from 2.5x to 5x faster than Maxima. This is an absolutely fantastic result and also a perfect certificate for Python in scientific computing. Considering that we compare pure Python to LISP.

Ok, this made us excited, so we dugg deeper and ran more benchmarks. But first, let me say a few general remarks. I want a fast CAS (Computer Algebra System) in Python. General CAS, that people use, that is useful, that is easily extensible (!), that is not missing anything, that is comparable to Mathematica and Maple -- and most importantly -- I want it now and I don't care about 30 years horizons (I won't be able to do any serious programming in 30 years anyway). All right. How to do that? Well, many people tried... And failed. The only opensource CAS system, that has any chance of becoming the opensource CAS, in my own opinion, is Sage. You can read more about my impressions form Sage here. I am actually only interested in mathematical physics, so basically Sage.calculus. Currently Sage uses Maxima, because Maxima is old, proven, working system and it's reasonably fast and quite reliable, but written in LISP. Some people like LISP. I don't and I find it extremely difficult to extend Maxima. Also even though Maxima is in LISP, it uses it's own language for interacting with the user (well, that's not the way). I like python, so I want to use Python. Sage has written Python wrappers to Maxima, so Sage can do almost everything that Maxima can, plus many other things. Now. But the Sage.calculus has issues.

First, I don't know how to extend the wrappers with some new things, see my post in the sage-devel for details, it's almost 2 months old with no reaction, which shows that it's a difficult issue (or nonsense:)).

And second, it's slow. For some examples that Sage users have found out, even SymPy, as it is now, is 7x faster than Sage and sympycore 23x faster and with the recent speed improvements 40x faster than Sage.

So let's improve Sage.calculus. How? Well, no one knows for sure, but
I believe in my original idea of pure Python CAS (SymPy), possibly with some parts rewritten in C. Fortunately, quite a lot of us believe that this is the way.

What is this sympycore thing? In sympy, we wanted to have something now, instead of tomorrow, so we were adding a lot of features, not looking too much on speed. But then Pearu Peterson came and said, guys, we need speed too. So he rewrote the core (resulting in 10x to 100x speedup) and we moved to the new core. But first, the speed isn't sufficient, and second it destabilized SymPy a lot (there are still some problems with caching and assumptions half a year later). So with the next package of speed improvements, we decided to either port them to the current sympy, or wait until the new core stabilizes enough. So the new new core is called sympycore now, currently it only has the very basic arithmetics (and derivatives and simple integrals), but it's very fast. It's mainly done by Pearu. But for example the latest speed improvement using sexpressions was invented by Fredrik Johansson, another SymPy developer and the author of mpmath.

OK, let's go back to the benchmarks. First thing we realized is that Pearu was using CLISP 2.41 (2006-10-13) and compiled Maxima by hand in the above timings, but when I tried Maxima in Debian (which is compiled with GNU Common Lisp (GCL) GCL 2.6.8), I got different results, Maxima did beat sympycore.

SymPyCore:

In [5]: %time e=((x+y+z)**100).expand()
CPU times: user 0.57 s, sys: 0.00 s, total: 0.57 s
Wall time: 0.57

In [6]: %time e=((x+y+z)**20 * (y+x)**19).expand()
CPU times: user 0.25 s, sys: 0.00 s, total: 0.25 s
Wall time: 0.25

Maxima:

(%i7) t0:elapsed_real_time ()$ expand ((x+y+z)^100)$ elapsed_real_time ()-t0;
(%o9) 0.41
(%i16) t0:elapsed_real_time ()$ expand ((x + y+z)^20*(x+z)^19)$ elapsed_real_time ()-t0;
(%o18) 0.080000000000005


So when expanding, Maxima is comparable to sympycore (0.41 vs 0.57), but for general arithmetics, Maxima is 3.5x faster. We also compared GiNaC (resp. swiginac):

>>> %time e=((x+y+z)**20 * (y+x)**19).expand()
CPU times: user 0.03 s, sys: 0.00 s, total: 0.03 s
Wall time: 0.03


Then we compared just the (x+y+z)**200:

sympycore:
>>> %time e=((x+y+z)**200).expand()
CPU times: user 1.80 s, sys: 0.06 s, total: 1.86 s
Wall time: 1.92
swiginac:
>>> %time e=((x+y+z)**200).expand()
CPU times: user 0.52 s, sys: 0.02 s, total: 0.53 s
maxima:
(%i41) t0:elapsed_real_time ()$ expand ((x + y+z)^200)$ elapsed_real_time ()-t0;
(%o43) 2.220000000000027


Where GiNaC still wins, but sympycore beats Maxima, but the timings really depend on the algorithm used, sympycore uses Millers algorithm which is the most efficient.

So then we tried a fair comparison: compare expanding x * y where x and y are expanded powers (to make more terms):

sympycore:
>>> from sympy import *
>>> x,y,z=map(Symbol,'xyz')
>>> xx=((x+y+z)**20).expand()
>>> yy=((x+y+z)**21).expand()
>>> %time e=(xx*yy).expand()
CPU times: user 2.21 s, sys: 0.10 s, total: 2.32 s
Wall time: 2.31
swiginac:
>>> xx=((x+y+z)**20).expand()
>>> yy=((x+y+z)**21).expand()
>>> %time e=(xx*yy).expand()
CPU times: user 0.30 s, sys: 0.00 s, total: 0.30 s
Wall time: 0.30
maxima:
(%i44) xx:expand((x+y+z)^20)$
(%i45) yy:expand((x+y+z)^21)$
(%i46) t0:elapsed_real_time ()$ expand (xx*yy)$ elapsed_real_time ()-t0;
(%o48) 0.57999999999993

So, sympycore is 7x slower than swiginac and 3x slower than maxima. We are still using pure Python, so that's very promising.

When using sexpr functions directly then 3*(a*x+..) is 4-5x faster than Maxima in Debian/Ubuntu. So, the headline of this post is justified. :)

Conclusion

Let's build the car. Sage has the most features and it is the most complete car. It has issues, some wheels need to be improved (Sage.calculus). Let's change them then. Maybe SymPy could be the new wheel, maybe not, we'll see. SymPy is quite a reasonable car for calculus (it has plotting, it has exports to latex, nice, simple but powerfull command line with ipython and all those bells and whistles and it can also be used as a regular python library). But it also has issues, one wheel should be improved. That's the sympycore project.

All those smaller and smaller wheels show, that this is indeed the way to go, but very important thing is to put them back in the car. I.e. sympycore back to sympy and sympy back to Sage and integrate them well. While also leaving them as separate modules, so that users, that only need one particular wheel, can use them.

David Welton: No next big language?

Ola Bini, one of the JRuby hackers, and a very bright guy, posits that there "won't be a next big language": http://ola-bini.blogspot.com/2008/01/language-explorations.html
There might be some that are more popular than others, but the way development will happen will be much more divided into using different languages in the same project, where the different languages are suited for different things. This is the whole Polyglot idea.
I'm dubious, and wonder what he would consider to be the underlying sociological and economic factors driving this change. Programming languages are, in the end, about people in all their weirdness, so to understand where languages are going to go, you have to consider those human factors, as I've attempted to do here. One trend that points to a slow proliferation of languages is of course the lock-in cited in my article. Today's big languages (Java, and on the web, PHP) won't just go away from one day to the next, just as C, Cobol and Fortran have not disappeared with the advent of Java. That process will continue, making it likely that new languages will carve out new territory for themselves rather than exclusively cannibalizing existing installations from older languages. This naturally leads (slowly) to more languages, even if the next generation has a Next Big Language. And why shouldn't there be one? Ola talks about a situation where various languages run and interact on top of a runtime (JVM). Isn't that similar to what we've had with C, though? Perl, Tcl, Python, etc... all run on top of C. Sure, the JVM is a step up from that in some ways, most notably GC, being a bit more consistently cross-platform, and having a wider array of libraries, but in the end, it still comes down to the network effects of being able to read and write a common language, whatever it runs on. Obviously, the network externalities of programming languages are not so strong that they hit a tipping point after which one language crushes all the others, but they are strong enough to consolidate leadership in one or a few languages. Programmatically, Jython, JRuby, (and Hecl?:-) may even find it easier to interact on the same platform, but the humans writing the code will still push for consistency and the minimum set of common tools in order to aid the sharing and review of code. Another way to look at it might be from an organizational point of view. Today's biggest fish in the pond, Google, only allows four languages for their production systems. What would a "no big language future" organization look like? I can't believe they'd welcome a big hodge podge of things. In conclusion, as computers are ever more widely used, it's certain that more languages will be utilized. However, it's also likely that from time to time a few languages, with one in the lead, will emerge as the leaders.

Stefano Zacchiroli: happy newS year

happy newS year So this was Christmas, new year, et al. Too bad Murphy struck again on the first day of my (theoretical) vacation, delivering me a nice flu (not yet completely absorbed). Nevertheless I managed to do something funny in the last few days:
  • I remembered that I do have, as all other DDs, a personal lintian page and I have thus spent some time fixing warning ignored since ever (almost all uploads are still in delayed queues however); while I was at it I also decided to "destroy" my lintian page moving my name to Uploaders for packages that still weren't co-maintained. (/me eager to see this implemented, go Russ, go!)
  • I have fixed the vcs field stats page to consider as equal Vcs-Svn fields pointing to URLs like svn://svn.debian.org/foo and svn://svn.debian.org/svn/foo which are indeed the same final path on alioth. That's why in the graph you now see an inversion in the relative positions of field with wrong value and correct field, in case you were wondering. Thanks to Sandro Tosi for the hint.
  • I have finally managed to leave the 18th century my mail set-up was living in: goodbye pop + fetchmail + mutt, welcome imap4 + offlineimap + mutt. In the former setup I was unable to read my old mails while away from my laptop, now I have decent replicas of my full mailboxes in various places. I'm still a bit unhappy about how can I force a send/receive email cycle: with fetchmail I used to have a demonized fetchmail running in the background and just executing fetchmail I was able to force that. Now, with offineimap, I've an autorefresh-ed offlineimap running inside screen and I need to bring up the screen and then hit 1 to trigger a sync cycle.
... now back to that paper introduction due for December 26th :-/

2 January 2008

Erich Schubert: Earth and the sun

When I first read this comic strip [sinfest], I was about to complain "Dude, you know that the earth doesn't go round the sun once a day".Then I realized that this was the 1.1.2008 strip, and well, I guess once a year it is more appropriate.Thats what you get for reading comic strips in the morning (and mind you, it's a comic strip showing a sunrise, with no hint about it being an annual and not a daily thing)So, did you pat earth for completing another lap around the sun yet? :-)Happy new year, everybody!

29 December 2007

Joerg Jaspert: SSH &#8220;triggers&#8221;

Following my post about SSH jumphost I have another SSH feature to share with you, which IMO isn’t known or used enough. Imagine you have some data on one host which gets regenerated every now and then and you also need this data on other hosts too. There are multiple ways to sync the data to other machines. You can
  • make them available via http/ftp and cron a wget run
  • use some rsync or ftp daemon on the target host(s) and write data to them (brrrrr)
  • simply use limited SSH keys to signal the target host(s) when there is something new to fetch
The problem of the first option is that you need to make the data available via http/ftp, which needs an extra daemon to run. And then you need to cron a script fetching it, which wastes resources if the data is updated infrequently. The problem of the second option is that you need some daemon available which allows you to write to the host. Depending on what you chose that might be with passwords transferred unencrypted. Or you aren’t able to set it up yourself, as you are not the admin of the machine. So, lets take the third solution. Its safe to assume that SSH is available on the hosts you login to (don’t tell me you still use telnet/rlogin on remote machines), so lets use SSH features. I assume that usage of basic SSH public key authentication is known, if not you first want to google for it and learn about that. Using SSH keys to trigger actions on remote hosts is basically the same as a login to the machine and manually running a command - except you don’t need to do it yourself, a script is doing it. Which leads to one big difference for a “trigger” key compared to a “login” key: You do not set a passphrase for that SSH key! As a consequence of that you use such a key only for exactly one purpose, nothing else. Generate a new key for another trigger. Never, ever, use such a passphrase-less SSH key for normal logins to other machines… Having generated a passphrase-less key on host a (let it have IP 1.2.3.4) we copy the .pub file over to the target host b and put it into the authorized_keys file (usually in ~/.ssh/). The difference to normal login keys is the large line of options we put in front of it:
command="rsync --server -vlHogDtprz --delete --delete-after --ignore-errors . /org/backup/something",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,from="1.2.3.4" ssh-key-here
or
command="/var/mirror/ftpsync &",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,from="1.2.3.4" ssh-key-here
Both commands allow to use the SSH key from host a with its IP 1.2.3.4 and will run the command thats defined here. No matter what command or options are are specified on the commandline on host a, host b will only do the above. There wont be any port, X11 or agent forwarding and due to the forced command there also wont ever open a shell. With the first command we trigger the remote hosts rsync to write the data we send into that path, using the given options. Of course that means we send data using rsync from our host. The SSH connection stays available for the whole time the command runs. The second command instead runs the given binary, a shell script in this case, in background. As we also forbid tty allocation the SSH connection is closed right after the script is started. To avoid DOSing the remote machine such a script should have some kind of lockfile check and abort if a second copy is already running. SSH also allows to use such limited SSH keys for the root user, while disallowing normal root logins: Change PermitRootLogin in sshd_config from no to forced-commands-only and you wont be able to login to the remote host as root but you can trigger actions that should be run as root. If you want to know what else you can do using options specified in the authorized_keys file then read its man-page, it has some details how you can use it to limit port forward requests, use more than just one IP in the from= parameter, … Some users of the above way of automating tasks on remote machines are
  • The Debian mirror network. About all tier-1/2 mirrors are pushed using the described technique, making updates to Debian available as soon as possible.
  • da-backup, the Debian Admin Backup tool, used by Debian and DebConf to backup their hosts to central backup servers.
  • packages.debian.org updates his data way faster than in the past, thanks to this.
  • Possibly many many more, its a useful feature.

27 December 2007

Stefano Zacchiroli: xmas gift new homepage

my christmas gift: a new homepage Funnily enough, while Dato was migrating his homepage to ikiwiki, I was doing the same as my own personal Xmas gift to myself. I moved from the bononia.it domain to upsilon.cc (a virtual host machine on slicehost.com), where you can now see my shiny new homepage. The migration was easy, though quite boring, especially the part of adding redirects. Indeed, I have set up almost all redirects, including the blog feed, but at the point where I've reorganized my initial blog posts (which weren't following the YEAR/MONTH/NAME scheme) I gave up in setting up the redirects for that. That means that my ancient blog posts (including some about the VCS fields) will have some 404 on Google for some time: that's life, that's the Web. I now also have a more specific feed for Planet Debian on my blog, and playing with ikiwiki's show parameter for blogging I'll do my best to avoid flooding. Follows a big fat warning post to inform people about the blog new feed ...

Benjamin Mako Hill: Japan Trip

I'm going to be visiting Mika's family in Tokyo for the next two and half a weeks. We're planning a trip to Kansai at some point as well. Currently, I've got no free software related plans or meetings lined up but if folks in Kyoto, Osaka, or the greater Tokyo area would like to meet up for drinks, that would be great. Additionally, I'd be happy to put together something more organized (e.g., a short talk, workshop, etc). If you'll be around and are interested in either, contact me and let's figure something out.

25 December 2007

Ben Armstrong: Fr-eee-dom: roaming on the bus with wpa_supplicant

As I wrote earlier, I bought an Eee PC to use on the bus. Initially, I only used it offline to do some Debian packaging and blogging, but I soon discovered that I could do a few things online on the many open networks on my daily commute. While the connections are normally brief in good traffic conditions, even a few seconds here and there is enough to participate in irc discussions. Manually connecting to each got old very quickly. So, to automate those connections I settled on wpa_supplicant, (wpasupplicant is the Debian package name,) which is quite easy to set up in roaming mode, as outlined in /usr/share/doc/wpasupplicant/README.modes.gz. All you need is the following in /etc/network/interfaces:
allow-hotplug ath0
iface ath0 inet manual
        wpa-driver wext
        wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
Then create your wpa_supplicant.conf as follows:
ctrl_interface=/var/run/wpa_supplicant
network= 
     key_mgmt=NONE
 
That’s it. You can now roam from one open network to the next and the supplicant will connect you to each one in turn. I have used this to reconnect to my screen session each time the bus slows down for a stop or gets stuck in traffic to carry on conversations on irc for the whole length of the commute. If you also connect to networks requiring authentication, you’ll need to add a “network” clause for each network. See /usr/share/doc/wpasupplicant/examples/ for help with that, particularly the annotated conf file, README.wpa_supplicant.conf.gz. For finer control over the process, I use sudo wpa_cli so i can:
  • keep an eye on what networks are around: scan, then scan_results,
  • disconnect from a network once I know it’s out of range
  • and reassociate as needed.
The only things that I haven’t figured out how to do yet are to make the process of switching from one network to the next a bit faster (it normally takes about a minute; precious seconds of online time are lost when the bus is in motion,) and to blacklist certain essids, e.g. commercial hotspots that make you pay before they’ll route your packets to the Internet. For the first problem, I have already tweaked my dhcp settings, which helps a bit. For the second, I tried adding network blocks for specific networks and setting them to a lower priority, but that doesn’t work because then the ‘catch all’ open network block kicks in and picks them up anyway.

24 December 2007

Manoj Srivastava: Manoj: The Choice of the cat

The second in the Vampire Earth series by E. E. Knight, this book has our protagonist moving to the next step in the genetically augmented hunter clans of the human resistance. Well written, though the next stage of the character development would require the people to actually get into some relationships. Apart from that, the series is moving well along, with other team members being brought into the fold.

Erich Schubert: Spamassassin precompiled rules

Spamassassin supposedly can precompile rules to allow faster operation. It compiles an optimized matching automaton that will process all the regular expressions in parallel.Anyway, in order to use it, you need to manuall install some dependencies. They probably can't be introduced as real dependencies to not enforce them upon people who don't want to use this feature. So far, I've identified
aptitude install re2c gcc libc6-dev make
I'm aware that many people will have the libc6-dev and gcc stuff already installed, and the re2c dependency is well-documented. But I actually had removed the C compiler from my mail server.

Adam Rosi-Kessel: Jason s Lesson s Learned About the Legal Academy and Getting In

Pelican has decided to abandon his efforts to scale the walls of the legal academy and proposes this simple eight-step program for others who would follow in his footsteps:
  1. Don t do interdisciplinary work. The legal academy doesn t know what to make of it unless it is economics.
  2. Don t go to a school without a law review or grades. I did and it was a huge problem.
  3. Get on law review, clerk, write.
  4. Check the faculty listings at most law schools. Go to the most often listed schools: Harvard, Yale, etc. It does matter as law faculty select their own, usually. I was told by a faculty member as a 1L expressing interest in the academy that I should transfer immediately to Harvard if possible. I didn t.
  5. If you think you want to be a legal academic, look at what is on the FAR form in your first or second year. Orient your academic career to produce a good looking FAR.
  6. Remarkably, the legal academy does not care about your ability to raise research money or bring in grants.
  7. Don t publish in interdisciplinary journals. Publish only in law reviews writing only dense and impenetrable texts.
  8. Demonstrated impact of your work in policy or law is not relevant.
This critique could be extended broadly to much of the academy (not just law schools). I had once thought I might like to teach law, but now I can only really see myself as a clinician. Two contradictory academic trends:
  • The tendency toward dense, impractical writing that only makes sense to other insiders. This is often but not exclusively the influence of postmodernism. Steve recently brought Chomsky’s response to postmodern thought to my attention. It boils down to this: sure, it might be complicated. But there are lots of other fields that are complicated–quantum mechanics, nanotechnology, even game theory–in which an expert can still explain to a bright layperson the contours of the field sufficient for the layperson to get the gist of it, even if they can’t grasp all the details without extensive study. This doesn’t seem to be true of postmodern philosophy. Is the emperor naked? Paul Graham’s How to Do Philosophy reflects a similar attitude with less of a political bent, although my friends pursuing philosophy doctorates claim Graham has got it all wrong.
  • The tendency to reduce all disciplines to a science, particular the humanities and social sciences. This can either mean approaching the study with some method that approaches “scientific rigor” or literally basing the analysis in an already recognized science, preferably neuroscience. While I can see some sense in incorporating neuroscience, cognitive science, and certainly psychology into fields such as economics, I don’t think this is a good thing to the extent it overwhelms other modes of analysis, particular for literature. Neuroscientist/author Jonah Lehrer discussed this phenomenon (and its drawbacks) with Christopher Lydon in an episode of the newly-revived Radio Open Source. (Warning to Debian readers: Radio Open Source has very little to do with open source software.)
Technorati Tags: , , , , , , , ,

23 December 2007

Adam Rosi-Kessel: Media for Kids


One great thing about kids — especially kids who have not been overexposed to TV media — is how easily entertained they are. We recently showed Esther some Looney Tunes cartoons from the 1940’s (on a real TV, no less!) and she couldn’t stop laughing. It is refreshing to see someone spellbound by animated content that didn’t require a rendering farm to create. I’m sure she’ll be jaded soon enough, but we’re trying to drag it out as long as possible by providing only very occasional small doses. This piece on the end of reading as a cultural activity is chock full of interesting statistics. This excerpt, in particular, is relevant here:
In August, scientists at the University of Washington revealed that babies aged between eight and sixteen months know on average six to eight fewer words for every hour of baby DVDs and videos they watch daily. A 2005 study in Northern California found that a television in the bedroom lowered the standardized-test scores of third graders. And the conflict continues throughout a child s development. In 2001, after analyzing data on more than a million students around the world, the researcher Micha Razel found little room for doubt that television worsened performance in reading, science, and math. The relationship wasn t a straight line but an inverted check mark : a small amount of television seemed to benefit children; more hurt. For nine-year-olds, the optimum was two hours a day; for seventeen-year-olds, half an hour. Razel guessed that the younger children were watching educational shows, and, indeed, researchers have shown that a five-year-old boy who watches Sesame Street is likely to have higher grades even in high school. Razel noted, however, that fifty-five per cent of students were exceeding their optimal viewing time by three hours a day, thereby lowering their academic achievement by roughly one grade level.
It certainly doesn’t surprise me that television–even educational TV–is generally not good for kids. The “inverted check mark” bit was a surprise, though. Who would guess that two hours a day is optimal for the nine-year-old brain? On a related note, Kiddie Records Weekly (recommended by about.com here) is an astonishingly extensive source of free children’s books in MP3 format. It’s also an excellent example of the increasing relevance of bittorent in enabling wide economical distribution of legitimate content. The content is primarily (exclusively?) digitized vinyl records from the 1930’s and 1940’s. Fortunately, Alice in Wonderland gets no worse with age. To complete the nostalgia circuit, who can forget “loaf of bread, a container of milk, and a stick of butter” (via sketchelement, quite a while ago): These are good times to be a kid. Or maybe just to be a parent. Technorati Tags: , , , , , ,

22 December 2007

Manoj Srivastava: Manoj: The way of the wolf

This is a SCi-Fi series from E. E. Knight. It is not often that one comes across a brand new series in fantasy or science fiction; and more rarely still when it has the quality of this one (The Vampire Earth series). This is a post-apocolaptic novel, the apocalypse being a virus that killed most of the human population, unleashed by a gate-travelling extra-solar species to disrupt human resistance as they took over. Ostensibly about vampires, it provides an interesting back story to explain master vampires and their reaper thralls. What was captivating about this book is the detailed and generally coherent world building, the swaths of land under outsider control, where there is law and order and culling of humans for food; and the rag tag resistance. The characters are fairly well developed (though the author shies away from romantic relationships of any kind). Not since the Recluse novels have I felt this way about a new series.

Rodrigo Gallardo: What other Debian planets are there?

Liferea’s upstream has decided to stop puting Debian Planet as a default example feed for new installs. I agree with that change for the upstream distribution, but I’m still going to revert it for the Debian package. And, of course, that gives us the chance to do even better. I’ve added Planeta Debian to the Spanish default feed list and Planet Debian-FR to the French one. I wanted to add Debian Brasil to pt_BR, but their feed returns a 404. So, do you know about Debian planets in other languages? Please open a wishlist bug against liferea with the name, URL, the URL of the feed, the language it’s in, as well as some way I can verify the site is “official”, where that means it’s mostly published to by developers and heavy users. The languages which currently have a customized list are bg, es, nl, pt_BR, sk, ca, eu, pt, sv, de, fr, pl, ru and tr. If your language is not one of those, you’d do better to start by contributing a non-Debian specific feed list for your language, directly to upstream (and maybe translating the whole program, while you’re at it ;) ).

20 December 2007

Simon Richter: The Debian package namespace

A Debian package is uniquely identified by its name and version; there is no further information taken into account by dpkg. This keeps the packaging system rather simple, but has the disadvantage that all information that would otherwise be placed in extra data fields has to be encoded into these two fields. As the names of all binary packages generated from a single source package need to be known when building the .dsc file, no substitutions are allowed in the package name. Back in the Good Old Days, the only extra bit of information in the name field used to be the SONAME from the library in the package (thus implicitly saying that each library should have its own package), which meant that the maintainer had to adapt the debian/control file every time a library's ABI changed. This being the Good Old Days, people writing libraries actually thought about things, so ABI changes were few and far between. These days, libraries are stacked on top of each other, re-exporting the underlying libraries' ABI. The SONAME system could in theory handle these situations if the SONAME of every library whose ABI is re-exported is encoded into the SONAME of the newly built library. So if my own ABI version is "2", and some of the functions I export use types defined in glib, then after building against glib1.2, my SONAME would be somewhat similar to "2+glib1.2". For languages that don't have a well-defined ABI (i.e. anything but C), the version of the language ABI(s) used will also have to be added. Obviously, this would also have to be encoded into the package name, which is where trouble starts during transitions. The last C++ ABI transition had all C++ library maintainers add a suffix to the package name; dependent ABIs are not handled at all in Debian right now (which led to the libpng mess). The existing encoding rules are not entirely formalized, and it is not guaranteed that they are bijective. For example, packages used together with cross compilers have been following the convention of using the name that the native package would use, followed by a dash, the target architecture name, another dash and the word "cross". There is no guarantee that the architecture name does not contain a dash, however, so we cannot simply take the second to last word as the architecture name if the last word is "cross" (think "libgcc1-kfreebsd-i386-cross"), which makes it difficult to write a plug-in to apt that would automatically download the binary for the target architecture and convert it using dpkg-cross. I've decided to nonetheless go ahead with my request that the "-*-cross" part of the package name space be reserved for packages related to cross compilation; it will work for the time being.

Next.

Previous.