Search Results: "tar"

15 September 2009

David Welton: Tcl and the Tk Toolkit, 2nd edition

<iframe frameborder="0" marginheight="0" marginwidth="0" scrolling="no" src="http://rcm.amazon.com/e/cm?lt1=_blank&amp;bc1=000000&amp;IS2=1&amp;bg1=FFFFFF&amp;fc1=000000&amp;lc1=0000FF&amp;t=dedasys-20&amp;o=1&amp;p=8&amp;l=as1&amp;m=amazon&amp;f=ifr&amp;md=10FE9736YVPPT7A0FBG2&amp;asins=032133633X" style="width:120px;height:240px; float: left; padding-bottom: 10px; padding-right: 10px;"></iframe> I'm not sure what I was expecting, but the news that "my" book is finally out is cause for excitement, anxiety, and relief. It's not actually "my" book at all; it was originally written by Dr. John Ousterhout, who created the Tcl programming language, and the Tk toolkit. And Ken Jones is listed as the co author of the 2nd edition, because he's the guy that pulled everything together and managed to actually bring the project to fruition. However, I did play a part a part in the project. About 5 years ago, I was doing a lot with Tcl, and then, as now, I thought it was a shame that the language was not promoted as well as it could be. It's not a perfect language, but at nearly 20 years old, it has stood the test of time, and it has a lot going for it. In any case, at the time, I had noticed that there was no good, publicly available tutorial. Tcl has always had an excellent set of man pages describing all of its various commands and subsystems, but did not ship with a tutorial that would help new users get started. I thought that that would be a good problem to solve in order to make the language more attractive to newcomers. However, writing a tutorial from scratch is no simple task, so I started looking around for material that might be adapted. One thing that sprung to mind was the first edition of "Tcl and the Tk Toolkit". At that point the book was nearly 10 years old, which in tech years is practically forever, and indeed, the material in the book, while good, was quite dated for some portions of Tcl. Figuring it couldn't hurt, I emailed Dr. Ousterhout to see what he thought of open sourcing the book, or at least the first part of it, with the idea of updating and utilizing that as a tutorial. He was actually ok with that idea, but the copyright belonged to the publisher rather than him, so he gave me a contact with the publisher, who I then sent my idea to. They weren't necessarily against the idea of releasing the material under an open license, but came back with another idea: why not update the book? That sounded like an interesting project in its own right. I don't recall all the details (this was in 2004), but the publisher ended up with Ken Jones as the guy to run the project, as he'd participated successfully in updating another popular Tcl book. In the end, there were several of us (Eric Foster-Johnson, Donal Fellows and Brian Griffin, in addition to Ken and myself), that worked on the project, divying things up by sections, with Ken coordinating. I am honored to have my name listed amongst such august company. Which brings me back to being nervous: the idea of having something I wrote "set in stone" is a source of some anxiety for someone like me used to working with such a malleable world as that of software. Also; I wrote most of my contribution (I focused on the C API) several years ago, and I hope it's still valid, and of course that there aren't any huge and glaring errors. For a while, the book seemed like it was in limbo - just when I thought it was dead, Ken would come back with an update. This was also a cause for concern, as I wanted the thing to either die a dignified death, and get on with my life, or appear in print. As for the tutorial, I did manage to make that happen as well. Clif Flynt, who authored a Tcl/Tk book of his own, was kind enough to donate some material from a tutorial of his own, which several of us hacked at to get it into shape. It's available here: http://www.tcl.tk/man/tcl/tutorial/tcltutorial.html In any case, I sincerely hope that the material I wrote manages to do justice to the years and years of work that have gone into the language and its implementation, and hope you find the book valuable if you pick up a copy.

Daniel Leidert: Writing manual pages in GROFF (1)

I always wanted to write a short howto for Debian package maintainers, how to write a manual page in GROFF. This is useful for short documents (for longer documents, docbook/docbook-xsl can be a good choice). I already have written some parts. However, here is the first part of hints. More might follow later. The delimiter in the NAME section The name section is (probably) the only place, where exactly one hypen-minus \- must appear. The hyphen-minus is the delimiter between the command name and one-line description.
.SH NAME
foo-bar \- foos the template foo-what-bar
Typical mistakes regarding paragraphs Using a .PP macro directly following a .SH or .SS macro is useless. This macro should be used between paragraphs:
.SH OPTIONS
The program follows the usual GNU command line syntax, with long options starting with
two dashes ( \-').
.PP
A summary of options is included below.
Options/File descriptions To describe options or files it s usually useful to make use of the .TP macro.
.SH OPTIONS
.TP
.B \-f, \-\-force
Force the execution of the specified command.
.SH FILES
.TP
.I ~/.foobar
Per user configuration file.
To create more than one intended paragraph the .IP macro can be used.
.TP
.B \-f, \-\-force
Force the execution of the specified command.
.IP
This option has no effect in conjunction with \fB\-\-foo\fP.
Avoid hyphenation in URLs/URIs/paths Usually we don t want URLs or URIs to be hyphenated. This can be done using the \% sequence. Typical examples:
A short tutorial is available online at \fI\%http://foo.tld/some/path/here/manual.html\fP.
On a Debian system the complete text of the GNU General Public License
version 2 can be found in the file \fI\%/usr/share/common-licenses/GPL\-2\fP.
Referencing persons and their mail address The common markup is to write the person name in bold letters and the mail address in roman letters is put into angle brackets. It s usually a good idea to mark where the mail address starts and ends. We can use the \& sequence as shown in the following example:
\fBDaniel Leidert\fP <\&daniel.leidert@wgdd.de\&>

Robert Collins: 16 Aug 2009


Hudson seemed quite nice when I was looking at how drizzle use it. I proposed it to the squid project to replace a collection of cron scripts and we re now in the final stages of deployment: Hudson is running, doing test builds. We are now polishing the deployment, tweaking where and how often reports are made, and adding more coverage to the build farm. I thought I d post a few observations about the friction involved in getting it up and running. Understand that I think its a lovely product these blemishes are really quite minor. Installation of the master machine on Ubuntu add an apt repository, apt-get update, apt-get install. Installation of the master machine on CentOS 5.2 (Now 5.3): make a user by hand, download a .war file, create an init.d script by copy-and-adjusting an example off the web. Plenty of room to improve here. Installing slave machines: make a user by hand, add an rc.local entry to run java on slave.jar as that new user. This could be more polished Installing a FreeBSD 6.4 slave: manually download various java sources to my laptop, scp them up to the FreeBSD machine, build *java* overnight, then make a user, add an rc.local entry etc. _Painful_. The next thing we noticed was that the model in Hudson doesn t really expose platforms but we want to test on a broad array of architectures, vendors and releases. i386-Ubuntu-intrepid building doesn t imply that i386-Debian-lenny will build. We started putting in tags on the slaves that will let us say this build is for amd64-CentOS-5.2 , so that if we have multiple machines for a platform, we ll have some redundancy, and so that its easy to get a sense of whats failing. This had some trouble its very manual, and as its manually entered data it can get out of date quite easily. So in the weekend I set out make a plugin, and ran into some yak shaving. Hudson plugins use maven2 to build and deploy. So I added the maven2 plugin to my eclipse (after updating eclipse to get the shiniest bzr-eclipse), and found the first bug issue 1580 maven2 and teamplugins in eclipse 3.5 don t play all that nice. Push. Removing bzr-eclipse temporarily allowed eclipse s maven plugin to work, but for some reason many dependencies were not found, and various discussions found on the net suggest manually adding them to the CLASSPATH for the project but not how to identify which ones they were. Pop. So, I switched to netbeans a 200MB download, as Ubuntu only has 6.5 in the archive. netbeans has the ability to treat a maven2 project as a directly editable project. I have to say that it works beautifully. Push. I made a new plugin, looked around for an appropriate interface, (DynamicLabellers, designed for exactly our intended use). Sadly, in my test environment, it didn t work the master didn t call into the plugin at all, and no node labels were attached. Push. Grab the source for hudson itself, find the trick-for-newcomers here do a full build outside netbeans, in netbeans open main/war as a project, and main/core as well, not just the top level pom.xml. To run, with main/war selected in the project list hit the debug button. However changes made to the main/core sources are not deployed until you build them (F11) the debug environment looks nearly identical to a real environment. Pop. There is a buglet in DynamicLabeller support in Hudson, where inconsistent code between general slave support and the master node Hudson.java causes different behaviour with dynamic labels. Specifically the master node will never get dynamic labels. So I fixed this, cleaned up the code to remove the duplication as much as possible (there are comments in the code base that different synchronisation styles are needed for some reason) and submitted upstream. Pop. I ll make the plugin for squid pretty some evening this week, and we should be able to start asking for volunteers for the squid build farm. Yay!

Robert Collins: 31 Aug 2009


Hi Rich! Re hour+long unit tests I agree that you need a comprehensive test suite, and that it should test all the dark and hidden corners of your code base. But time is not free! A long test suite inhibits:
  • cycle time the fastest you can release a hot fix to a customer
  • developer productivity you can t forget about a patch till its passed the regression test suite
  • community involvement if it takes an hour to run the test suite, an opportunistic developer that wanted to tweak something in your code will have walked away long ago Note that these points are orthogonal to whether developers edit-test cycle runs some or all tests, or whether you use a CI tool, or a test-commit tool, or some other workflow. All that said though, I m extremely interested in *why* any given test suite takes hours: does it need to? What is it doing? Can you decrease the time by 90% and coverage by 2%? I got another response back, which talks about keeping the working set of tests @ about 5 minutes long and splitting the rest off (via declared metadata on each test) into run after commit or during CI . This has merits for reducing the burden on a developer in their test-commit cycle, but as I claim above, I believe there is still an overhead from those other tests that are pending execution at some later time. From a LEAN perspective, the cycle time is very important. Another important thing is handoffs. Each time we hand over something (e.g. a code change that I *think* works because it passed my local tests), there is a cost. Handing over to a machine to do CI is just as expensive as handing to a colleague. Add that contributors sending in patches from the internet may not hang around to find out that their patch *fails* in your CI build, and you can see why I think CI tools are an adjunct to keeping a clean trunk, rather than a key tool. The key tool is to not commit regressions :) Oh, and I certainly accept that test suites should be comprehensive I just don t accept that more time == more coverage, or that there isn t a trade off between comprehensive and timeliness.
  • Robert Collins: 30 Aug 2009


    Made some time to hack the results: config-manager 0.4 released, re-uploaded to debian (it was removed due to some confusion a while back). This most notably drops the hard dependency on pybaz and adds specific-revision support for bzr. subunit snapshot packaging sorted out to work better with subunit from Ubuntu/Debian. This latest snapshot has nested progress and subunit2gtk included. PQM got a bit of a cleanup:
  • The status region shown during merges is ~ twice as tall now.
  • if the precommit_hook outputs subunit it will be picked up automatically and shown in the status region.
  • all deprecation warnings in python2.6 are cleaned up
  • Pending bugfixes were merged from Tim Cole and Daniel Watkins thanks guys!
  • Robert Collins: 27 Aug 2009


    Does your test suite take too long (e.g. 5 minutes). Or did it and you solved it? Or it doesn t but its getting worse? Tell me more, I d like to know :-)

    Robert Collins: New blog location


    My blog has moved: http://rbtcollins.wordpress.com/. If you re syndicating me, please update to this location; if you don t thats fine advogato will be syndicating the blog indefinitely, but doesn t support comments. Mega thanks to Jeff for doing an export from advogato for me :)

    14 September 2009

    Jo Shields: Flash Wins! Hoo-freaking-ray! Adobe are so awesome!

    For those who aren t clinically dead, you may have heard of the BBC . The BBC are the state-ish-funded TV network in the UK, and the country s biggest broadcaster, alongside three other major terrestrial broadcasters who make their content widely available without payment ITV, Channel 4, and Five. These broadcasters also make some or all of their programming available for streaming over the Internets usually their home-grown programming only, not licensed stuff from America. Now, once upon a time, the online functionality was mostly offered via a proprietary P2P-and-DRM-based system called Kontiki. Kontiki was unpopular for various reasons for example, it was Windows-only, and banned by several Internet providers due to the use of P2P (e.g. I know such technology is still banned on the University of Oxford network). As a result, this lead to the introduction and eventual replacement of browser-based streaming solutions, starting with the changes made to BBC s iPlayer. iPlayer is nowadays a combination of an Adobe Flash service for web surfers, an unencrypted MP4 streaming service for users of mobile devices such as iPhones, and as of a couple of weeks ago, has streaming support built directly into the PlayStation 3 s user interface too. Oh, and on the PC, it supports DRM-based downloading courtesy of the Adobe Air platform, on Windows, Mac, and Air-capable (x86) Linux. Channel 4 s service 4od, and Five s Demand Five, are also based on Adobe Flash, and streaming-only. They re basic and functional. The final interesting one here is ITV s ITV Player. ITV Player was, until about a week ago, the only non-Flash service, instead making use of Microsoft Silverlight. They have now changed to be Flash-based, like their peers, meaning the entire market mandates use of the Adobe Flash plugin or, at a push, command-line utilities or browser plugins which grab the raw video files from the broadcasters servers, in violation of their licenses. Why is this interesting? Well, when ITV Player used Silverlight, we could watch TV using Free Software: Novell Moonlight 2.0 Beta 1.1 and above worked fine with it, on i386 and AMD64 (and other architectures with a recompile). Now that it s using Flash rather than Silverlight, where do we stand for watching streaming TV legitimately with Free Software? Let s take a peek! Firstly, some preamble. I m running Ubuntu Jaunty, and I hand-compiled a SVN (I think SVN? Maybe Bzr) snapshot of Gnash revision 11485 to ensure I had an up-to-date view of proceedings. It definitely seems to be working, as I m introduced to the world of Flash advertising via Gnash, and Youtube.com also works, more or less. And I m not discussing Adobe s Flash plugin here, for various reasons: So, Gnash it is. Firstly, it s a PITA to compile, as upstream seem to have misunderstood how AutoFoo works i.e. it detects that you don t have headers installed for certain features, and tells you so, yet still enables those features at configure-time (and obviously fails when building). Hey guys, if I don t have qapplication.h, take the hint and disable KDE support for me like every other bloody app does. With that out of the way, let s get to it! BBC iPlayer Well, what does the landing look like?
    BBC iPlayer Landing

    BBC iPlayer Landing

    Looks fine to me! Let s try playing a show!
    BBC iPlayer Playback

    BBC iPlayer Playback

    Oh. Um Never mind, then. Seems iPlayer s JavaScript to detect Flash presence doesn t pick up on Gnash, and it bails out. Score so far: 0/1 Demand Five Another former Kontiki partner, how do these guys fare?
    Demand Five Landing

    Demand Five Landing

    Hey, that looks pretty good to me! Perhaps we re onto a winner this time?
    Demand Five Playback

    Demand Five Playback

    Poot. Gnash is definitely being invoked see the context menu there but it sure isn t doing anything useful. Score: 0/2 Channel 4 4od How do those hip cats at Channel 4 fare?
    Channel 4 4od Landing

    Channel 4 4od Landing

    Hm Gotta confess, not feeling too hopeful about this one
    Channel 4 4od Playback

    Channel 4 4od Playback

    Oh, well, even worse than iPlayer. What you folks might not recognise is the missing Play buttons which are supposed to be to the right of the program descriptions in that list of episodes. Presumably more Javascript/Flash interop failure. 0/3 ITV Player Last, but not least, how does the newest entrant into Microsoft-free streaming fare?
    ITV Player Landing

    ITV Player Landing

    Now, now, in fairness, it s always been this bad, even when they used Silverlight weirdos that they are, ITV have always used Flash for their navigation, even when they used Silverlight for playback. The Flash-based navigation you (don t) see here is barely any better even with the proprietary Flash plugin. What the proprietary plugin does NOT do, however, is consume the 750 or so meg of RAM that Gnash did when sat idle on this screen. I mean, that s not a problem, that s why I have 6 gig in here, but still, not wise on Wifey s netbook.
    ITV Player Playback

    ITV Player Playback

    Oh. Well then. 0/4 it is. So? What annoys me here isn t so much that nothing works. I m used to there being temporary gaps in Free Software functionality, that s pretty normal. But I m greatly vexed that one of these four used to work on a Free platform, and now it doesn t and that places like UbuntuForums are filled with people celebrating that fact. Celebrating that ITV have stopped using the evil nasty Microsoft system which happened to have a functional Free replacement and that they ve now moved to a non-Microsoft system which mandates a proprietary plugin. It s not the first time either Major League Baseball in the USA used to use Silverlight for their HD streaming, and now they use a combination of Flash and a proprietary Windows-only extension to Flash to make it load WMV files this is considered a victory. It really isn t. We should NEVER be forced to use Proprietary software in order to surf the web yet now we ve gone from having access to 25% of the UK s streaming TV services via Free Software to 0% of them, and people are happy about it. So, I want you to think long and hard about this question, one which seems to get the oddball answer far too often: What is more important promoting Free Software, or demoting Microsoft Corp?

    13 September 2009

    Jelmer Vernooij: CtrlProxy: Looking for a new maintainer

    After over 7 years of working on it off and on, I'm looking for somebody to help maintain (and eventually take over) CtrlProxy. I started working on CtrlProxy somewhere in 2002, only a short while after Wilmer started hacking on BitlBee. If I remember correctly I started working on it because I didn't want to run a separate dircproxy (the only real competitor at the time) instance (with configuration) for each IRC network that I connected to. It was also just a good excuse to play with the IRC protocol a bit. Over the years, CtrlProxy has served as a playground for me to try out new and interesting things. It's been rewritten or severely refactored several times in its early history, the latest time being the 3.0 release (from 2005). I've tried different build systems, I've tried different implementation languages, I've tried different configuration file formats, I've tried different support libraries, I've tried different version control systems, I've tried different documentation formats. So while it's definitely been a very educational project for me personally, I haven't really had the time or the interest to dedicate to the project that it deserved during the last few years. This was mostly because there were other more interesting FOSS projects I spent my spare cycles on. These days there are plenty of other good IRC proxies out there, including BIP, ZNC and ShroudBNC so I doubt CtrlProxy will be missed if it were to disappear. Despite that, if anybody is interested in taking over, please send me an email (jelmer@samba.org) or contact me on IRC (jelmer on the OFTC and Freenode networks). cp: Anathema - Shroud of False1

    Clint Adams: Hello world!

    Welcome to WordPress. This is your first post. Edit or delete it, then start blogging!

    Joachim Breitner: The Automatic Rule-Based Time Tracker

    A while ago, I thought: I seem to be less productive than I used to be. Why might that be? Do I spend more and more time with e-Mails? Or is it something else? I couldn t tell, so I needed a time tracking tool. There are a few of those for Linux that allow you, while working, specify what you are doing and then generate statistics about it. This approach has a few disadvantages:So I created arbtt, the Automatic Rule-Based Time Tracker. It comes with a background program that is started with your desktop session and will, each minute, record what windows are open, which one is active, what their titles and corresponding programs are. It also checks how long the user has been idle. No interaction required, no distraction possible. This information is stored in a log file. A separate tool allows the user to investigate this data. It is called rule-based because the mapping from the raw data to sensible tags that give information about the time sample is specified by a simple, but hopefully sufficiently powerful language.A simple example for a rule that indicates the currently used program would be:
    tag Program:$current.program,
    The prefix Program: is a category. Arbtt will ensure that for each time sample and category, at most one tag is specified. A more complex rule is
    current window $title =~ m!(?:~ home/jojo)/projekte/(?:programming/(?:haskell/)?)?([^/)]*)!
    ==> tag Project:$1,
    which makes use of the fact that both GVim and gnome-terminal display the full path to the currently edited file resp. to the working directory in the window title. This rule will track all my projects separately, and even automatically pick up new projects when they appear! You can see more rules in the example file.The statistics program then allows you to query the tags with some options:
    Usage: arbtt-stats [OPTIONS...]
    -h, -? --help show this help
    -V --version show the version number
    -x TAG --exclude=TAG ignore samples containing this tag
    -o TAG --only=TAG only consider samples containing this tag
    --also-inactive include samples with the tag "inactive"
    -m PERC --min-percentage=PERC do not show tags with a percentage lower than PERC% (default: 1)
    -i --information show general statistics about the data
    -t --total-time show total time for each tag
    -c CATEGORY --category=CATEGORY show statistics about category CATEGORY
    For example, if I want to know what folders I have open while using evolution, I can run arbtt-stats -o Program:evolution -c Evo-Folder -m 3 to see this:
    Statistics for category Evo-Folder
    ==================================
    Time Percentage
    =========================++======================
    Evo-Folder:Eingang 22h20m00s 17.0%
    Evo-Folder:Bekannte 16h40m00s 12.7%
    Evo-Folder:d-haskell 9h40m00s 7.4%
    Evo-Folder:Itomig 8h20m00s 6.4%
    Evo-Folder:Verschickt 7h40m00s 5.8%
    Evo-Folder:pkg-fso-maint 7h00m00s 5.3%
    Evo-Folder:Bugs 6h00m00s 4.6%
    Evo-Folder:planet debian 4h00m00s 3.0%
    (60 entries omitted) 4h49m00s 36.7%
    (unmatched time) 8m00s 1.0%
    One big advantage of this approach is that you do not need to know in advance what queries you are interested in. Since the rules are applied when you are evaluating your data, and not when recording it, you can add more tags and forgotten special cases later.At this point, a big warning is due: This program will record a lot of very sensitive information about you. Be aware of this before you start using arbtt, and make sure you protect your data. You can get rid of all logs by deleting ~/.arbtt/capture.log.I have published arbtt on hackage. If you have cabal-install installed, you can install it with cabal install arbtt. See the README file for more information about setting it up. Depending on the feedback I get I will also consider packaging it for Debian.If you want to contribute, you are very welcome. The code is available at the darcs repository http://darcs.nomeata.de/arbtt/ (DarcsWeb). See the README for some ideas what to implement and feel free to come up with new ideas.

    Joey Hess: debhelper talk

    In Spain this summer, I gave a talk about new features in debhelper. It seemed to help a lot of people get started. So if you were not at DebConf, this is your chance to learn about dh, overrides, addons, and mimimal rules files. (low quality higher quality) The audio gets much better half-way through, and slides become visible. (We all also wake up a bit midway through -- it had been a late night!) The full slides are here.

    Joey Hess: d-i code swarm

    In May, Frans and I worked on making a code swarm video that covers most of the history of commits to debian-installer. Then at DebConf, Christian bravely gave a short presentation of the video, with witty commentary. Here I've combined the two, so we have Christian's commentary, with video that allows reading the names of the committers. Click image to play. (Also on vimeo).

    Brice Goglin: Debian/X.org notes - i865 fixed, Xserver 1.6 entering testing soon

    No Xorg update entered testing since Lenny was released. The last big remaining bug in unstable was the Intel driver locking up on i865 when the UXA/GEM acceleration is used (and 2.8.x only supports UXA so there is no work around). See #541307. Fortunately, Eric Anholt found out that it was caused by a kernel bug in the intel-agp driver. The fix is not in vanilla 2.6.31, so you'll have to apply the patch or wait for an updated 2.6.31.x kernel to be released. Anyway, the Intel driver 2.8.1 as well as Xserver 1.6 and Mesa 7.5 will enter testing soon. If you have a i865, make sure your kernel contains the above fix or you'll likely experience lockups soon after X startup. Update: If building the intel-agp driver as a module, you will also need another small patch to export the clflush_cache_range() function to modules. Update: Everything just entered testing of real.
    (Permanent link)

    12 September 2009

    Kees Cook: uninstall sun-java6

    With the vrms meme raging on Planet Ubuntu, I noticed some people still have sun-java6 installed. I ve been using openjdk-6 since Hardy, and everything I use works fine with it (e.g. Vuze, Catan, Eclipse, FreeMind, and even Facebook s photo uploader thing). Given the Ubuntu Tech Board s remove sun-java6 from the archive Agenda item, it s possible sun-java6 (being redundant and orphaned) would go away in Ubuntu Karmic (sun-java5 is already gone because Sun will drop support for it in October, and sun-java6 is not far behind). I ve heard some noises about openjdk-6 not working for people, but I haven t actually seen any direct technical evidence of something working in one and not the other. Seeing as Jaunty s OpenJDK was certified by Sun, it would be very interesting to find failures, as this would indicate that Sun s certifications are missing something. Looking for examples of failures in Google, I couldn t find anything with obvious test-cases that failed with openjdk-6. I suspect I m just not trying hard enough, but I m curious what other people have run into. Test for yourself:
    sudo apt-get purge sun-java6-jdk sun-java6-jre sun-java6-bin sun-java6-plugin
    sudo apt-get install openjdk-6-jdk icedtea6-plugin
    (Don t forget to restart your browser.)

    C.J. Adams-Collier: Well, that was an eventful day!

    *whew* I did a bunch of things yesterday. We took our kinderg rtner to her first Friday at her new school (and were about 10 minutes tardy. oops). We then took our toddler to a nearby playground with swings and slides and let her expend some energy. After she had been sufficiently exercised, we walked back home, stopping at a coffee shop on the way. The baristo (you call male baristas baristos, right? :) ) recognized my MC Frontalot shirt and asked whether I had caught him the previous weekend at PAX. Unfortunately, I have not attended PAX since 2006, but I *did* purchase the tee directly from The Front himself ;) When we got home, I worked a bit on an English Language parser implementation and then went to the University of Washington to meet with Emily Bender about getting in to the Professional Master s program in Computational Linguistics. It all looks good, and I even got the good news that the GRE is no longer required! After the meeting, I headed home and poked at the parser for a little while longer. I then picked Scarlet up from after-school care and brought her home. I then hopped in the car and drove toward Bellevue to meet up with Monty while he s in town. I over-estimated the amount of time traffic would steal on my way to Bellevue, and had an extra hour to blow. So I dropped by building 41 and shot the IronPython bull with Dino. It turns out he s got an android phone, too. I told him it was possible to put a debian chroot on it and that he should even be able to apt-get install ironpython to his phone soon ;) We talked briefly about the CodePlex Foundation and Sam Ramji s departure from The Evil Empire. Dino seems skeptical about the project. I don t have enough information to have much of an opinion. However, it sounds like some folks I trust are involved, so I m hopeful. I left MS just in time to make it to the wrong address at the specified time. My phone had just enough juice to call Monty to get the right address and then use the navigation system to find my way there. I wasn t able to make reservations at the place we intended to go for dinner until 8:15, so we went to the Barnes & Noble for a bit. They only had one NLP book in stock and the examples are all in Python. I should learn that language one of these days As we were leaving the Pacific Place, Monty mentioned to me that he is on the advisory board for the CodePlex Foundation, and that they have been responsive enough to his input that they changed the Mission statement, at his recommendation, just one day before the Foundation was publicized. He feels that this is a very good direction for Microsoft to be heading. My brother Chris was kind enough to watch the kids while we went out to dinner. Quick note: he recently graduated from UW with a BA in Electrical Engineering and is looking for work using his acquired knowledge, in case anyone needs one of those ;) We met up with my wife, Hannah and our friends, Mike & Cynthia at our place. Monty graciously avoided mentioning the terrible state in which our apartment has recently found itself. The kids were super cute and polite and said hi/bye. Over dinner we discussed building an android app (Monty has one, too ;) ) to automate the process of creating bounties for apps and getting folks to implement them. We also talked about MySQL and MariaDB, of course. Hannah and I recalled my time working for MySQL, Inc. on the MaxDB project and some subtle cultural differences we noticed while traveling. It was interesting getting the inside scoop about the Sun acquisition and some of the recent goings-on in the MySQL/Sun/Oracle world. I wasn t aware, for instance, that the EU is balking on the merger because of monopoly concerns.
    <script>//<script language="javascript" src="http://reddit.com/button.js?t=1"></script>

    Bastian Venthur: Gridengine and Qlogin Zombies

    Dear Lazyweb, we re using gridengine 6.2 on our cluster system at work. On this system we have a queue for interactive jobs. The queue has a fixed number of slots. If someone wants to start an interactive session and no free slots are available the interactive job cannot start otherwise it will consume one slot during the session and free it again after the user quit the interactive session. Unfortunately, if the user forgets to quit the interactive session properly (e.g. by entering quit in the terminal) and just closes the terminal, the interactive session becomes a zombie. The user cannot use this session anymore, but the process is still running and thus blocking a slot. Since it happens quite often that users fail to quit the interactive session properly, the free slots are used up pretty soon and no one can start new interactive sessions anymore quite a nice bug in the gridengine software (available in Debian/Lenny by the way ) You can see that those processes are still running using ps and qstat. The question is: how can I effectively search for those processes and kill them? Gridengine does not seem to be aware of those zombies and the user might run other interactive jobs in parallel which must not be killed. So how does one find the zombies and kill them reliably?

    11 September 2009

    Zak B. Elep: New Server

    Site was down for almost a week, due to my provider being DDoSed. Email was broken more so as SMTP was the target, but it was rather odd that the data center decided to block outgoing SMTP connections from a block of IPs, where my VPS is unfortunately a member of. I've since switched to a new provider. This makes my third OpenVZ VPS, though I was meaning to move on to Xen. At 6 US dollars a month though, it seems quite a bargain, so I'll have to see where this goes. The move also afforded me to change some stuff under the hood. Apache is gone, replaced by nginx (although this required having blosxom running under a thttpd backend, until I can whip up a FastCGI equivalent.) BIND gave way to maradns, cutting down significant memory usage. And dropbear replaces the OpenSSH server. Exim 4 and dovecot remain though, as they seem to quite big (or small) enough for my purposes. I'll document more about the changes in later posts. In the meantime, I need to catch up on emails and packaging...

    John Goerzen: Hello world!

    Welcome to WordPress. This is your first post. Edit or delete it, then start blogging!

    Ian Wienand: Django toolchain on Debian

    Although Django is well packaged for Debian, I've recently come to the conculsion that the packages are really not what I want. The problem is that my server runs Debian stable, while my development laptop runs unstable, and Django revisions definitely fall into the "unstable" category. There really is no way to use a system Django 1.1 on one side, and a system Django 1.0 on the other. After a bit of work, I think I've got something together that works, and I post it here in the hope it is useful for someone else. This info has been gleaned from similar references such as this and this. This is aimed at running a server using Debian stable (5.0) for production and an unstable environment for development. You actually need both to get this running. This is based on a project called "project" that lives in /var/www
    1. First step is to install python-virtualenv on both.
    2. Create a virtualenv on both, using the --no-site-packages to make it a stand-alone environment. This is like a chroot for python.
      $ virtualenv --no-site-packages project
      New python executable in project/bin/python
      Installing setuptools............done.
      
    3. The unstable environment has a file you'll need to copy into the stable environment - bin/activate_this.py. The stable version of python-virtualenv isn't recent enough to include this file, but you need it to essentially switch the system python into the chrooted environment. This will come in handy later when setting up the webserver.
    4. There are probably better ways to keep the two environments in sync, but I simply take a manual approach of doing everything twice, once in each. So from now on, do the following in both environments.
    5. Activate the environment
      /var/www$ cd project
      /var/www/project$ . bin/activate
      (project) /var/www/project$
      
    6. Use easy_install to install pip
      (project) /var/www/project$ easy_install pip
      Searching for pip
      Reading http://pypi.python.org/simple/pip/
      Reading http://pip.openplans.org
      Best match: pip 0.4
      Downloading http://pypi.python.org/packages/source/p/pip/pip-0.4.tar.gz#md5=b45714d04f8fd38fe8e3d4c7600b91a2
      Processing pip-0.4.tar.gz
      Running pip-0.4/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Wu9O-U/pip-0.4/egg-dist-tmp-xjSdxq
      warning: no previously-included files matching '*.txt' found under directory 'docs/_build'
      no previously-included directories found matching 'docs/_build/_sources'
      zip_safe flag not set; analyzing archive contents...
      pip: module references __file__
      Adding pip 0.4 to easy-install.pth file
      Installing pip script to /var/www/project/bin
      Installed /var/www/project/lib/python2.5/site-packages/pip-0.4-py2.5.egg
      Processing dependencies for pip
      Finished processing dependencies for pip
      
    7. Install setuptools, also using easy_install (for some reason, pip can't install it). There is a trick here, you need to specify at least version 0.6c9 or there will be issues with the SVN version on Debian stable when you try to get Django in the next step.
      (project) /var/www/project$ easy_install setuptools==0.6c9
      Searching for setuptools==0.6c9
      Reading http://pypi.python.org/simple/setuptools/
      Best match: setuptools 0.6c9
      Downloading http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c9-py2.5.egg#md5=fe67c3e5a17b12c0e7c541b7ea43a8e6
      Processing setuptools-0.6c9-py2.5.egg
      Moving setuptools-0.6c9-py2.5.egg to /var/www/project/lib/python2.5/site-packages
      Removing setuptools 0.6c8 from easy-install.pth file
      Adding setuptools 0.6c9 to easy-install.pth file
      Installing easy_install script to /var/www/project/bin
      Installing easy_install-2.5 script to /var/www/project/bin
      Installed /var/www/project/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg
      Processing dependencies for setuptools==0.6c9
      Finished processing dependencies for setuptools==0.6c9
      
    8. Create a requirements.txt with the path to the Django SVN for pip to install, then and then install it.
      (project) /var/www/project$ cat requirements.txt
      -e svn+http://code.djangoproject.com/svn/django/tags/releases/1.0.3/#egg=Django
      (project) /var/www/project$ pip install -r requirements.txt
      Obtaining Django from svn+http://code.djangoproject.com/svn/django/tags/releases/1.0.3/#egg=Django (from -r requirements.txt (line 1))
        Checking out http://code.djangoproject.com/svn/django/tags/releases/1.0.3/ to ./src/django
      (project) /var/www/project$ pip install -r requirements.txt
      Obtaining Django from svn+http://code.djangoproject.com/svn/django/tags/releases/1.0.3/#egg=Django (from -r requirements.txt (line 1))
        Checking out http://code.djangoproject.com/svn/django/tags/releases/1.0.3/ to ./src/django
      ... so on ...
      
    9. Almost there! You can keep installing more Python requirements with pip if you need, but we've got enough here to start.
    10. Create a file in /var/www/project called project-python.py. This will be the Python interpreter the webserver uses, and basically exec's itself into the virtalenv. The file should contain the following:
      activate_this = "/var/www/project/bin/activate_this.py"
      execfile(activate_this, dict(__file__=activate_this))
      from django.core.handlers.modpython import handler
      
    11. Now it's time to start the Django project. I like to create a new directory called project, which will be the parent directory kept in the SCM with all the code, media, templates, database (if using SQLite) etc. In this way to keep the two environments up-to-date I simply svn ci on one side, and svn co on the other.
      (project) /var/www/project$ mkdir project
      (project) /var/www/project/project$ mkdir db django media www
      (project) /var/www/project/project$ cd django/
      (project) /var/www/project/project/django$ django-admin startproject myproject
      
    12. Last step now is to wire-up Apache to serve it all up. The magic is making sure you specify the correct PythonHandler that you made before to use the virtualenv, and include the right paths so you can find it and all the required Django settings.
      DocumentRoot /var/www/project
      <Location "/">
          SetHandler python-program
          PythonHandler project-python
          PythonPath "['/var/www/project/','/var/www/project/project/django/'] + sys.path"
          SetEnv DJANGO_SETTINGS_MODULE myproject.settings
          PythonDebug On
      </Location>
      Alias /media /var/www/project/project/media
      <Location "/media">
          SetHandler none
      </Location>
      <Directory "/var/www/project/project/media">
          AllowOverride none
          Order allow,deny
          Allow from all
          Options FollowSymLinks Indexes
      </Directory>
      
    With all this, you should be up and running in a basic but stable environment. It's easy enough to update packages for security fixes, etc via pip after activating your virtualenv.

    Next.

    Previous.