Search Results: "nate"

31 December 2009

John Goerzen: My Reading List for 2010

I can hear the question now: What kind of guy puts The Iliad and War and Peace on a list of things to read for fun? Well, me. I think that reading things by authors I ve never read before, people that take positions I haven t heard of before or don t agree with, or works that are challenging, will teach me something. And learning is fun. My entire list for 2010 is at Goodreads. I ve highlighted a few below. I don t expect to read all 34 books on the Goodreads list necessarily, but there is the chance. The Iliad by Homer, 750BC, trans. by Alexander Pope, 704 pages. A recent NPR story kindled my interest in this work. I m looking forward to it. The Oxford History of the Classical World by Boardman, Griffin, and Murray, 1986, 882 pages. It covers ancient Greece and Rome up through the fall of the Roman empire. The Fires of Heaven (Wheel of Time #5) by Robert Jordan, 1994, 912 pages. I ve read books 1 through 4 already, and would like to continue on the series. War and Peace by Lev Leo Nikolayevich Tolstoy, 1869, 1392 pages. Been on my list for way too long. Time to get to it. Haven t read anything by Tolstoy before. The Politics of Jesus by John Howard Yoder, 1972, 2nd ed., 270 pages. Aims to dispel the notion of Jesus as apolitical. An Intimate History of Humanity by Theodore Zeldin, 1996, 496 pages. Picked this up at Powell s in Portland on a whim, and it s about time I get to it. The Myth of a Christian Nation: How the Quest for Political Power Is Destroying the Church by Gregory A. Boyd, 2007, 224 pages. An argument that the American evangelical church allowed itself to be co-opted by the political right (and some on the left) and argues this is harmful to the church. Also challenges the notion that America ever was a Christian nation. Daily Life in Ancient Rome: The People and the City at the Height of the Empire, by Jerome Carcopino, 2003, 368 pages. I ve always been fascinated with how things were on the ground rather than at the perspective of generals and kings, and this promises to be interesting. Slavery, Sabbath, War, and Women: Case Issues in Biblical Interpretation (Conrad Grebel Lectures) by Willard M. Swartley, 1983, 368 pages. Looking at how people have argued from different Biblical perspectives about various issues over the years. To the Lighthouse by Virginia Woolf, 1927, 252 pages. I can t believe I ve never read Woolf before. Yet another one I m really looking forward to. Tales of the Jazz Age by F. Scott Fitzgerald, 1922, 319 pages. Per Goodreads: This book of five confessional essays from the 1930s follows Fitzgerald and his wife Zelda from the height of their celebrity as the darlings of the 1920s to years of rapid decline leading to the self-proclaimed Crack Up in 1936. Ulysses by James Joyce, 1922 (1961 unabridged version), 783 pages. The Future of Faith by Harvey Cox, 2009, 256 pages. Per Goodreads, Cox explains why Christian beliefs and dogma are giving way to new grassroots movements rooted in social justice and spiritual experience. Heard about this one in an interview with Diane Rehm. Being There by Jerzy Kosi ski, 1970, 128 pages. Jesus: Uncovering the Life, Teachings, and Relevance of a Religious Revolutionary by Marcus Borg, 2006, 352 pages. Whether or not you agree with Borg, this has got to be a thought-provoking title. The Three Musketeers by Alexandre Dumas, 1844, 640 pages. The Book of Tea by Kakuzo Okakura, 1906, 154 pages. Per Goodreads: In 1906 in turn-of-the century Boston, a small, esoteric book about tea was written with the intention of being read aloud in the famous salon of Isabella Gardner. It was authored by Okakura Kakuzo, a Japanese philosopher, art expert, and curator. Little known at the time, Kakuzo would emerge as one of the great thinkers of the early 20th century, a genius who was insightful, witty and greatly responsible for bridging Western and Eastern cultures. Nearly a century later, Kakuzo s The Book of Tea is still beloved the world over. Interwoven with a rich history of tea and its place in Japanese society is poignant commentary on Eastern culture and our ongoing fascination with it, as well as illuminating essays on art, spirituality, poetry, and more. More of my list is at Goodreads.

30 December 2009

David Welton: Detecting BlackBerry JDE Version

Recently, I went back and added some preprocessor code (it's pretty much necessary in the world of J2ME) to ensure that Hecl would compile with older versions of the BlackBerry JDE. However, I also faced a problem: how to figure out what version of the JDE we're using. It could be my latest cold clouding my mind, but I couldn't find a simple way to do this. It never seems to be simple with the BlackBerry platform, unfortunately. I did, however, finally find a nice way to obtain this information programmatically: the bin/rapc.jar file, which ships with the JDE, contains a file called app.version, which, indeed, contains the version of the JDE in use. I hacked up this code to read it and print it out: <script src="http://gist.github.com/266010.js"></script>

25 December 2009

Joachim Breitner: Building arbtt for Windows

A friend of mine is interested in trying out the Automatic Rule Based Time-Tracker arbtt which I programmed. Unfortunately, he is using Windows and up to now, arbtt only worked on Linux. But as I wanted to check out Haskell s cross-platform abilities for a while, this was a good opportunity to do so. I don t have Windows installed myself (and did not plan to do so), so I did all this under WINE, the Windows compatibility layer, which works very well: It takes only a few minutes to install the Haskell Platform for Windows and then I was able to run wine ghc --make and cabal install. I played around with some simple programs and was surprised by these timings:
$ rm *.o *.hi; ghc --make fourfours.hs ; time ./fourfours > /dev/null
[1 of 1] Compiling Main             ( fourfours.hs, fourfours.o )
Linking fourfours ...
real	0m1.909s
user	0m1.692s
sys	0m0.208s
$ rm *.o *.hi; wine ghc --make fourfours.hs ; time wine ./fourfours.exe > /dev/null
[1 of 1] Compiling Main             ( fourfours.hs, fourfours.o )
Linking fourfours.exe ...
real	0m1.631s
user	0m1.376s
sys	0m0.092s
So it is faster to run a compiled Haskell program on top of a compatibility layer than directly on Linux! The world is in order again, though, if optimization is enabled:
$ rm *.o *.hi; ghc -O --make fourfours.hs ; time ./fourfours > /dev/null
[1 of 1] Compiling Main             ( fourfours.hs, fourfours.o )
Linking fourfours ...
real	0m0.981s
user	0m0.876s
sys	0m0.108s
$ rm *.o *.hi; wine ghc -O --make fourfours.hs ; time wine ./fourfours.exe > /dev/null
[1 of 1] Compiling Main             ( fourfours.hs, fourfours.o )
Linking fourfours.exe ...
real	0m1.270s
user	0m1.036s
sys	0m0.072s
Funny. Anyways, I wanted to port arbtt. The only platform-dependent part is the capture module that gathers the list of open Windows. The Win32 package that comes with the Haskell Platform did not cover all the functions needed to do so, but creating additional function bindings is really easy with Haskell, as can be seen in the Graphics.Win32.Window.Extra module. I also replaced the locking code that prevents two instances of arbtt-capture to run at the same time by equivalent code using Windows mutexes (module System.Win32.Mutex). With these small changes and some CPP conditionals to make the code compile for either platform, the porting was done! Even accessing the files in ~/.arbtt works correctly on Windows, where it will look in the Application Data folder, without changing the code, thanks to System.Directory.getAppUserDataDirectory. But Windows users won t like compiling software on their own. They won t even like installing software by copying various files to certain directories. Therefore, I also had to create a Windows Installer. I picked Inno Setup, because it s Free Software and seems to be simpler than NSIS. The installer puts the compiled .exe files, the example categorize.cfg and the HTML documentation in the right spot, adds icons to the Start Menu ( Edit categorize.cfg , which fires up wordpad, a link to the documentation and the uninstaller), puts arbtt-capture in the Autorun folder, puts the path to arbtt-stats in the PATH variable and starts arbtt-capture at the end (the last three points being optional). Of course it undoes all this when removing the program again. I integrated the call to the Inno Setup installer into the usual ./Setup build process of Haskell packages. Some more details of how to create the Windows installer are mentioned in the README file. Now all this does not magically add a graphical user interface to arbtt, so users will still have to work with arbtt-stats on the command line even on Windows. If this is not a problem for you then you can fetch the latest installer from the arbtt homepage. And if you happen to become a serious user of arbtt on Windows and want to help maintaining the Windows port, I ll gladly share some responsibilities. I m very satisfied with the process and the result and I m happy to know that I can offer some of my programs also to Windows users in the future.It is also a big plus for Haskell with Python, shipping a program for Windows users is likely more difficult. The next step will be providing gtk-based graphical Haskell applications for Windows, including a nice installer that ideally includes all dependencies (gtk etc.).

Martin-&#201;ric Racine: Bad, bad Linux kernel!

Ever since Linux kernel 2.6.31 was released, my AMD Geode LX800-based FIC ION 603 can no longer boot; I get a fatal kernel crash near the end of the initramfs loading phase. At the crux of the issue seems to be a recently added kernel feature for caching filesystem ACL. I don't even use the ACL feature on my hosts, but there you go; it still crashes the kernel. Sigh. Having filed a bug on Launchpad and reported the issue upstream failed to produce a fix - despite the sustained involvement of two members of the Ubuntu kernel team members to help me narrow down the cause of this fatal kernel crash. Mentioning the issue to OLPC kernel developer Andres Salomon, he commented how he finds the new ACL caching code in the ext2/ext3/ext4 filesystem drivers downright creepy, but added that he currently lacks the time to further investigate the issue. I'm thus wondering if anyone else is experiencing this issue and might possibly have a fix to propose? Alternately, help towards debugging and fixing this issue is extremely welcome, as this bug will seriously affect users upgrading from older Debian and Ubuntu releases based on pre-2.6.31 kernels, when Debian/Squeeze and Ubuntu/Lucid are published in spring. PS: yes, the issue still exists in kernel 2.6.32 also.

Steve Kemp: And if someone gets upset you say, "chill out"!

It was interesting to see Clint Adams describe love and dissatification with configuration management. At work I've got control of 150(ish) machines which are managed via CFEngine. These machines are exclusively running Debian Lenny. In addition to these hosts we also have several machines running Solaris, OpenBSD, and various Ubuntu releases for different purposes. Unfortunately I made a mistake when I setup the CFEngine infrastructure and when writing all the policies, files, etc, I essentially said "OK CFEngine controlled? Then it is Debian". (This has been slowly changing over time, but not very quickly.) But in short this means that the machines running *BSD, Solaris, and non-Debian distributions haven't been managed as well via CFEngine as the rest, even though technically they could have been. A while back I decided that it was time to deal with this situation. Looking around the various options it seemed Puppet was the way of the future and using that we could rewrite/port our policies and make sure they were both cleanly organised and made no assumptions. So I setup a puppetmaster machine, then I installed the client on a range of client machines (openbsd, debian lenny, ubuntu, solaris) so that I could convince myself my approach was valid, and that the tool itself could do everything I wanted it to do. Unfortunately using puppet soon became painful. It has primitives for doing various things such as maintaining local users, working with cronjobs, and similar. Unfortunately not all primitives work upon all platforms, which kinda makes me think "what's the point?". For example the puppet client running upon FreeBSD will let you add a local user, setup a ~/.ssh/authorized_keys file but will not let you setup a password. (Which means you can add users who can login, but then cannot use sudo. Subpar) At this point I've taken a step back. As I think I've mentioned before I don't actually do too much with CFEngine. Just a few jobs: (i.e. I don't use cron facilities, I add files to cron directories. Similarly I don't use process monitoring, instead I install the monit package and drop /etc/monit/monitrc into place.) There is a pretty big decision to make in the future with the alternatives being: If we ignore the handling of local users, and sudo setup, then the tasks that remain are almost trivial. Creating a simple parser for a "toy-language" which can let you define copies, edits, and package operations would be an afternoons work. Then add some openssl key authentication and you've got a cfengine-lite. For the moment I'm punting the decision but I'm 90% certain that the choice is CFEngine vs. Chef vs. In-House-Fu - and that puppet is no longer under consideration. Anyway despite having taken months to arrive at this point I'm going to continue to punt. Instead my plan is to move toward using LDAP for all user management, login stuff, and sudo management. That will be useful in its own right, and it will coincidentally mean that whatever management system we do end up using will have on less task to deal with. (Which can only be a good thing.) ObFilm: Terminator II

Steve Kemp: 907

It was interesting to see Clint Adams describe love and dissatification with configuration management. At work I've got control of 150(ish) machines which are managed via CFEngine. These machines are exclusively running Debian Lenny. In addition to these hosts we also have several machines running Solaris, OpenBSD, and various Ubuntu releases for different purposes. Unfortunately I made a mistake when I setup the CFEngine infrastructure and when writing all the policies, files, etc, I essentially said "OK CFEngine controlled? Then it is Debian". (This has been slowly changing over time, but not very quickly.) But in short this means that the machines running *BSD, Solaris, and non-Debian distributions haven't been managed as well via CFEngine as the rest, even though technically they could have been. A while back I decided that it was time to deal with this situation. Looking around the various options it seemed Puppet was the way of the future and using that we could rewrite/port our policies and make sure they were both cleanly organised and made no assumptions. So I setup a puppetmaster machine, then I installed the client on a range of client machines (openbsd, debian lenny, ubuntu, solaris) so that I could convince myself my approach was valid, and that the tool itself could do everything I wanted it to do. Unfortunately using puppet soon became painful. It has primitives for doing various things such as maintaining local users, working with cronjobs, and similar. Unfortunately not all primitives work upon all platforms, which kinda makes me think "what's the point?". For example the puppet client running upon FreeBSD will let you add a local user, setup a ~/.ssh/authorized_keys file but will not let you setup a password. (Which means you can add users who can login, but then cannot use sudo. Subpar) At this point I've taken a step back. As I think I've mentioned before I don't actually do too much with CFEngine. Just a few jobs: (i.e. I don't use cron facilities, I add files to cron directories. Similarly I don't use process monitoring, instead I install the monit package and drop /etc/monit/monitrc into place.) There is a pretty big decision to make in the future with the alternatives being: If we ignore the handling of local users, and sudo setup, then the tasks that remain are almost trivial. Creating a simple parser for a "toy-language" which can let you define copies, edits, and package operations would be an afternoons work. Then add some openssl key authentication and you've got a cfengine-lite. For the moment I'm punting the decision but I'm 90% certain that the choice is CFEngine vs. Chef vs. In-House-Fu - and that puppet is no longer under consideration. Anyway despite having taken months to arrive at this point I'm going to continue to punt. Instead my plan is to move toward using LDAP for all user management, login stuff, and sudo management. That will be useful in its own right, and it will coincidentally mean that whatever management system we do end up using will have on less task to deal with. (Which can only be a good thing.) ObFilm: Terminator II

23 December 2009

Gunnar Wolf: Preparing for the geeky holidays with suitable wine

Preparing for the geeky holidays with suitable wine
Quoting The Klezmatics: It's everybody-else's annual end-of-the-year-time holiday, the name of which respectfully we do not choose to say, but that does not prohibit us in any concievable way from wishing you a very merry everybody-else's end-of-the-year-time holiday!
In order to properly welcome this 2010 in a geeky fashion, I got the following wines to share with my friends: Happy $ joyful_ocassion !

Benjamin Mako Hill: What's in a name?

Over the summer, there was a bit of a tussle at the highest level of Ubuntu governance over whether or not Canonical Ltd., the company that funds the majority of work done directly in Ubuntu, should name its file syncing and backup service Ubuntu One. Canonical's service involved a freely licensed client included in the Ubuntu distribution but, as a network service running on Canonical servers, it was not clearly a part of Ubuntu (the GNU/Linux distribution) or Ubuntu (the community) in the way the term was usually used within the community. Although the network service itself was not Franklin Street Statement free, this was not the most important issue for everyone who objected to the name. The major issue for many seemed to boil down to the fact that, free or not, Ubuntu One is a service run entirely by Canonical outside the reaches of the Ubuntu governance structures. Decisions were made and not everybody -- and maybe not anybody -- was absolutely happy with the outcome. My goal is not try to revive old arguments here. As far as I'm concerned, the issues are settled and the service is called Ubuntu One. That said, the questions raised during the episode are fundamental to Ubuntu and to other firm-sponsored FLOSS projects. Now that the dust has settled, they are worth reflecting on. From a legal perspective, there never was any ambiguity. Canonical "owns" the Ubuntu trademark. In this important sense, "Ubuntu" means whatever Canonical says it means. This is hardly new. As just one example, the Official Ubuntu Book (of which I am an author) was written by community members but became official because Canonical blessed it. But despite the fact that they don't need to, Canonical has often consulted with the community and its governance structures about trademark licensing policy. This was also not a case of Canonical not listening to the community. Canonical employees approached the Ubuntu Community Council (Ubuntu's highest governance board of which I am a member), listened carefully to concerns, and responded thoughtfully. The question was not even about a clash between what Canonical and the CC thought about the issue. An unambiguous majority of the board, including all the Canonical employees and several of the community members, supported the idea of Canonical using the trademark. The question was one about who gets to make the decisions about the Ubuntu name and about what role the community and Canonical would each play. Despite the fact that a majority of the Ubuntu community council was likely to support the proposed name, the CC was told by Canonical CEO Mark Shuttleworth that a vote was irrelevant. Canonical made it clear that decisions about the Ubuntu trademark were simply not in the CC's purview. The decision on how the name was to be used was something that Canonical was not willing to delegate to an "outside" (the firm) governance body. Few businesses would. And although I don't agree with the decision as a community representative, I might have even made the same call from Canonical's shoes. In traditional firms, it's usually pretty clear where the organization's boundary lies. In FLOSS projects -- and especially in FLOSS project like Ubuntu who are sponsored in very large part by a single for-profit company -- boundaries are fuzzier. The conversation about "Ubuntu One" can be seen as a fight over what "Ubuntu" refers to, and, more importantly, who gets to answer that question. In deciding whether to call a service "Ubuntu", a decision is made on what Ubuntu is. Names are powerful. Is Ubuntu just a Canonical project? Are Ubuntu's contributors really just Canonical contributors by proxy? I think the answer to both questions is "no" but the boundary issues involved are complicated and under constant negotiation. Every time Canonical uses the Ubuntu name itself or grants others the ability to do so, these boundary issues are negotiated, one way or another. This boundary setting work reveals an important tension that firms releasing FLOSS must all struggle with. To what extent and in what ways do communities get to decide what a FLOSS project is and to what extent do sponsoring firms get to do so? How should projects and firms do this most effectively? What should we even be optimizing for? I think that any resulting balance has a huge effect on whether a FLOSS project is, on one hand, released under a free license but run like any old corporate project or, on the other, a true "bazaar" style project where no single firm dominates -- or where they fall on that spectrum. Names and trademarks are one way that projects define their own identities and act as an important frontier in this balancing act. As every firm/project negotiates their own answers to questions of names and boundaries, there are important implications for the project's ability to attract volunteers, solicit contributions from other firms, and more. The confusion around conversations about Ubuntu One shows that we still have a lot to learn.

21 December 2009

Robert Millan: GRUB i18n


Better late than never. We promised internationalization and here comes GRUB with gettext support. Many thanks to Carles Pina for bringing GRUB a bit closer to end users. Here s GRUB menu in Catalan: GRUB menu And here s the Catalan version of GRUB command-line interface: GRUB command-line Help is much welcome from translators who want to add their own languages! Like many other GNU projects, translation support for GRUB is coordinated by the Translation Project. If you re excited by the idea that your strings will be the very first localized message that is displayed on every user s box, or simply want to bring our cause for computer freedom further, get in touch with your language team.

Daniel Kahn Gillmor: dd, netcat, and disk throughput

I was trying to dump a large Logical Volume (LV) over ethernet from one machine to another. I found some behavior which surprised me.fun constraints(how do i get myself into these messes?)This entry has been truncated read the full entry.Tags: lvm, netcat

20 December 2009

Steve Kemp: You're gonna need a bigger boat.

I'd like to nominate this week of the year to be : (I'd pick a single date but the only date I care about is March 10th, and that has already been assigned National Fishnet Day.) So, national backup week? This week is the week I'd like to suggest that you ensure that all the hosts, systems, and machines you care about are backed up. Maybe not today, maybe not tomorrow, but soon they will die. If you have a system die without a backup you're screwed. If this week is a quiet time at work, or if you're having time off, take an hour, take a few, and ensure you have backups .. current backups. ObFilm: Jaws

Stefano Zacchiroli: RC bugs of the week - week 15

RCBW - week #15 Here are last weektoday squashes, by yours truly: Some highlights:

Wouter Verhelst: QNAP NAS device

A customer called me a while back. They have a pretty high-end NAS (2U server with external disk enclosure) that they use for most of their needs: both document and other storage for management types, as well as storage for their EDA cluster. People familiar with EDA tools will know that they are pretty disk intensive. They tend to require shitloads of diskspace, preferably at high performance. The problem was that extending the storage space of this high-end NAS device is rather expensive. Supported disks only come at 300G units with prices that are higher than the cheap TB disks available today for the desktop market; and while the price and performance of those disks is worth it for the EDA requirements, the number of people storing documents and similar on the same device is not so high that performance would be an issue. Thus, what they wanted was a cheap solution to augment, rather than replace, their current storage solution, with the focus being on low cost at the possible expense of performance and, since bacula runs well there, reliability. They were quite surprised when I offered them a solution based around a QNAP device; they had expected a price, without hard disks, of about three to four times the price of the QNAP, and therefore were very happy with this suggestion. Unfortunately, the devices produced by this manufacturer that are supported by Lenny, the current Debian stable release, were EOL'd just before we placed the order; however, their replacements will be supported by Squeeze, the next release, and with a bit of help from Martin Michlmayr, we were able to install Lenny with a somewhat more recent kernel. This drove the price up somewhat, both because the newer devices are a few tens of euros more expensive, and because we agreed to pay Martin to prioritize work on the specific device that we'd need, so that we wouldn't have to wait for him to get around to it. As of this Thursday, the device, which had finally arrived after some initial delays, has been installed. Unfortunately, one of the disks that they'd bought was DOA; but they had the foresight to buy five, rather than four, disks, so that was not a major problem. In fact, the only real issues that I ran into were this one arm-specific bug, and the fact that the high-end NAS device is still running etch, which means that rsync --acl won't work. That means I'll have to go back there soon to do an upgrade of the high-end device (which had been planned for a while already); for the arm-specific bug, a workaround is already in place. All in all, a pretty good experience.

Francois Marier: Debugging logcheck rule files

logcheck is a neat little log file monitoring tool I use on all of my machines.

I recently noticed however that I hadn't received any logcheck messages in a while from one of my servers. Either that was a sign that things were going really well or, more likely, that logcheck wasn't producing any output anymore.

Manually logging an error to syslogHere's what I did to force a message to be printed to the logs:
logger -p kern.error This is a test
Which I would expect to produce this logcheck notice:
Dec 20 15:34:08 hostname username: This is a test
Unfortunately, that didn't happen on the next scheduled run.

Forcing a logcheck runTo rule out the following:
I ran logcheck manually:
sudo -u logcheck /usr/sbin/logcheck -o -d >& logcheck.out
Looking at the output file however, my test message still wasn't there. Either logcheck was broken or one of my rule files was swallowing everything.

Finding the broken rule fileTo find the broken rule, I started by ignoring rules defined in /etc/logcheck/ignore.d.server/ and /etc/logcheck/ignore.d.workstation/ by running logcheck in paranoid mode:
logger -p kern.error This is a test
sudo -u logcheck /usr/sbin/logcheck -o -d -p >& logcheck.out
This worked, so I then ran logcheck in server mode:
logger -p kern.error This is a test
sudo -u logcheck /usr/sbin/logcheck -o -d -s >& logcheck.out
Given that this also worked, it meant that the offending rule file was in /etc/logcheck/ignore.d.workstation/. So I moved all of my custom local-* rule files out of the way and ran logcheck in workstation mode:
logger -p kern.error This is a test
sudo -u logcheck /usr/sbin/logcheck -o -d -w >& logcheck.out
Once I verified that this worked, I started to put my local files back one by one until it broke again. Then slowly removed lines from the offending file until it worked.

Solution to my problemIt turns out that one of my rule files had a line like this:
path != NULL   column != NULL
Escaping the pipe symbols with backslashes solved the problem:
path != NULL \ \  column != NULL

Maybe I should periodically print a message to syslog to make sure that logcheck is still working...

18 December 2009

Sandro Tosi: Remote Apache logging with syslog: is there anything better?

We're doing a pilot to do remote logging for some Apache logs (possibly other services in the future). We've heard of remote syslog capability (and since syslog is on all Linux system) we're giving it a try.

The configuration is quite simple:
  1. [srv] prepare a machine to do the log server
  2. [srv] open it's rsyslogd to receive messages on UDP (or TCP) port
  3. [srv] log the selected facility.level to a log file
  4. [cli] forward the above facility.level to @srv<srv address="" ip=""></srv>
Ok, easy and it works. Unfortunately we face pretty fast the limitation of syslog:
  1. only 8 facilities for users custom log (local0-local7)
  2. only 8 levels for logs severity
this is a big block (only 64 combinations, if you agree to do some "dirty" stuff) if you want to log remotely several services on several different platform on the syslog server.

rsyslog is quite flexible and it allows you to filter messages based on the tag in them, and log in different files, but it's still something very "home-made" and not that professional.

I don't we're the only one needing a remote logging tool, and while syslog is the classic solution, is very bind to the system logs and not to the applications logs: any suggestions for this task? I'd like to hear how you solved this task, possibly without a custom tool, but using something standard.

17 December 2009

Vincent Sanders: What is it with embedded computing?

Last week I was given a development board for a system on chip (SoC) we were looking at using. All i had to do was ensure I could get a reasonable, supportable, bootloader and Linux kernel running on it.

Now I am a fairly competent engineer, I have worked with the ARM processors and SoC for many years. It should not therefor take me four days of almost continuous frustration to get a bootloader and kernel onto a system, but it has.

Firstly the system arrives and we look it over, the hardware design is not exactly elegant but seems like it will get the job done. That is at least a step up from several previous offerings which have been just plain broken. So I power it up and it boots into Fedora core 8, possibly not my first OS choice for a diskless system, but it does work.

Perhaps its worth mentioning that my experience with doing any Open Source development has caused me to have a severe aversion to software which is not upstream, especially kernels. The effort required to run your own patch series and keep it up to date is nothing short of Herculean.

Let us be honest, how many developers do you know who are talented enough to generate good code and patient enough to separately maintain it over a long period of time? I shall not belabor this point as others have repeatedly made it for me in much more eloquent ways. Lets us just say from my point of view. Upstream - good. Long term forks - bad.

With this in mind, the solutions Simtec develop are always with a view to our customers being able to simply download and use the standard upstream software wherever possible. Here is where the story stops being smooth and becomes a rant.

The system was running 2.6.22, one that was patched all which ways. "maybe they just shipped an old kernel" I thought. I opened the CD they provided and...oh, OK its just zips and tarballs of the binaries and source they used (actually that in itself is a minor miracle with a lot of vendors right now - see the almost continuous stream of copyright infringement litigation from the FSF) but sill stuck back in July 2007.

So i did teh normal thing, entered the URL printed on the outside of the box and went looking for the updates. No dice, just the same old stuff. So I contacted the company and recived the news that no there were no updates, that was what was published for the device, thats what I could have.

My initial reaction was of tired resignation...this would not be my first SoC port (probably around the fifth or sixth!) So I responded with cd linux-2.6;git checkout master;git pull and went to see what I could see. and lo and behold, there in the kernel, was full support for not only the SoC but for the other version of the very board I had.

Fortunately after mentioning this on IRC my giraffe loving friend. pointed me at the correct community git tree to find the tiny patch required to support my board.

So the vendor was not only shipping old code but actively dissuading customers from even looking for new stuff, which it turns out was mainline anyway! To add to this somewhere along the lines the company name on the box and contact information has changed and...well confusion reigned.

Right Plain sailing from here on? not quite...that would have been too easy. So you can build a mainline kernel...but you cannot boot it. you guessed it the bootloader they ship on the board is ancient, specifically a u-boot fork from December 2005. Having finally figured out how this was going to go, I went to the uboot repo, pulled the latest and built it for my target. And then I needed to upgrade the u-boot and joy of joys bricked the system.

Fortunately the system has a integrated FDTI serial/JTAG port so re-writing the flash should be as simple as running openOCD configured correctly. Of course this did not work...in the end I resorted to running their binary built versions from the CD (tarfile inside a zip inside a tarfile...whatever) and wrote their original image back to the board...which worked.

Long story short, next day I discover that i need a magic different uboot target (make u-boot.kwb for those that care) write this with the binary openocd build and...success
From then on its all been a bit anticlimactic and straightforward and I now have a system capable of starting a kernel over the network and a root fs on network, disc or flash.

My main complaint from all of this is what the hell are vendors doing shipping crap like this? Why do they all insist on taking a dreadful code drop from the SoC manufacturer when they first brought the part up and shipping that prototype junk until the part reaches the end of its life?!?

All the usual excuses about support etc. will come out, but seriously, you recive *no* support from vendors unless you pay for it and even then they will simply tell you whats on the CD is "it". Btw the diff for u-boot to support this board is larger than the original sources and the 2.6.22 is attacked with a 65MB diff which cannot be good.

The Open Source option is continuously innovating and improving, producing ever better software...but the embedded world seems to be forever stuck shipping crap. Is this just me? Does anyone have any solutions? Or am I stuck with this until I change profession?

15 December 2009

Russell Coker: Play Machine Online Again

I have returned from the US and my SE Linux Play Machine [1] is online again. It was unfortunate that I forgot to pack one of my Play machine shirts, I ended up attending a meeting of the SDForum [2] on the topic of Cloud Security (it was a joint meeting of the Cloud Services and Security SIGs) and it would have been good to have been wearing a root password.

14 December 2009

David Welton: File Selector for Java ME

I recently did some work to make Hecl read files, which also means that it can execute Hecl scripts from the phone's memory. This is especially important for environments like Blackberry, where we will be distributing a signed version of the Hecl application. To create your own Hecl applications, instead of simply replacing the script in the .jar file, you can point to a script on the device's file system. This is also available for normal Java ME phones, but unfortunately, for an open source project, the cost of a code signing certificate are prohibitive (on Blackberry, it's only $20, so I bought one with my own money). In any case, as part of this effort, I developed a very simple 'file browser', which is used in Hecl to select a script to execute. The results are, like all of Hecl, available under an Apache license, which means that you can use it pretty much wherever you want: http://github.com/davidw/hecl/blob/master/files/org/hecl/files/FileFinder.java http://github.com/davidw/hecl/blob/master/files/org/hecl/files/FileFinderCallback.java Of course, if you spot any ways to improve it or fix problems with it, I'd appreciate it if you sent patches back.

Ond&#345;ej &#268;ert&iacute;k: ESCO 2010 conference

An interesting conference 2nd European Seminar on Coupled Problems (ESCO) will be held on June 28 July 2, 2010 in Pilsen, Czech Republic.
Among the topics are solving PDEs and applications and using Python for scientific computing. In particular, Ga l Varoquaux is the keynote speaker.

Unfortunately, it was later announced that the SciPy 2010 conference is going to be at the same time, which is really unfortunate. But here are some reasons why you should consider going to ESCO 2010 instead:


If you have time, I can fully recommend to go to ESCO 2010.

13 December 2009

Russell Coker: Links December 2009

Dan Gilbert gave an insightful TED talk about our mistaken expectations of happiness [1]. Don Marti has an insightful post about net neutrality and public property [2]. When net access requires access to public property then it should be sold in a neutral manner. Rachel Pike gave an interesting TED talk about the scientific research behind a climate headline [3]. The people who claim to be skeptical of the science should watch this. Mark Peters wrote an interesting article A Happy Writer Is a Lousy Writer about the correlation between emotional state and work quality [4]. Apparently watching a film about cancer will make people more careful and focussed on details. CERIAS has an interesting short article about Firefox security as well as some philosophy on why web browser security generally sucks [5]. Cory Doctorow writes in the Guardian about Peter Mandelson s new stupidity in trying to legislate against file sharing [6]. This is going to seriously damage the economy of every country that implements it. Charles Stross has been blogging a series of non-fiction essays about space colonisation, in The Myth of the Starship he describes how most ideas of space travel are bad and how the word ship is always going to be unsuitable [7]. Brent T. White is an associate professor of law at the University of Arizona who has written an interesting paper about mortgages [8]. He says that anyone who is underwater (IE owing more than the value of their house) should walk away. The credit damage from abandoning a bad mortgage apparently isn t that bad, and there is the possibility of negotiating with the bank to reduce the value of the loan to match the value of the house. Mako is working on a project to allow prisoners to blog [9]. It s basically a snail-mail to web gateway as the prisonsers are not allowed Internet access. PracticalEthicsnews.com has an article about the special status that homeopathy is given [10]. It also notes that homeopathic medicines include arsenic and mercury. Such quackery should be outlawed, a life sentence for homeopathy would be appropriate IMHO. Cory Doctorow wrote an interesting essay about why he is not selling one of his books in audio form (he s giving it away) [11]. He concludes by noting that he wants no license agreement except don t violate copyright law . The fact that he can t get anyone to sell an audio book under such terms is a good demonstration of how broken the marketplace is. Thulasiraj Ravilla gave an inspiring TED talk about the Aravind Eye Care System a program to bring the efficiency of McDonalds to eye surgery [12]. Hopefully that program can spawn similar programs for other branches of medicine and spread to more countries. In many ways they are providing better service (both in quality and speed) than people in first-world countries who pay a lot of money can expect to receive. Scott Kim gave an interesting TED talk about his work designing puzzles [13]. He is also a big fan of social networking, unfortunately (for people like me who don t like social networking) his web site ShuffleBrain.com relies on Facebook. Gordon Brown (UK Prime Minister) gave an inspiring TED talk about global ethic vs the national interest [14], with a particular focus on the global effort required to tackle the climate change problem. Now if only we could get Kevin Rudd to listen to that. Brough has written an interesting analysis of the AT&T network problems that are blamed on the iPhone [15]. His essential claim is that the problem is due to overly large buffers which don t cause TCP implementations to throttle the throughput. This seems similar to my observations of the Three network in Australia where ping times of 8 seconds or more will periodically occur. One particularly nasty corner case with this is when using a local DNS server I can have a DNS packet storm where basic requests time out while BIND uses a significant portion of available bandwidth (including ICMP messages from receiving ports that BIND has closed). To alleviate this I am now using the Google public DNS service [16] (the Three DNS servers never worked properly).

Next.

Previous.