Search Results: "tar"

20 December 2009

Debian News: Updates on LXDE

Several LXDE components have been updated recently in sid. They bring some new features and improve flexibility of customization. Below you have a summary of the most important changes:

Andrew Lee

Francois Marier: Debugging logcheck rule files

logcheck is a neat little log file monitoring tool I use on all of my machines.

I recently noticed however that I hadn't received any logcheck messages in a while from one of my servers. Either that was a sign that things were going really well or, more likely, that logcheck wasn't producing any output anymore.

Manually logging an error to syslogHere's what I did to force a message to be printed to the logs:
logger -p kern.error This is a test
Which I would expect to produce this logcheck notice:
Dec 20 15:34:08 hostname username: This is a test
Unfortunately, that didn't happen on the next scheduled run.

Forcing a logcheck runTo rule out the following:
I ran logcheck manually:
sudo -u logcheck /usr/sbin/logcheck -o -d >& logcheck.out
Looking at the output file however, my test message still wasn't there. Either logcheck was broken or one of my rule files was swallowing everything.

Finding the broken rule fileTo find the broken rule, I started by ignoring rules defined in /etc/logcheck/ignore.d.server/ and /etc/logcheck/ignore.d.workstation/ by running logcheck in paranoid mode:
logger -p kern.error This is a test
sudo -u logcheck /usr/sbin/logcheck -o -d -p >& logcheck.out
This worked, so I then ran logcheck in server mode:
logger -p kern.error This is a test
sudo -u logcheck /usr/sbin/logcheck -o -d -s >& logcheck.out
Given that this also worked, it meant that the offending rule file was in /etc/logcheck/ignore.d.workstation/. So I moved all of my custom local-* rule files out of the way and ran logcheck in workstation mode:
logger -p kern.error This is a test
sudo -u logcheck /usr/sbin/logcheck -o -d -w >& logcheck.out
Once I verified that this worked, I started to put my local files back one by one until it broke again. Then slowly removed lines from the offending file until it worked.

Solution to my problemIt turns out that one of my rule files had a line like this:
path != NULL   column != NULL
Escaping the pipe symbols with backslashes solved the problem:
path != NULL \ \  column != NULL

Maybe I should periodically print a message to syslog to make sure that logcheck is still working...

19 December 2009

Stephan Peijnik: Rest in peace Flo: 13.11.1986 16.12.2009

Today is a sad day. Everything feels like I am having a bad nightmare. That's because today I learnt from the too early death of my friend Florian Hufksy.

I am sitting here and do not really know what to write. I keep on thinking about the great times we spent together. The time we started programming when we were twelve. The time we spent learning BASIC. All the times you knew more than me and could teach me a thing or two. I remember our geek talks. How we would discuss latest games. How we lost contact and how we met again. I am thinking about how sorry I am for not having met you often enough. I keep on trying to understand what drove you that far. How you could just end it all. More and more memories come to my mind, like the moment when you showed me one of your projects, Super Mario War. The moments we had playing video games together. All those moments, all that time, I miss you my friend. You were a genius, always a step ahead, not only of me, but seemingly the whole world. I can't stop thinking about your brilliant ideas and how you always finished your projects. You were a real hacker, a real genius, a person trying to make the world a better place, a person who will be missed, not only by me.

You were a genius and I always respected you, not only as a hacker, but as a beloved friend. Why did we not spend more time together? Why did you have to go? Why do I have to write this now, sitting here in my chair with tears in my eyes? And all those memories come up again and again. There is so much more that comes to my mind, but I can't keep on writing, it just hurts too much.

The world is a sad place today. I am sad. I am mourning the too early death of my beloved friend, Florian. You will always have a special place in my heart.

Michael Banck: 19 Dec 2009

Tutorials at the Garching Debian Stammtisch Munich traditionally used to have a lot of Debian Developers, but over the last couple of years quite a few of us who used to be students graduated and moved elsewhere or became very busy with their day jobs. We still meet for having a beer and a chat, but not as much as some years ago. We used to meet about once a month, but in 2009 we only managed to meet four times (however, we organized a Bug Squashing Party in November and had a special meeting as Lenny Release Party in February) As the meetings are really rather informal and not necessarily very Debian related, it is difficult to attract new people this way. So Johannes Wiedersich and I decided to try a more hands-on approach by having a second meeting in Garching, in the student-run bar on the campus of the Technische Universit t M nchen (TUM). The idea was to get more of the local science, mathematics and computer science students (as well as possibly interested faculty members) involved. We tried a first time about a year ago, but after two or three meeting in late 2008 and early 2009, we lost momentum. However, we began organizing the meetings again with the start of the winter term, and had two rather successful meetings so far. For the first meeting, we basically handed out some information on how to get involved locally (the Debian-Munich list, its subscription address, the wiki etc.) and discussed Debian in general and Debconf11 in Munich in particular. About half a dozen people showed up, and two of these attended the Bug Squashing Party later that month, and another one (a faculty member) got very active in Debconf11 organization. Thus, I was quite happy with the outcame of that meeting. Some days ago, we had another meeting, and this time I was doing a live-tutorial on Debian package building. As Johannes was ill, we did not manage to announce or publicize the meeting well in advance, so only three people showed up. Still, I think it went rather OK, and we will be doing another Debian package-building tutorial for the next meeting, and possibly other turorials/workshops afterwards (ideas so far include library maintenance, Debconf, how the Debian community is organized and how to get involved in it). As doing a live-tutorial on one notebook is a bit difficult if you both have to type on it and people should see what happens, we will either use some extra hardware next time, or move to some nearby seminar room with a projector, this will be announced in advance. So if you are on Garching campus or nearby and interested in Debian development (and Debian package-building in particular), come to the next meeting on January 13th! We decided to meet on the second Wednesday of each month, at 18h. Subscribe to the Debian-Munich list to get the invitation or watch out for the flyers on the campus.

Russell Coker: Shared Objects and Big Applications

Some time ago I wrote a little utility named memlockd [1]. Memlockd will lock files into memory which allows significantly faster access when the system pages heavily, in my simulated tests I have found that having the programs and shared objects needed for logging in locked in memory can make it possible to login without a timeout when there is heavy paging, this can make the difference between recovering a system with some processes that are out of control and having to reboot it (often without discovering the root cause). As always happens some people use my software in ways that I never planned. One guy is using it to try and make OpenOffice.org load faster. I m not sure that this is a good idea. In a typical installation when configured for the purpose that I intended it (system recovery from a rabbit process) memlockd will take a bit less than 10M of RAM on an i386 platform (that is for bash, login, sshd, getty, busybox, and all necessary shared objects and a few data files. Since RHEL 4 Red Hat distributions have whinged at boot time if there was less than 256M of RAM available, installation of a Red Hat based system on anything less than 128M of RAM has been impossible for some years, and Debian systems perform very poorly with less than about 128M of RAM when you run apt-get. I initially designed memlockd to run on my SE Linux Play Machine which has 128M of RAM in it s current incarnation. Locking 7.5% of RAM on the system may impact performance, but as a large part of that RAM is used for things like libc and bash which tend to be partially paged in at all times this shouldn t be a noticeable impact. But locking 100M or more of OpenOffice seems more likely to have the potential to hurt performance, I often run OpenOffice on a machine with 512M of RAM and the biggest desktop machine I use has 1.5G of RAM for me it wouldn t make sense to lock OpenOffice into memory. But it could be that there is some unusual aspect of his system that makes running memlockd with OpenOffice likely to give worthy benefits in performance without significantly hurting performance for other programs, for example it could have 4G of RAM and a really slow disk. It is also a possibility that the usage of the system makes OpenOffice so much more important than other programs that any decrease in performance in other areas is not relevant. In any case I m happy to help people use my software to do unusual things so I ll support this use. I ve been asked why memlockd doesn t seem to give much benefit when starting up OpenOffice when run with +/opt/openoffice.org3/program/soffice.bin in the config file, where the + means to lock all shared objects that ldd reports that the binary needs. for n in ldd /usr/lib/openoffice/program/soffice.bin sed -e s/^.*=..// -e s/\ .*// sort -u ; do readlink -f $n ; done sort -u > ldd.txt I used the above command to get a list of all shared objects that ldd reports for the soffice.bin program. On my system (Debian/Lenny i386) it reports 77 shared objects loaded. When memlockd locks all those ps reports that the RSS is 51944K. cat /proc/1234/maps sed -e s/^.*\ // sort -u > /tmp/map.txt Then I used the above command to get a list of the files that are memory mapped by OpenOffice when running OpenOffice calc where 1234 is the PID of the soffice.bin process (I expect that the numbers will be similar for writer, impress, etc I just happened to have a spreadsheet open). It reports 172 memory mapped files which include 9 files related to fonts and 64 shared objects under /usr/lib/openoffice/program which are not found by ldd among other things. It s quite common for a large application to use dlopen(3) at run-time to map shared objects instead of linking against them. Running memlockd with this list gave an RSS of 118644K, which is more likely to give a useful performance boost to OpenOffice load times.

MJ Ray: Similarities between the co-op and Google

This is a bit of a silly note for a Saturday while I m currently working on translating the co-op s web pages into Esperanto. The old site was translated, but the new site launched as English-only. I d like to have other languages too, but I ll start with the one that s easiest for me as a test case to get the internationalisation working again. That means that I m currently lost in django-blocks language settings. Apparently both our co-op and Google speak Esperanto. Facebook is mostly translated. Most Plone-based sites have some translation, but often it s only the standard Plone parts and any customisations remain in English, which looks a bit odd. Have you seen Esperanto on other major websites where you weren t expecting it?

Robert Collins: Debianising with bzr-builddeb


Bzr build-deb is very nice, but it can be very tricky to get started. I recently did a fresh debianisation of a project that is in bzr upstream, and I thought I d record the recipe to make it work (at least until the various bugs making it hard re fixed). Assuming that the upstream uses bzr, it goes like this:
  1. Start with a branch that is close to the code you want to Debianise. E.g. if the release was off trunk, 3 commits back: bzr branch trunk -r -3 debian
  2. Debianise as normal: put the tarball with the right name in the parent dir, add a debian directory and fiddle until you build a package you re happy with. Don t commit while doing this.
  3. Build a source package- debuild -S, or bzr builddeb -S
  4. Revert your changes bzr revert.
  5. Import the dsc bzr import-dsc ../*.dsc
  6. Now, you may find that some dot files, such as .bzrignore have been discarded inappropriately (there is a bug open on this). If that happened, keep going. Otherwise, you re done: you can now use merge-upstream on future upstream releases, and debcommit etc.
  7. bzr uncommit
  8. bzr revert .bzrignore (and any other files that you want to get back)
  9. debcommit
  10. All done, see point 6 for details.
Hope-this-helps

18 December 2009

Asheesh Laroia: Diversity in Free Software: South Asians as an example

As someone born in India, I sometimes look around and wonder, Where are the Indians (and other South Asians) in Free Software? (I don't mean to exclude South Asians from other countries, so I will lump us together. I believe that we are more similar than we are different, although I know more about India than about the rest of South Asia.) There is no shortage of Indians performing information technology jobs in the United States. The same is true in academia; the Computing Research Association uses National Science Foundation data to show about 15% of computer science bacholor's degrees are awarded to "Asians or Pacific Islanders." These are not precise numbers targeted at South Asians in particular, but they confirm a general feeling that plenty of technologists in the United States are from that part of the world. South Asia is quite a populous region, coming in at over one billion people. It, too, has plenty of technology workers. So much FLOSS conversation happens in English, and India is well-suited to handle this; English is an "official language". Indian academia reports that there are 350 million English users and about 90 million English speakers. So let's visually compare the Debian developers map for South Asia (over one billion people) and that of New Zealand, a country of four million. India: New Zealand: These two countries have about the same number of Debian developers (at least, who have marked their location in the Debian LDAP database). About four. South Asians comprise about one sixth of the world's population. There are about one thousand Debian developers; we represent at best 1% of that. These numbers are comparable to the under-representation of women in Free Software, especially when you compare the figure to South Asians' over-representation in the rest of information technology. That makes me sad. Take a look at the Debian developer map again. You'll see that Debian is certainly not an Americans-only project, or even an English-speakers-only project. South America has a respectable dotting of developers, and Western- to Central-Europe are packed. I have strong feelings about Free Software. It emerges from an ethos of personal empowerment, and with open source it has become a dominant force in computing. Yet there are plenty of sharp people -- at least women and South Asians -- who, somehow, become culturally excluded from participating.

Why care about diversity? Consider the diversity of contributors we already have. Some contribute to Free Software because of particular business needs, such as what caused Avi Kivity to write KVM, the new leader in Linux-based virtualization. Everaldo's art background gave us the "Crystal" icon set that set the standard for sharp-looking icons on the Free Desktop for years. Josh Coalson knew about compressing sound, and his Free Lossless Audio Codec is now the standard in high quality audio. We already have a great deal of diversity. We should be celebrating! Back in 2001, FLAC's users were celebrating. In that year, I decided to ditch proprietary operating systems because I felt I could achieve all my computing needs in the Free world. A happy user of FLAC myself, I lurked on the mailing list as I watched grateful people thank Josh for the great software he wrote. Different contributions will excite different sorts of users. The more different people we have improving FLOSS, the more happy users we can make. Happy users of FLOSS are Free users. Happy users can become contributors, putting forth code, documentation, translations, and word-of-mouth marketing. The first reason to improve diversity in FLOSS is to better suit our users' needs. The more diversity we have in our contributors, the more chance we have of tickling our users in the ways that please them the most. I wish to see an end to software that restricts users' freedom, so I want to see us build the tools that users want. One thing that pleases me is when I see other people contributing who seem similar to me. When I went to Debconf, I was thrilled to be surrounded by people who cared about software freedom and technical excellence. I had even more fun being social, chatting about rainforests, mutual friends, websites, and music. I might have had the most fun playing the card game Mao. A second reason, then, to improve diversity in FLOSS is to increase contributor retention by increasing joy. Mao was an example of a cultural bond I happened to share with a handful of Debianites. The more diversity we have, the more frequent these sorts of coincidences will be. The final, most obvious, reason to reach out to groups of people who do not typically contribute is that we can increase our numbers. That by itself is so valuable. Ubuntu sees 100 new bugs per week, even after the bug squad's efforts. If we can do a better job of recruiting new contributors, the raw numbers give us more strength in creating and maintaining world-class software as well as letting the world know about it.

Changing the balance I believe that there are plenty of South Asians quite capable of contributing to FLOSS. I believe the same of women. I believe the same of men. Back to the topic at hand. Why do the South Asians vanish when we look at Free Software, not tech in general? There are plenty of reasons I can dream up, based on my experience with Indians.
  • Plenty of South Asian parents urge their children to make low-risk career choices.
  • My mother reports that schools in India focus on memorization instead of creativity. This can leave little room for extracurricular pursuits.
It's tough for FLOSS advocates to work directly on these distant issues. But I think we can focus some problems we can help solve. Crucially, awareness of Free Software spreads best by social circles. I learned about Linux from a friend at a summer camp. I'll repeat that:
  • Awareness of Free Software spreads best by social networks.
So if you want to spread that awareness, try to be a bridge. If you meet someone from an unusual background for open source who needs support or mentorship, try to help. That is an investment in the diversity and growth of Free Software. Those people can now unlock more "open source minorities."

What success looks like Google Summer of Code helps some new contributors get started and provides that mentorship. Rachel McCreary was invited to the SciPy conference after a successful summer. Her father left a comment explaining how her sisters participated in FLOSS via Google's Highly Open Participation (GHOP) Contest:
Rachel was inspired and motivated by BOTH of her little sisters, each completing six GHOP tasks (if memory serves).
GHOP and GSOC has been a game-changer for these girls. Rachel's younger sister is applying to schools such as MIT with an interest in a science major. The youngest daughter now has a Caltech poster on her wall with the intent to eventually attend.
Their proud Dad
Soon, these stories will be commonplace. Until then, we have work to do. (I'm still researching these topics. If you can help me find any sort of data to help me learn more about diversity in FLOSS, even if it seems like I wouldn't like it, leave a comment.)

Joey Hess: upstreams and packaging

As distributors we should not discourage upstreams that wish to generate binary packages themselves, rather we should cooperate with them, and ideally they will end up maintaining their stable release packages in our distributions.
-- Robert Collins I agree. Robert goes on to talk about the tendancy Debian (and apparently also Ubuntu) have to dislike upstream providing a debian directory. Funny thing is that we also like to say team maintenance is a good idea. An upstream who is doing packaging work is a readymade half of a team; if you write them telling them to rm -rf debian, you are turning away a team member. With a slap in the face. Worse, it's a team member who has demonstrated that they are capable of working in far more complex systems than a debian directory, since they wrote/maintain the software itself. BTW, for those who are skeptical of teams, on the basis that they dilute responsibility: A team consisting of an upstream developer and a distribution maintainer is inherently the more healthy sort of team, where each member has a well-defined area of expertise, but can also venture outside their area when needed. When I packaged FBReader for Debian, upstream already had their own packaging, and I worked with them to fix problems with it and make it something I could be happy maintaining. This was sometimes tricky, since upstream was also maintaining packages for maemo. Sometimes the tools were not ideal. It was still worth it. On the other side of the coin, d-i is an upstream for several distributions. If they told us we needed to rm -rf debian, we'd not have a lot of d-i left. tools I also agree with Robert that most of the trouble comes down to problems with tools. BTW, RPM does not have these problems, and in that world it's typical for upstream to provide a spec file. Much of the problem comes down to the crummyness of dpkg's source format, which cannot rename files, delete files, etc. That the source format directs us to 1970's source management (ie, tarballs and patches), instead of 21st century source management (ie, DVCS), doesn't help either. dpkg's 3.0 quilt source format tries to address the issue by removing any debian directory in upstream's tarball. I am not sure that this is at all the right approach; it makes it harder, not easier to work with upstream as a team. A better approach might be to consider anything that hardcodes the debian directory as having a bug. If upstream can easily package debs using a deb, or maemo, or ubuntu directory, you sidestep any potential conflict while still being able to work with them via a symlink. To put my time where my mouth is: Anytime debhelper prevents working with an upstream who wants to ship a debian directory; that is a bug. I will fix them. (Note that debhelper already provides a --ignore that can be used if upstream has provided a dehelper control file that you can't delete and don't want used.) Previously: Look who's packaging

Robert Collins: Why upstreams should do distribution packaging


Software comes in many shapes and styles. One of the problems the author of software faces is distributing it to their users. As distributors we should not discourage upstreams that wish to generate binary packages themselves, rather we should cooperate with them, and ideally they will end up maintaining their stable release packages in our distributions. Currently the Debian and Ubuntu communities have a tendancy to actively discourage this by objecting when an upstream software author includes a debian/ directory in their shipped code. I don t know if Redhat or Suse have similar concerns, but for the dpkg toolchain, the presence of an upstream debian directory can cause toolchain issues. In this blog post, I hope to make a case that we should consider the toolchain issues bugs rather than just-the-way-it-is, or even features. To start at the beginning, consider the difficulty of installing software: the harder it is to install a piece of software, the more important having it has to be for a user to jump through hoops to install it. Thus projects which care about users will make it easy to install and there is a spectrum of ease. At one end,
checkout from version control, install various build dependencies like autoconf, gcc and so on
through to
download and run this installer
Now, where some software authors get lucky, is when someone else makes it easy to install their software, they make binary packages, so that users can simply do
apt-get install product
Now some platforms like MacOSX and Microsoft Windows really do need an installer, but in the Unix world we generally have packaging systems that can track interdependencies between libraries, download needed dependencies automatically, perform uninstalls and so on. Binary packaging in a Linux distribution has numerous benefits including better management of security updates (because a binary package can sensibly use shared libraries that are not part of the LSB). So given the above, its no surprise to me to see the following sort of discussion on #ubuntu-motu:
  1. upstream> Hi, I want to package product.
  2. developer> Hi, you should start by reading the packaging guide
  3. (upstream is understandably daunted the packaging guide is a substantial amount of information, but only a small fraction is needed to package any one product.)
or (less usefully)
  1. upstream> Hi, I want to package product.
  2. developer> If you want to contribute, you should start with existing bugs
  3. upstream> But I want to package product.
Another conversation, which I think is very closely related is
  1. developer> Argh, product has a debian dir, why do they do this to me?!
The reasons for this should be pretty obvious at this point:
  • Folk want to make their product easy to install and are not themselves DD s, DM s or MOTU s.
  • So they package it privately such as in a PPA, or their own archive.
  • When they package it, they naturally put the packaging rules in their source tree.
Now, why should we encourage this, rather than ask the upstream to delete their debian directory? Because it lets us, distributors, share the packaging effort with the upstream. Upstreams that are making packages will likely be doing this for betas, or even daily builds. As such they will find issues related to new binaries, libraries and so on well in advance of their actual release. And if we are building on their efforts, rather than discarding them, we can spend less time repeating what they did and more packaging other things. We can also encourage the upstream to become a maintainer in the distro and do their own uploads: many upstreams will come to this on their own, but by working with them as they take their early steps we can make this more likely and an easier path.

Jurij Smakov: Slow boot

For a very long time my laptop (running sid for a few years now) has been booting kind of slowly. It looked like there was a 2-3 second delay every time a kernel module was loaded, which resulted in something like 3 minutes total boot time. Never bothered me enough, but yesterday I've finally got some time to investigate. After increasing the verbosity of udev logging in /etc/udev/udev.conf I've found lots and lots of messages like this in the log:
Dec 18 00:41:46 droopy udevd-work[425]: wait for '/sys/some/path/bInterfaceProtocol' for 20 mseconds
The only udev rule file containing a reference to bInterfaceProtocol was /etc/udev/rules.d/z60_libccid.rules with the following lines in it:
# last file created by the kernel, if this is present everything should be
WAIT_FOR_SYSFS="bInterfaceProtocol"
The libccid package itself is long gone (I don't remember installing it at all), and the versions of the package starting with 1.3.4-1 (uploaded in February 2008) do not contain this rule. However, the file survived on my system, so this rule was continuing to trigger for every device, even the ones not providing the bInterfaceProtocol at all. Removing the now-redundant file resulted in a dramatic boot speedup (it now takes about 30 seconds), so if you are experiencing similar problems, you might want to check it out.

17 December 2009

Vincent Sanders: What is it with embedded computing?

Last week I was given a development board for a system on chip (SoC) we were looking at using. All i had to do was ensure I could get a reasonable, supportable, bootloader and Linux kernel running on it.

Now I am a fairly competent engineer, I have worked with the ARM processors and SoC for many years. It should not therefor take me four days of almost continuous frustration to get a bootloader and kernel onto a system, but it has.

Firstly the system arrives and we look it over, the hardware design is not exactly elegant but seems like it will get the job done. That is at least a step up from several previous offerings which have been just plain broken. So I power it up and it boots into Fedora core 8, possibly not my first OS choice for a diskless system, but it does work.

Perhaps its worth mentioning that my experience with doing any Open Source development has caused me to have a severe aversion to software which is not upstream, especially kernels. The effort required to run your own patch series and keep it up to date is nothing short of Herculean.

Let us be honest, how many developers do you know who are talented enough to generate good code and patient enough to separately maintain it over a long period of time? I shall not belabor this point as others have repeatedly made it for me in much more eloquent ways. Lets us just say from my point of view. Upstream - good. Long term forks - bad.

With this in mind, the solutions Simtec develop are always with a view to our customers being able to simply download and use the standard upstream software wherever possible. Here is where the story stops being smooth and becomes a rant.

The system was running 2.6.22, one that was patched all which ways. "maybe they just shipped an old kernel" I thought. I opened the CD they provided and...oh, OK its just zips and tarballs of the binaries and source they used (actually that in itself is a minor miracle with a lot of vendors right now - see the almost continuous stream of copyright infringement litigation from the FSF) but sill stuck back in July 2007.

So i did teh normal thing, entered the URL printed on the outside of the box and went looking for the updates. No dice, just the same old stuff. So I contacted the company and recived the news that no there were no updates, that was what was published for the device, thats what I could have.

My initial reaction was of tired resignation...this would not be my first SoC port (probably around the fifth or sixth!) So I responded with cd linux-2.6;git checkout master;git pull and went to see what I could see. and lo and behold, there in the kernel, was full support for not only the SoC but for the other version of the very board I had.

Fortunately after mentioning this on IRC my giraffe loving friend. pointed me at the correct community git tree to find the tiny patch required to support my board.

So the vendor was not only shipping old code but actively dissuading customers from even looking for new stuff, which it turns out was mainline anyway! To add to this somewhere along the lines the company name on the box and contact information has changed and...well confusion reigned.

Right Plain sailing from here on? not quite...that would have been too easy. So you can build a mainline kernel...but you cannot boot it. you guessed it the bootloader they ship on the board is ancient, specifically a u-boot fork from December 2005. Having finally figured out how this was going to go, I went to the uboot repo, pulled the latest and built it for my target. And then I needed to upgrade the u-boot and joy of joys bricked the system.

Fortunately the system has a integrated FDTI serial/JTAG port so re-writing the flash should be as simple as running openOCD configured correctly. Of course this did not work...in the end I resorted to running their binary built versions from the CD (tarfile inside a zip inside a tarfile...whatever) and wrote their original image back to the board...which worked.

Long story short, next day I discover that i need a magic different uboot target (make u-boot.kwb for those that care) write this with the binary openocd build and...success
From then on its all been a bit anticlimactic and straightforward and I now have a system capable of starting a kernel over the network and a root fs on network, disc or flash.

My main complaint from all of this is what the hell are vendors doing shipping crap like this? Why do they all insist on taking a dreadful code drop from the SoC manufacturer when they first brought the part up and shipping that prototype junk until the part reaches the end of its life?!?

All the usual excuses about support etc. will come out, but seriously, you recive *no* support from vendors unless you pay for it and even then they will simply tell you whats on the CD is "it". Btw the diff for u-boot to support this board is larger than the original sources and the 2.6.22 is attacked with a 65MB diff which cannot be good.

The Open Source option is continuously innovating and improving, producing ever better software...but the embedded world seems to be forever stuck shipping crap. Is this just me? Does anyone have any solutions? Or am I stuck with this until I change profession?

16 December 2009

Gintautas Miliauskas: Android apps

I've recently gotten myself an Android phone. The Android app market is not as mature as iPhone's App Store, but there is a lot of applications, and plenty of chaff to separate the wheat from. I found most of the apps I use by browsing "top Android apps" lists on the web, so I am putting up a similar list here in case someone finds it useful.

Raphael Geissert: what could you tell about spammers after receiving one billion messages?

This is a question that Project Honey Pot's report answers.

They have released a report of the analysis of not only spam messages, but also of the harvesters behind spam.
Available with a simple overview the complete report provides some rather interesting details.

Do spammers take holidays?
The volume of spam drops approximately 21% on Christmas Day and 32% on New Year's Day. Saturday is the quietest day of the week.


And for those sending mails to public mailing lists, the BTS, etc:
How much harm does a harvester visiting my site do?
You can expect to receive 869 spam email messages every time a harvester picks up your email address from a website online.


The interesting bits are really worth the five-ten minutes it takes to read the complete report, enjoy it (and yay, it is CC-BY).

14 December 2009

Russell Coker: Nagios and SSL in Debian

I was doing some work on NRPE (the Nagios Remote Plugin Executor) and I noticed bug report #547092 [1] which concerns the fact that the default configuration uses the same SSL certificate for all Debian servers and provides a patch to fix the problem. After building the patched package I followed the advice of the DebianAdministration.org article on creating self-signed SSL certificates [2]. cert_file=/etc/ssl/certs/FOO-cert.pem
privatekey_file=/etc/ssl/private/FOO-key.pem
cacert_file=/etc/ssl/certs/cacert.pem Then I added the above lines to /etc/nagios/nrpe.cfg to instruct the nrpe to use the certificates. For the Nagios server I had the problem that most of the systems I monitor run old versions of NRPE while only a few are recent Debian systems that allow me to easily install a new SSL checking nrpe. So I installed the following script as /usr/lib/nagios/plugins/check_nrpe to run either the old or the new check_nrpe: #!/bin/sh -e
if echo $2 egrep -q server0\ server2\ mail ; then
/usr/local/sbin/check_nrpe -C /etc/cert/cert.pem -k /etc/cert/key.pem -r /etc/cert/cacert.pem $*
else
/usr/lib/nagios/plugins/check_nrpe.orig $*
fi The reason I started working on Nagios was to try and solve bug #560002 [3] which I filed. The bug concerns the fact that applications such as mailq which are run as part of Nagios checks were inheriting a TCP socket file handle from the nrpe. SE Linux prevents such file handles from being inherited, but it does mean that I get audit messages (and this is not a good case for a dontaudit rule). Update:
One thing I forgot to mention is that the SSL key checking requires that the server common name used in the SSL certificate of the nrpe system matches the name that is used by the check_nrpe program. So if you check by IP address then you need to use the IP address in the certificate name which is rather ugly. So I have moved to putting the hostname of each server in /etc/hosts on the NAGIOS system and using the hostname in the SSL certificate. This required using $HOSTNAME$ instead of $HOSTADDRESS$ in the NAGIOS configuration (thanks to John Slee for a tip in that regard). Update2:
I removed some printf debugging from the script. It seems that I included a pre-production version of the script in the first version of this blog post.

Joey Hess: two films

I've been watching some good films recently. Two I especially liked. baby camel in sidecar man in taxi Goodbye Solo starts in Winston-Salem and ends up at Blowing Rock. Both settings, as well as the taxi in between, are very familiar. The characters could easily be people I've met. Tulpan is set in a steppe in Kazakhstan. The desert and dust devils mostly reminded me of Mars. Particularly of this amazing shot taken by the Spirit rover. dust devils on Mars! Still, I felt so much more at home in Tulpan. Kept pausing it to check out how the family had their yurt set up and furnished, and watched their very real-seeming life unfold, right down to the lambing scene. While Solo took the very familiar, and confounded expectations and biases at every step. They're both fine films, from directors I want to see more of. I've seen Ramin Bahrani's work before Solo, and especially liked Man Push Cart. Don't know if I will ever find any of Sergei Dvortsevoy's earlier films.

13 December 2009

Russell Coker: Links December 2009

Dan Gilbert gave an insightful TED talk about our mistaken expectations of happiness [1]. Don Marti has an insightful post about net neutrality and public property [2]. When net access requires access to public property then it should be sold in a neutral manner. Rachel Pike gave an interesting TED talk about the scientific research behind a climate headline [3]. The people who claim to be skeptical of the science should watch this. Mark Peters wrote an interesting article A Happy Writer Is a Lousy Writer about the correlation between emotional state and work quality [4]. Apparently watching a film about cancer will make people more careful and focussed on details. CERIAS has an interesting short article about Firefox security as well as some philosophy on why web browser security generally sucks [5]. Cory Doctorow writes in the Guardian about Peter Mandelson s new stupidity in trying to legislate against file sharing [6]. This is going to seriously damage the economy of every country that implements it. Charles Stross has been blogging a series of non-fiction essays about space colonisation, in The Myth of the Starship he describes how most ideas of space travel are bad and how the word ship is always going to be unsuitable [7]. Brent T. White is an associate professor of law at the University of Arizona who has written an interesting paper about mortgages [8]. He says that anyone who is underwater (IE owing more than the value of their house) should walk away. The credit damage from abandoning a bad mortgage apparently isn t that bad, and there is the possibility of negotiating with the bank to reduce the value of the loan to match the value of the house. Mako is working on a project to allow prisoners to blog [9]. It s basically a snail-mail to web gateway as the prisonsers are not allowed Internet access. PracticalEthicsnews.com has an article about the special status that homeopathy is given [10]. It also notes that homeopathic medicines include arsenic and mercury. Such quackery should be outlawed, a life sentence for homeopathy would be appropriate IMHO. Cory Doctorow wrote an interesting essay about why he is not selling one of his books in audio form (he s giving it away) [11]. He concludes by noting that he wants no license agreement except don t violate copyright law . The fact that he can t get anyone to sell an audio book under such terms is a good demonstration of how broken the marketplace is. Thulasiraj Ravilla gave an inspiring TED talk about the Aravind Eye Care System a program to bring the efficiency of McDonalds to eye surgery [12]. Hopefully that program can spawn similar programs for other branches of medicine and spread to more countries. In many ways they are providing better service (both in quality and speed) than people in first-world countries who pay a lot of money can expect to receive. Scott Kim gave an interesting TED talk about his work designing puzzles [13]. He is also a big fan of social networking, unfortunately (for people like me who don t like social networking) his web site ShuffleBrain.com relies on Facebook. Gordon Brown (UK Prime Minister) gave an inspiring TED talk about global ethic vs the national interest [14], with a particular focus on the global effort required to tackle the climate change problem. Now if only we could get Kevin Rudd to listen to that. Brough has written an interesting analysis of the AT&T network problems that are blamed on the iPhone [15]. His essential claim is that the problem is due to overly large buffers which don t cause TCP implementations to throttle the throughput. This seems similar to my observations of the Three network in Australia where ping times of 8 seconds or more will periodically occur. One particularly nasty corner case with this is when using a local DNS server I can have a DNS packet storm where basic requests time out while BIND uses a significant portion of available bandwidth (including ICMP messages from receiving ports that BIND has closed). To alleviate this I am now using the Google public DNS service [16] (the Three DNS servers never worked properly).

Debian News: Debian is moving sieve to its IANA allocated port

The IANA port allocated for ManageSieve is 4190/tcp, and the old port used by timsieved and other managesieve software in many distros (2000/tcp) is allocated for Cisco SCCP usage, according to the IANA registry.

Starting with the version 4.38 of the Debian netbase package, the sieve service will be moved from port 2000 to port 4190 in the /etc/services file. This affects to those using Debian Squeeze, and Debian Sid. Debian Etch and Debian Lenny users are NOT affected.

Any installs which used the sieve service name instead of a numeric port number will switch to the new port number as soon as the services are restarted/reloaded, and in some cases, immediately after /etc/services is updated.

This will affect Cyrus IMAP. This may also affect other sieve-enabled software such as DoveCot.

In order to avoid downtime problems, mail cluster administrators using Debian are urged to verify their Cyrus (and probably also DoveCot) installs, and take measures to avoid services moving from port 2000/tcp to port 4190/tcp by surprise in either servers or clients.

It is worth noting that:

  1. /etc/services will only be automatically updated if you never made any modifications to it. Otherwise, you will be presented with a prompt by ucf or dpkg asking you about the changes.
  2. You can edit /etc/services and change the sieve port back to 2000 if you want (this is not recommended, though).
  3. You can edit /etc/cyrus.conf and any other relevant config files for your mail/webmail cluster (e.g. on the sieve web frontends) ahead of time to force them all to a static port number.
  4. You can configure cyrus master to listen on *both* ports (2000 and 4190) at the same time, and thus avoid the problem entirely. This also allows for a much more smooth migration from port 2000 to port 4190.

Henrique de Moraes Holschuh

Clint Adams: Sanjana Kumar goes to the movies

I have an imaginary automobile. It is the finest hybrid of German and Japanese engineering. It has a peppy little engine and a turning radius so small it would make a baby camel cry, and a six-speed manual transmission, and it is powered by the urine of vegan bears. My imaginary father asked me why I would possibly want to buy an imaginary car with a manual transmission. Obviously, whichever marketing campaign brainwashed him in the first place wasn't strong enough on the memetics, because after borrowing my imaginary car, he shut the hell up. You see, it's considerably more fun to drive than an automatic. There are other benefits as well, of course. The primary one is that the transmission does what I want. An automatic transmission is configured for maximum comfort , so it wastes a lot of power when shifting. With a manual, you can slip from gear to gear with minimal clutch time, or in some cases, if you're feeling sassy, without using the clutch at all. You can control your engine speed so that your shift will be just as smooth as an automatic without the egregiously long clutch disengagement or shift overlap. You also can see the road in front of you, so you're always going to be smarter than an automatic transmission, unless you don't understand how slopes work or what collisions with other cars mean to your well-being. Now some NRA member is going to point out that you can get a shift kit and tweak your automatic to be less awful. While spending money on this type of thing sounds like a great idea, a shift kit is not going to bring an automatic into the same league as my car. I can still start the car when the battery's dead. I can get higher fuel efficiency (vegan bears don't exactly urinate for free). I can shift into fifth gear, and coming from a full stop, drive up a hill without stalling (this is particularly useful when a squirrel has gnawed the teeth off of all your other gears). Furthermore, shift kit or not, an automatic is absolutely no fun. Someday, when transmissions have advanced into the realm of liquid hyperspace buckminsterfullerene cones that commune with the unicorns, and they'll have cameras and radar feeding the car's computer with useful information about its surroundings so it can make intelligent shift decisions faster than a human. Then I'll have to choose between saner and more fun. That era has not yet arrived. This is a lousy metaphor for debhelper.

12 December 2009

Benjamin Mako Hill: FLOSS Wins

Very often, folks want to refer to both the free and open source software communities in a way that is inclusive of and respectful of groups who identify with either term. Saying "free and open source software" is a mouthful. That said, there was no been consensus on what we should do instead. The Wikipedia article on alternative terms for free software suggests that FOSS, F/OSS, FLOSS, and "software libre" are contenders. I've heard all. Of course, the choice of 4+ competing alternative terms is probably worse than the problem we were seeking to solve. In academic circles, the big debate seems to be between FOSS and FLOSS. I was always a FOSS partisan. But I've seen increased momentum on the FLOSS side and I'm ready to declare that FLOSS has won. I know it makes you think of dental hygiene and I agree that it is unfortunate. It wasn't my first choice either. But I can see where things are going. FLOSS Manuals and the folks at the FLOSS Research Group at Syracuse who launched the FLOSSHub and the associated FLOSS Papers deserve some of the credit. If we can get over the irony of having spent so much time arguing over what was intended to be compromise terminology in the first place, lets see if we actually start talking to each other.

Next.

Previous.