Search Results: "ag"

21 December 2007

Erich Schubert: Thoughts on Google Calendar embedding

I've recently considered using a Google calendar for a project, and tried to embed it in a web site.However, there are a few issues I'm having with it: Also the "multiple calendars" feature is a bit hackish. I'd like to be able to differentiate events by flags such as "city", "outskirts", "training", "dance event", "music event". Obviously, entries might have more than one, so I'd need 6 calendars for this already. Usually, you can combine five...Guess I'd need to do this all in Ajax by myself. It would be cool if Google Calendar hat an API for embedding like it has for maps. The Calendar API I've seen so far is basically polling the raw data via JSON or XML. Which is already great, but I do like some of the calendar layouting the do, and I'd like to avoid having to replicate that myself.

Brice Goglin: Debian X.org notes - ATI 6.7.197 in unstable, AtomBIOS r500/600 support in experimental

I finally uploaded the ATI RandR 1.2 driver to unstable. This is xserver-xorg-video-ati 1:6.7.197-1. Lots of people were already using the experimental packages to get the RandR 1.2 support. And it looks pretty good, at least not worse than the obsolete 1:6.6.193-3 that has been in unstable during the last months or the prehistoric 1:6.6.3-2 still in testing. Things are (again) moving fast upstream. Right after the 6.7.197 released, they merged 2 important branches in master: Given how important all this is, I also uploaded a new snapshot of upstream master into experimental. This is xserver-xorg-video-ati 1:6.7.198~git20071221.be7f8fd3-1. Merry Christmas! Update: Looks like this is Radeon day. A new version (1.1.0) of the radeonhd driver has been released, so I just uploaded it to unstable too.
(Permanent link) 

Phil Hands: Virgin Media (ex. NTL) support Linux just fine, despite their claims not to.

Today I helped a friend sign up with Virgin Media's broadband service. He was having trouble, as their front-page at http://activation.virginmedia.com/ does a brain-damaged test, and tells you that they only support Windows and MacOS X and that if you have anything else you'll need to phone them at a pound a minute so you can talk to some idiot until you lose the will to live (or some such). After a brief rummage, I noticed that if instead of doing what they tell you to do, you go to https://autoreg.autoregister.net/ you get prompted for whether you're signing up for Broadband or Dialup, and on clicking Broadband, you're lead through the sign-up procedure with no complaints about compatibility, and within 5 minutes he was online with his GNU/Linux (Ubuntu) system and Firefox browser. This makes me wonder why Virgin Media bother putting extra effort into turning users away when in fact the sign-up procedure would work perfectly if they hadn't bothered with the stupid browser test.

Ian Wienand: Playing with the x86 PMU

A discussion about commas lead to an excuse to have a play with the IA-32 processor performance managment unit (PMU). To start, take two versions of a program to count the number of commas in a text file — one in C and one in Python. The C one runs faster on the input data set of ~60MiB of random data, but why? The CPU performance monitors give are the key to getting some idea of where the programs spend their time. I like to use perfmon2 because it's what I know, but Oprofile can do it too. All the available events for IA-32 are described in the manual; I currently of no better way of finding out about them than just reading it. On Itanium I reccommend Caliper which, for the most common situations, does most of the work for you and presents it in a nice report. Intel's Vtune also does a similar thing. The first thing to investigate is if the CPU is getting enough instructions to keep busy. The IFU_MEM_STALL metric is a good place to start as it is triggered when the instruction fetch pipeline is stalled, presumably waiting on either the ITLB or the trace buffer (Icache).
$ pfmon -e CPU_CLK_UNHALTED,IFU_MEM_STALL ./comma < ./randomcommas
340559375 CPU_CLK_UNHALTED
   192115 IFU_MEM_STALL
$ pfmon -e CPU_CLK_UNHALTED,IFU_MEM_STALL python -Sc "import sys; print sum(l.count(',') for l in sys.stdin)" < ./randomcommas
1287100
4571257047 CPU_CLK_UNHALTED
 71981750 IFU_MEM_STALL
That works out to 0.05% of total cycles for the C version and 1.5% for the Python version, neither of which sets off immediate warning bells. If it did, we could start drilling down to things like the L2_IFETCH and ITLB_MISS events, or the BR_* branch events to try and see why the CPU is having to wait to get its next instruction. Next it is useful to find the CPI (cycles per instruction). This is calculated by the ratio of retired instructions against the number of CPU cycles; since a superscalar machine can issue more than one instruction per cycle this should ideally be much greater than 1 (for example, an Itanium can execute up to 6 instructions each cycle).
$ pfmon -e INST_RETIRED,CPU_CLK_UNHALTED ./comma < ./randomcommas
542953593 INST_RETIRED
340612036 CPU_CLK_UNHALTED
$ pfmon -e INST_RETIRED,CPU_CLK_UNHALTED python -Sc "import sys; print sum(l.count(',') for l in sys.stdin)" < ./randomcommas
1194455205 INST_RETIRED
4569931735 CPU_CLK_UNHALTED
This works out at a CPI of 1.59 for the C version and 0.26 for the Python version. The Python version is clearly spending a lot of time waiting, because it isn't even able to issue one instruction every cycle. At this point it seems the CPU has enough instructions to do, but it is sitting around waiting to get through those instructions. This suggests the waiting is related to getting data from the cache. The load and store requests from the L2 cache are accounted to the L2_LD and L2_ST events respectively. These have the ability to mask out cache lines in different states of the MESI protocol, but for this we don't care so just ask pfmon to show us everything.
$ pfmon -e L2_LD:M:E:S:I,L2_ST:M:E:S:I ./comma < randomcommas 
102505 L2_LD:M:E:S:I
   167 L2_ST:M:E:S:I
$ pfmon -e L2_LD:M:E:S:I,L2_ST:M:E:S:I python -Sc "import sys; print sum(l.count(',') for l in sys.stdin)" < ./randomcommas
3278774 L2_LD:M:E:S:I
  10457 L2_ST:M:E:S:I
This shows us that the Python version does quite a few more stores than the C counterpart. Considering this program should simply be reading the input stream and counting the number of commas, we do not expect much store traffic at all. This suggests the Python version is doing some extra copying, for whatever reason (maybe some Python expert can pinpoint it?). We can drill down a bit more into the memory related latencies. The DCU_LINES_IN event gives the total number of lines allocated in the cache. Another event, DCU_MISS_OUTSTANDING, gives a weighted measure of the cycles spent waiting for a cache line to be brought in. Each cycle spent waiting is weighted by the number of outstanding cache misses (I think the Pentium M I'm using can have up to 4 cache miss requests outstanding at once) and has some caveats, but can be considered a rough estimate of the time spent waiting for a cache line to be brought in. Therefore divding DCU_MISS_OUTSTANDING by DCU_LINES_IN gives us an approximate metric of how long a cache miss takes.
$ pfmon -e DCU_MISS_OUTSTANDING,DCU_LINES_IN ./comma < randomcommas 
769736 DCU_MISS_OUTSTANDING
102387 DCU_LINES_IN
$ pfmon -e DCU_MISS_OUTSTANDING,DCU_LINES_IN python -Sc "import sys; print sum(l.count(',') for l in sys.stdin)" < ./randomcommas
99810150 DCU_MISS_OUTSTANDING
 4240179 DCU_LINES_IN
So that works out to 7.5 cycles for the C version and 23 cycles for the Python version. This seems to strongly suggest that it is memory traffic that is weighing the Python version down. That is only the initial phase; the results give a high level idea of why one program is running slower than the other. The initial analysis phase generally consists of taking the ratios of certain events to try and get some idea in your head of what the program is doing. Then comes the really hard work; drilling down to figure out how to fix it! Some useful references:

20 December 2007

Jeff Bailey: UPS

I love UPS, I really do.

Their autoattendant: "Please speak want you want, you can say ..."
me: "Agent"
AA: "I'm sorry, I didn't understand that. You can say ..."
me: "Agent"
AA: "I'm sorry, I didn't understand that. You can say ..."
me: "Do I have to say 'Fucking Agent?'"
AA: "One moment please, I will transfer you to an agent."

Brett Parker: rss2maildir.py

I've been looking for a decent way to read rss feeds for a bit, I've finally given in to the fact that there's actually nothing about that I feel comfortable with... so, what I've started is writting a simple rss to maildir convertor (I know there's toursst, but that appears to want a galleon bookmarks file and appears to use deprecated bits of python). At the moment it is very very basic, there's still a way to go before I'll be fully happy with it, but it's much nicer being able to read rss using mutt than Yet Another Random Program, I'll be hacking up a mutt config to make it look different to e-mail later, but for now as it's still in testing phases, I'm just testing with a bog standard mutt config. I've made the code available through a git repository that should (when DNS has finished updating) be available with a git clone git://git.sommitrealweird.co.uk/rss2maildir.git/ before that it should be available from git clone http://miranda.sommitrealweird.co.uk/~brettp/rss2maildir.git/ . The HTML -> Text parser still needs some work, I'm mostly aiming for it to take HTML and generate almost OK ReST which is easier to read in a text mail client. It still needs paragraph wrapping etc, but I'll work on those over time. If anyones interested, or wants to get involved, or has some nicer ideas for how to make it work, as always I can be contacted at iDunno@sommitrealweird.co.uk :)

Stefano Zacchiroli: pts face lifted

PTS face lift(ed!) Ok, by now you've probably all already noticed, but I've finally committed the changes to the PTS package pages which implement a new layout and CSS. Thanks to Dan Callahan, Enrico Tassi and madcoder for their help with the CSS and to all who gave feedback (sorry, if it wasn't possible to satisfy all look and feel requests at the same time). There are still some minor, freshly introduced, glitches to fix (see for example 457168 and 457169), but I'm kinda tired to play with CSS and I'll give up for a while. But patches are always welcome!, I'll do my best to apply them timely. While I was at it, I've also fixed some other minor nuisances, such as the svnbuildstat link which is now shown only if the package is actually indexed by svnbuildstat. Thanks to Gon ri for having made available the info. PS totally unrelated, but thanks to Lucas now the homepage and vcs stats page sport wonderful graphs!

MJ Ray: Current Issues / Severn Barrage Opposition Gathers

The Wildfowl and Wetlands Trust is hosting a meeting about wildlife concerns over the Severn barrage tonight at their place further up the river.  I've heard these concerns mentioned vaguely whenever the dam is discussed, but I don't really know what they are - does anyone?

The press release is a bit short on details and a quick look around the websites of the invited organisations didn't show anything obvious, except for Severn Barrage is not the answer, News: Tidal energy projects must not override existing environmental protection and Engineering our future nature: the dilemma of the Severn barrage on the WWF Cymru site.  Does that cover all the concerns, or are there more?

I'm still sort of surprised that the nation is seriously considering trying to put a barrier against the second-largest tidal flow in the world (Canada's Bay of Fundy seems to be the largest) and Britain's longest river, but I guess that's how our energy use is these days.

Lucas Nussbaum: Re: Is it hard for new contributors to help Debian? Can we improve things a bit?

Andreas Schuldei writes about my blog post about making it easier to contribute to Debian:
It is much more gratifying for a contributor if his effords have
immediate effects.

[…]

So what can people who want to help Debian do to achive more imminent gratification? Pick projects that let you help directly with direct svn/git/whatnot access (Security team, Debian Edu, Debian-Installer…)
While I agree that Instant Gratification is very important, I don’t think that the proposed solutions solve anything. It’s easy to have things sleep in an SVN repository instead of the BTS (lots of teams do that, sometimes for good reasons, like the lack of sponsors - hint: Games team). It doesn’t make things any better. What we need is tasks whose results will be available without too much wait in Debian unstable, for everybody. Also, I must admit that I know very little about the inner workings of the Security Team. But I have the idea that it involves quite a lot of procedures (helping with bugs tagged security probably doesn’t, but going further than that probably does). Contributing to Debian already involves A LOT of procedures. A new contributor, even very good technically, can easily get lost in all the different ways to package stuff and solve common problems. So we should identify tasks that don’t have huge requirements. Ubuntu has the concept of bite-size bugs. I started working on a page explaining the different ways someone can contribute to Debian. The goal is to provide a good entry point (not something that replaces existing documentation) and keep it at a manageable size. Its focus is mainly to replace the emails we write to people asking “I’d like to contribute, but I don’t know what I can do or where I should start!”. The page is available on wiki.d.o/HelpDebian/Start. Don’t hesitate to improve it (see also wiki.d.o/HelpDebian for a TODO list). When it will be good enough, I plan to move it out of the wiki (the content shouldn’t change much anyway) to eg http://help.debian.net/, so we can improve the design (I really like the idea of adding pictures of past Debconfs like Christian Perrier did in his FOSS.in talk).

Russell Coker: Apprentice Computer Journalist Wanted

In a comment on my post about apprentices [1] Don Marti pointed me to a blog post of his from 6 months ago where he mentioned a need for an apprentice [2]. I had read the post in question before but didn’t think about it when writing my previous post. When I was about 17 I had my first article published in a computer magazine, it was a small magazine for a computer club and the article was merely a long email on the club BBS which someone decided to publish. The option of trying to make a career out of journalism occurred to me then as I had been published once without trying, and the quality of the magazines at the time indicated that anyone who wanted to write technical articles would face little competition. :-# If someone as skillful as Don had been offering apprenticeships during school holidays when I was at that age I would have been very interested, but at the time there was too much cultural pressure (and too much pain invested) to make quitting school entirely an option. I encourage other people who have the ability to take on an apprentice to blog about it. Please make sure to mention your approximate location and whether school-holidays work is OK.

Simon Richter: The Debian package namespace

A Debian package is uniquely identified by its name and version; there is no further information taken into account by dpkg. This keeps the packaging system rather simple, but has the disadvantage that all information that would otherwise be placed in extra data fields has to be encoded into these two fields. As the names of all binary packages generated from a single source package need to be known when building the .dsc file, no substitutions are allowed in the package name. Back in the Good Old Days, the only extra bit of information in the name field used to be the SONAME from the library in the package (thus implicitly saying that each library should have its own package), which meant that the maintainer had to adapt the debian/control file every time a library's ABI changed. This being the Good Old Days, people writing libraries actually thought about things, so ABI changes were few and far between. These days, libraries are stacked on top of each other, re-exporting the underlying libraries' ABI. The SONAME system could in theory handle these situations if the SONAME of every library whose ABI is re-exported is encoded into the SONAME of the newly built library. So if my own ABI version is "2", and some of the functions I export use types defined in glib, then after building against glib1.2, my SONAME would be somewhat similar to "2+glib1.2". For languages that don't have a well-defined ABI (i.e. anything but C), the version of the language ABI(s) used will also have to be added. Obviously, this would also have to be encoded into the package name, which is where trouble starts during transitions. The last C++ ABI transition had all C++ library maintainers add a suffix to the package name; dependent ABIs are not handled at all in Debian right now (which led to the libpng mess). The existing encoding rules are not entirely formalized, and it is not guaranteed that they are bijective. For example, packages used together with cross compilers have been following the convention of using the name that the native package would use, followed by a dash, the target architecture name, another dash and the word "cross". There is no guarantee that the architecture name does not contain a dash, however, so we cannot simply take the second to last word as the architecture name if the last word is "cross" (think "libgcc1-kfreebsd-i386-cross"), which makes it difficult to write a plug-in to apt that would automatically download the binary for the target architecture and convert it using dpkg-cross. I've decided to nonetheless go ahead with my request that the "-*-cross" part of the package name space be reserved for packages related to cross compilation; it will work for the time being.

Michal &#268;iha&#345;: Gammu stable version 1.17.0

I wanted to make 1.16.0 last release this year, but unfortunately one bug preventing reading of all SMS messages from some AT phones sneaked in, so I decided to make another quick release. Changelog is really short: Other change is that I started to produce additional binary build for Windows - it does not use any external libraries (so no MySQL, PostgreSQL or Gettext support) and links statically. So you can just copy single binary and it should work anywhere. You can download from usual place: http://cihar.com/gammu/

Riccardo Setti: new metacity package

hey guys,  here in italy are 3.02 a.m and i'm still drunk (as my friend XX said:"i have a problem with alcol.. i like it") , but i made a good thing for all the crazy guys who run experimental, i've build latest metacity (the GNOME window manager). The cool thing of version 2.21.5 (experimetal) is that it is build with the composition manager enabled by default. all my changes are in the debian pkg-gnome svn, but i'm not sure if i should upload it right now, so feel free to take it on my home on people . 

19 December 2007

Neil Williams: First working version of em_installtdeb in Emdebian SVN

Emdebian SVN r2618 includes my first working implementation of em_installtdeb. Barring a single annoyance message from dpkg, the script works. :-)
The warning arises from the need for a modified "source" tarball that can be uploaded alongside the .deb, containing only the sources needed for this specific translation: debian/control, debian/changelog, po/$lang.po /* if any */ and po/$GETTEXT_PACKAGE.pot. Where a package contains more than one po directory, all POT files are included and all po files for the requested translation. This allows translations to be updated independently of the upstream package.
I would advise using a test directory if you do want to try it, not your main package building directory. ;-)
Output (using gpe-expenses as the demo package) in translator mode:

$ em_installtdeb sv
dpkg-deb: building package gpe-expenses-locale-sv' in ../gpe-expenses-locale-sv_0.1.4-1_all.deb'.
dpkg-source: warning: .orig.tar name gpe-expenses_0.1.4.tdeb.tar.gz is not _.orig.tar (wanted gpe-expenses_0.1.4.orig.tar.(?:gz bz2 lzma))
dpkg-source: building gpe-expenses using existing gpe-expenses_0.1.4.tdeb.tar.gz
dpkg-source: building gpe-expenses in gpe-expenses_0.1.4-1.diff.gz
dpkg-source: building gpe-expenses in gpe-expenses_0.1.4-1.dsc
dpkg-genchanges: including full source code in upload
Format: 1.7
Date: Fri, 07 Dec 2007 00:08:39 +0000
Source: gpe-expenses
Binary: gpe-expenses-locale-sv
Architecture: source all
Version: 0.1.4-1
Distribution: unstable
Urgency: low
Maintainer: Neil Williams
Changed-By: Neil Williams
Description:
gpe-expenses-locale-sv - sv translation for gpe-expenses
Changes:
gpe-expenses (0.1.4-1) unstable; urgency=low
.
* New upstream release
* use machine operable debian/copyright, migrate to standards 3.7.3
Files:
781a60eccc4396ee3f0f86a01150e755 316 misc extra gpe-expenses_0.1.4-1.dsc
f9dd946db27ff6077f8e85f7100d3ea4 5863 misc extra gpe-expenses_0.1.4.tdeb.tar.gz
894f5789ee0348abfa61a5e7342ed6a7 20 misc extra gpe-expenses_0.1.4-1.diff.gz
56f8d22c30cb5502633ff4d26862db78 4532 misc extra gpe-expenses-locale-sv_0.1.4-1_all.deb
Package-Type: tdeb
Changes file: ../gpe-expenses_0.1.4-1_tdeb.changes
signfile ../gpe-expenses_0.1.4-1.dsc 0x28BCB3E3

You need a passphrase to unlock the secret key for
user: "Neil Williams (Debian) "
1024-bit DSA key, ID 28BCB3E3, created 2002-01-27


signfile ../gpe-expenses_0.1.4-1_tdeb.changes 0x28BCB3E3

You need a passphrase to unlock the secret key for
user: "Neil Williams (Debian) "
1024-bit DSA key, ID 28BCB3E3, created 2002-01-27


Successfully signed dsc and changes files
neil@holly:gpe-expenses$ dpkg -c ../gpe-expenses-locale-sv_0.1.4-1_all.deb
drwxr-xr-x neil/neil 0 2007-12-19 14:37 ./
drwxr-xr-x neil/neil 0 2007-12-19 14:37 ./usr/
drwxr-xr-x neil/neil 0 2007-12-19 14:37 ./usr/share/
drwxr-xr-x neil/neil 0 2007-12-19 14:37 ./usr/share/locale/
drwxr-xr-x neil/neil 0 2007-12-19 14:37 ./usr/share/locale/sv/
drwxr-xr-x neil/neil 0 2007-12-19 16:09 ./usr/share/locale/sv/LC_MESSAGES/
-rw-r--r-- neil/neil 6419 2007-12-19 16:09 ./usr/share/locale/sv/LC_MESSAGES/gpe-expenses.mo
-rw-r--r-- neil/neil 1796 2007-12-19 16:09 ./usr/share/locale/sv/LC_MESSAGES/qofexpensesobjects.mo

Hmm, that isn't right - need to check on the ownership of those files - might be enough to use fakeroot.

$ dpkg -I ../gpe-expenses-locale-sv_0.1.4-1_all.deb
new debian package, version 2.0.
size 4532 bytes: control archive= 302 bytes.
230 bytes, 9 lines control
Package: gpe-expenses-locale-sv
Source: gpe-expenses
Version: 0.1.4-1
Architecture: all
Maintainer: Neil Williams
Installed-Size: 44
Section: misc
Priority: extra
Description: sv translation for gpe-expenses


em_installtdeb is a debhelper add-on created by Emdebian to create translation packages (tdebs). em_installtdeb is intended to separate out the individual translation files from the current Debian packages into packages without any translation files and a series of tdeb locale packages, one per translation. Generated packages use the syntax: $package-locale-$language_code_$version_all.deb
Certain language codes need to be modified to make acceptable components of a debian package name. Underscores are converted to hyphens, '\@' is converted to '+' and all codes are made lowercase. These changes only apply to the package name, the installation location is unchanged.
The locale package must use GETTEXT_PACKAGE for the eventual filename of the binary translation file - not the $dh MAINPACKAGE . GETTEXT_PACKAGE is determined by upstream, not Debian. When building the whole package, the binary translation file may be in debian/tmp/usr/share/locale/$lang/LC_MESSAGES but when in translator mode, this location is not available. Instead retrieve GETTEXT_PACKAGE from the POT filename. debian/control is needed for various dpkg and debhelper calls only.

The "source" for translators therefore includes:
debian/control
debian/changelog
po/$lang.po /* if any */
po/$GETTEXT_PACKAGE.pot

Note that no rules file exists. (i.e. dpkg-source -x will complain when trying to extract it later so a new wrapper is needed (not difficult).)
em_installtdeb will try to generate the necessary POT file(s) and then create a $package_$version_tdeb.tar.gz containing the source files.
The default action is to process all available po files.
The above example uses translator mode where LANG_CODE is passed to only handle that language. In translator mode, em_installtdeb builds a tdeb package, a source tarball, a .dsc and a .changes file (and signs the .dsc and .changes). In default mode, em_installtdeb builds the package(s) but leaves the source, .dsc and .changes to the rest of the build process.
LANG_CODE can either be the name of the locale as specified in the PO filename (fr, en_GB, sr@Ltn etc.) or the name of the locale as specified in the eventual package name (fr, en-gb, sr+ltn etc.).
If a po file already exists for this locale, em_installtdeb processes just that one translation and builds a single tdeb package.
If the po file does not exist, em_installtdeb exits with an error. (Copy the POT file from the original source to create a new po file for the new locale and edit it to include the new translation, then run em_installtdeb again.)

Output in default mode:
$ em_installtdeb
dpkg-deb: building package gpe-expenses-locale-cs' in ../gpe-expenses-locale-cs_0.1.4-1_all.deb'.
dpkg-deb: building package gpe-expenses-locale-de' in ../gpe-expenses-locale-de_0.1.4-1_all.deb'.
dpkg-deb: building package gpe-expenses-locale-pt' in ../gpe-expenses-locale-pt_0.1.4-1_all.deb'.
dpkg-deb: building package gpe-expenses-locale-sv' in ../gpe-expenses-locale-sv_0.1.4-1_all.deb'.
dpkg-deb: building package gpe-expenses-locale-vi' in ../gpe-expenses-locale-vi_0.1.4-1_all.deb'.

Next stage is to fold this into emdebian-tools so that the emdebian patch files can be cleaned up and various complex patches to debian/rules can also be cleaned up.
Finally, note that this only supports gettext and only supports program translations. This is an Emdebian script written with an eye on usage in Debian but primarily intended to meet Emdebian requirements. As outlined before, Emdebian does not want language packs or translation grouping. One locale, one package.
See also langupdate in Emdebian SVN.

Ian Haken: Trials and Tribulations of ARM Programming

I spent a rather long time today trying to figure out a rather obscure bug. I few months ago I bought a gp2x, an open portable gaming system (and media player, etc.). I've really enjoyed it so far. The open platform for development has lent itself it many fun apps, console emulators the most conspicuous I think, but it's also fun to experiment and develop with it. I recently ported an asteroids clone I wrote and the whole process was actually fairly simple. As such, I decided to take on a larger task of porting something like FreeCNC, which as far as I know, no one has done yet.

After some simple trouble with library version conflicts, I got the program compiling correctly. However, if wasn't reading files quite right. I spent a little while investigating and discovered the problem was in a function which essentially hashes filenames. In the end, it was this snippet of code which was giving the trouble:

char buffer[13];
...
calc = ROL(calc)+*(long *)(buffer+i)


In particular, this is trying to read 4 bytes from this character array as an integer. At first I though the trouble might be endian-ness, but both the gp2x (an ARM processor) and x86 use big-endian*. I finally resorted to running gdb and sifting through assembly level instructions.

$r2 = 0xbffffd7c
$r3 = 0x41434f4c
0x0000a310 _ZN8MIXFiles7calc_idEPc+52>: str r3, [r2]


Here, the register $r2 points to buffer and $r3 is the value we're trying to save into the buffer. After this we would expect
*0xbffffd7c = 0x41434f4c
but instead see
*0xbffffd7c = 0x08db9841.
If I purposely set *0xbffffd7c = 0xffffffff and repeat this instruction we see
*0xbffffd7c = 0xffffff41.

Clearly the problem lies in this single str instruction, and was resident to the ARM architecture. So of course it was time to go to the processor documentation. Reading the fine print finally reveals what I was looking for.
A word store (STR) should generate a word aligned address. The word presented to the data bus is not affected if the address is not word aligned. That is, bit 31 of the register being stored always appears on data bus output 31.
Yep, since the buffer was an odd-sized array and stored on the stack, it started on an odd number in memory. Hence its address was not word aligned, and so as per the documentation, the store did not have the expected result. Here's a short program which shows what I'm talking about.

x86ARM
buf=BFBF3471, buf%4=1
Num: 4C4F4341
Buf+0: 4C4F4341
Buf+1: 4C4F4341
Buf+2: 4C4F4341
Buf+3: 4C4F4341
buf=BFFFFD75, buf%4=1
Num: 4C4F4341
Buf+0: 4F434100
Buf+1: 43410000
Buf+2: 41000000
Buf+3: 4C4F4341

Perhaps the lesson to learn here is that when casting pointers, it's buyer-beware, and there are safer bitwise manipulations to use, though certainly less efficient.


*Edit: Both x86 and the ARM processor in question are little-endian, not big-endian. Typo on my part, but notwithstanding the point is that they both had the same endian-ness, so it wasn't where the issue lay.

David Welton: Android Task List / Selector

Credit for finding out how to do this goes to dims, who has been doing lots of cool Android hacking lately: http://davanum.wordpress.com/2007/12/18/android-task-manager-primitive-prototype/ If you're interested in Android and what it can do, you should be following his online journal! As a way to keep pushing and testing Hecl, I decided to see if I could copy what he'd done in Hecl. It has indeed produced a few improvements to the Java code. That done, making it work was short and sweet:
set layoutparams [linearlayoutparams -new  FILL_PARENT WRAP_CONTENT 
set layout [linearlayout -new $context]
$layout setorientation VERTICAL
# Create ourselves some commands to access Android internals.
java android.app.ActivityManagerNative activitymanagernative
java android.app.IActivityManager iactivitymanager
java  android.app.IActivityManager$TaskInfo  taskinfo
java android.content.ComponentName componentname
java java.util.List javalist
set am [activitymanagernative getdefault]
set tasks [$am gettasks 10 0 [null]]
set len [$tasks size]
set tasklist [list]
for  set i 0   < $i $len   incr $i   
    set baseactivity [[$tasks get $i] -field baseactivity]
    lappend $tasklist "Task: [$baseactivity getpackagename]"
 
$layout addview [textview -new $context -layoutparams $layoutparams \
         -text "Currently running tasks"]
set lview [basiclist $context $tasklist -layoutparams $layoutparams]
$layout addview $lview
$lview requestfocus
set callback [callback -new [list [list SelectTask]]]
$lview setonitemclicklistener $callback
heclcmd setcontentview $layout
proc SelectTask  parent view position id   
    [activitymanagernative getdefault] movetasktofront $position
 
It could have been squeezed further by avoiding some of the temporary variables, and perhaps also by implementing a mapping between java.util.List and Hecl lists. I think one of the next things I want to do is work on some infrastructure so that people will be able to actually work on scripts on their phone. That's sort of a departure from the past (most phone keyboards would be sheer torture for inputting long scripts), but it's a cool demo in any case.

Russell Coker: Bad Project Management

I have just read a rant by Sean Middleditch about bad project management [1]. He describes his post as “personal, rather angsty, and especially whiny” but I think it’s useful and informative. He makes some interesting technical points about PHP programming (I wasn’t aware that there were so many ways of easily getting things wrong and having difficulty to get them right). But of course this isn’t all limited to PHP, the web site WorseThanFailure.com has anecdotes about mistakes of similar calibre being implemented in every language imaginable. Sean is apparently considering leaving the computer industry after having numerous bad experiences of having highly paid people mess up projects while he gets paid a lot less to try and fix the worst of the bugs and get the systems working in production. I understand what it’s like, I have occasionally idly contemplated leaving the industry after bad projects. However the fun of working on free software combined with the amounts of money that I can earn in the computer industry made me quickly abandon such ideas. His stories in some ways resemble my experiences in working as a contractor, most of my contracts have been profoundly weird for various reasons (I’ll use the WTF [2] category of this blog to document some of them). I had two theories as to why I ended up in so many strange contracts, one was that I was in some sort of Twilight Zone and the other was that taking contracts based on the amount of money offered puts you at high risk of being employed by people who have no financial pressure to do things in a sensible manner. My advice to anyone in such a situation is to try and find a contract position paying an unreasonable amount of money. Getting more than $80 an hour (the rate Sean cites as being paid to the idiots who cause problems) is going to be difficult, but getting $50 or $60 an hour is much easier to achieve and should be enough to alleviate the pain of working on doomed projects.

Lucas Nussbaum: Is it hard for new contributors to help Debian? Can we improve things a bit?

Debian could use more manpower, but is it actually easy for new contributors who would like to help to do useful things, and get the impression that they actually improved Debian? There are several problems here: I think that other projects are better than us at providing directions to newcomers. Gnome has the whole Gnome Love thing, for example. I’m thinking about building lists of “interesting” bugs. That would include RC bugs without patches, of course, but could also include a list of bugs based on an usertag that would mean “This bug is suitable for new contributors (it’s not too hard, and you could learn things fixing it). If you want to work on this bug, I’ll provide help and feedback and integrate your patch ASAP.” (This is very different from the “help” tag, which usually means “I couldn’t fix this, it’s too hard, I need help.“. It’s a bad idea to direct newcomers to bugs that are too hard for the package’s maintainer!) Additionally, a nice feature would be to be able to filter the list of bugs with the packages that the user has installed. Comments are open, please us them ;)

Michal &#268;iha&#345;: Looking for Windows testers of Wammu

After half day of playing with Wine, MinGW and friends, I managed to be able to automatically build Windows versions of both python-gammu and Wammu. However I do not have environment to actually test these versions (except for Wine), so I would really welcome if some Windows users could do some testing. I expect some strange crashes at beginning, but hopefully we will be able fix it to produce working Wammu version for Windows in future. You can get the installer here: http://tmp.cihar.com/wammu-0.26-setup.exe It's NOT final or stable version, but as the only changes which happened since last release were either minor or related to cross compilation, it should be equal to combination of Gammu 1.16.0, python-gammu 0.24 and Wammu 0.25, so don't be confused by versions. PS: Okay, when application works in Wine, it does not mean it will work in Windows, see bug report for details.

18 December 2007

Martin F. Krafft: bugscripts and rng

Yes, Bastian, there is a way to do what you want. I believe I told you from the very start: refactor reportbug and implement reportbug-ng on top of the existing code. But no, you chose to reimplement everything (a typical thing to do for GUI programmers) and now you are asking us maintainers to change the way we've been doing things for years? You also make presumptuous statements about how we deal with bug reports. If you personally hate how certain packages prepare bug reports, then maybe you should personally suck it up and use different software? Seems like a more logical and less intrusive way forward, then expecting others to "send a friendly mail" to ask for more information on a bug report. Maybe you have all the time in the world, but I don't. I can work on e.g. mdadm (which uses an interactive bug script) only on rare occasions, and then I'd rather fix and close bugs than to indulge in another iteration of bug submitter ping-pong. You are also personally annoyed by long pre-subject texts, which you doubt anyone reads? I don't even bother to challenge that, but let me say that pre-subject texts are there to prevent stupid bug reports, or bug reports filed in the wrong place, and to take load off the maintainer. So you chose to ignore them? Given the projected user-base of your tool, I'd say you should rather force people to read them by asking them to transcribe the text. If you don't want to pop up a terminal, well, why the heck didn't you integrate debconf properly, while reusing reportbug, and taking a step forward to provide a proper interface for bug scripts? Why do you even bother to whine about interface issues at this point? Feel free to take over my packages, which make use of interactive bug scripts. I see zero reason to change them, and I will not. NP: The Flower Kings: Paradox Hotel

Next.

Previous.