With this release come two long-awaited improvements:

• database speedups;
• fix for iTunes timing out after 30 minutes.

Release 0.18 is a small update to forked-daapd, adding a config knob for the ALSA mixer channel to use for volume setting; this is required in some setups. Starting with this release, the forked-daapd logfile needs to be owned by the user forked-daapd runs as. If you use logrotate, you want to add the create option to the config stanza for forked-daapd. The GCD codebase received a build fix for FreeBSD and I m told it works nicely there too. Tarballs available at http://alioth.debian.org/~jblache/forked-daapd/ or in the forked-daapd project on Alioth; GPG signatures made with my (new) Debian key FA1E5292. I m trying to transition the git repository to the forked-daapd project too, but I can t get it to show up under the SCM tab in the web interface nor does it show up in gitweb. Still waiting for a resolution on that.

forked-daapd v0.17 is out; the main purpose of this release is to add support for libav 0.7. There are also a few bugfixes that made it into this release, but nothing earth-shattering. One More Thing You ll notice an extra tarball and tag in this release, both marked 0.17gcd. With this release, the GCD (Grand Central Dispatch / libdispatch) codebase is making its debut. libdispatch replaces libevent and allows for greater concurrency inside forked-daapd. This makes forked-daapd snappier, which shows when using Remote or a SoundBridge. The filescanner also benefits from this; it now scans several directories concurrently, reducing the time needed for a rescan by making better use of the machine resources. Thanks to Mark Heily s efforts on libdispatch/libkqueue/libpthread_workqueue over the past year, we now have a Linux version of libdispatch that works well enough to support forked-daapd. This codebase will become the primary codebase for forked-daapd once the database performance issue will be fixed. After this point, the current libevent 1.4 codebase will be legacy and won t see any further development (we ll discuss bugfixes in due time). This new codebase comes with new requirements:

• the Clang compiler from the LLVM project is required, due to the use of the C Blocks extension that isn t supported by GCC;
• libdispatch, of course, is required;
• libtre is a new dependency for concurrent regexps;
• glibc 2.13+ is required, or any glibc version containing a fix for glibc BZ#10563.

libdispatch and its dependencies are available in Debian unstable; it s best and easier to use whatever is in unstable, instead of mixing and matching the libdispatch/libkqueue/libpthread_workqueue versions until they all work together. Due to platform support in Clang, the GCD codebase can only be built for and will only run on i386 and amd64, as far as Linux is concerned. Other architectures will have to wait until support for them is added to Clang. FreeBSD is untested and will probably require some modifications to work; recent enough versions of FreeBSD have a native pthread_workqueue implementation and do not need libpthread_workqueue. Packages are available in experimental for Debian users; they ll end up in unstable in a few weeks, before the libevent2 migration happens there. Tarballs available at http://alioth.debian.org/~jblache/forked-daapd/ ; GPG signatures made with my Debian key F5D65169.

forked-daapd v0.16 is a bugfix release, mainly addressing a bad regression introduced in v0.15. Fixes:

• DAAP songlist off-by-one with sort tags
• handling of non-UTF-8 tags under some circumstances
• file size in HTTP streaming

The DAAP bug is what causes an incorrect display and sorting in iTunes, the tags issue could cause a segfault in some cases, and finally fixing the file size in the HTTP streaming code may or may not fix some issues but was a bad bug anyway. Tarballs available at http://alioth.debian.org/~jblache/forked-daapd/ ; GPG signatures made with my Debian key F5D65169.

forked-daapd v0.15 is here! The database speedups did not make it into this release as they still need some more work. Now let s talk about what did make it into this release: AppleTV metadata support. When streaming to an AppleTV, you ll now get the cover art, artist, album, title and playback position displayed on your TV. This was made possible by a forked-daapd user who offered me one of the new AppleTVs, so thanks much to him! Although we haven t got the database speedups this time around, we ve got some other speedups:

• the code generating the song list was slightly reworked;
• hash tables of static data were replaced with static hash tables generated by gperf;
• the artwork code can now send out both PNG and JPEG to clients.

The first two items bring in a new build-dependency: gperf. I promise it s not some Java-based monstrosity ;) It s small and C++, so no issue getting that running on whatever your platform is. Pre-generated files are not provided for this. The last item means if your cover art is mostly JPEG, forked-daapd won t be spending time turning it into PNG before sending it out to the client. That s some CPU time that can be put to better use. Next release: when it s ready! For Squeeze users, I am providing backports on backports.debian.org; due to the update policy, they are lagging behind the releases. Tarballs available at http://alioth.debian.org/~jblache/forked-daapd/ ; GPG signatures made with my Debian key F5D65169.

forked-daapd v0.14 is available. This release includes a number of improvements to the sort headers/sort fields handling by Kai that got entangled into a larger database revamp at the end of last year. That will be the subject of the next release. Also in this release:

• better handling of tags for TV shows
• better handling of DRM-afflicted files
• configurable IPv6 support
• fixes to the URL file scanner
• misc fixes

This release performs a heavy database upgrade process that dumps and reloads the files table; this can take some time on big libraries, so please be patient while this process takes place. You should backup your database file before upgrading, just in case. See you in about two weeks for 0.15 with some database speedups that will finally see the light :-) Tarballs available at http://alioth.debian.org/~jblache/forked-daapd/ ; GPG signatures made with my Debian key F5D65169.

forked-daapd v0.13 is available! This release has been in the making for a long time, got delayed for a lot of reasons and had its content changed a few times along the way. Nonetheless, it s here now and if all goes to plan at least a couple releases should appear in the coming months, say before the summer. In this release:

• support for Remote 2.0
• support for per-speaker volume control
• support for RAOP retransmission
• improved compatibility with the 802.11g AirPort Express
• support for sort field metadata
• support for JPEG artwork and more alternative filenames
• fix for Murmur hash on 32bit platforms
• improved mDNS address resolution, making IPv4 usable again
• support for FFmpeg 0.6 (moving away from deprecated APIs)
• DAAP session expiration has been disabled
• misc fixes

Thanks to all the contributors that sent bugs and patches for this release! I know development doesn t go quite as fast as some would like, but keep in mind that I m doing this in whatever free time I have and that the public git tree doesn t tell the whole story. There are several parallel feature branches you don t get to see, for instance. Hopefully 0.14 will appear in a few weeks. Tarballs available at http://alioth.debian.org/~jblache/forked-daapd/ ; GPG signatures made with my Debian key F5D65169.

pommed v1.36 is out and brings support for the MacBook7,1, along with a few fixes and updates. A typo in the pmac keyboard backlight has been fixed, so fading now works there too. Product IDs for the ANSI and JIS keyboard variants for pmac machines were also added. It s been like forever since I last touched that code; not a lot of people still use those (great) machines. This release also contains a small update that will make it compatible with future kernel releases as applesmc will no longer have a fixed a name and location in sysfs. Update: I just noticed a bug in 1.36, so 1.37 is out now. It also contains two patches that were submitted through Alioth.

As part of my work for EDF, I ve had to package and integrate a set of tools for performance measurement and profiling of HPC code. This toolbox comprises tools for analysis of both sequential and parallel codes, MPI communications profiling and, of course, visualization frontends. Without going into too much details, here s the list:

• OpenSpeedShop: a complete performance and profiling workbench, including I/O and MPI
• PerfSuite: a relatively simple and easy to use performance analysis toolkit
• TAU: a complete performance analysis framework including automatic instrumentation with PDT
• Scalasca: a tool for performance optimization of parallel codes, including MPI communications
• a number of dependencies: slog2, Open Trace Format libraries, VampirTrace, dyninst, monitor, perfctr, pfmon, PAPI,
• visualization frontends: paraprof, jumpshot4, cube3,

If you ve been anywhere near HPC code, the names probably ring a bell; they re the best tools out there in their category, developed and used by the top laboratories. Over the past year, all the tools have received some level of testing, meaning we know they do work at the very least to some extent. As you can imagine, testing such tools is no easy task and takes an insane amount of time and resources of all kinds. Testing is all the more important that I had to produce a number of patches to integrate the tools properly in the distribution and, in some cases, to even get them to build. Now, we would like to share this work with the HPC community in and around Debian. How exactly we are going to do that isn t clear just yet; most probably, we ll end up building a team with other interested parties and offer our packages as a base to build upon. There are a number of challenges with these tools: they re not easy to build, they re not easy to maintain, they re not easy to use, they re not easy to understand. Basically, nothing is easy. Some of those tools were never meant to be packaged and integrated in a distribution and no sane amount of patching will fix that, so we have to live with packages that aren t quite as polished as we like them to be. And then, there are licenses. Some tools are non-free due to usage restrictions. Others rely on non-free dependencies. Although I ve been looking at the licenses, a thorough license check will be required and decisions will need to be made. It s not for the faint of heart! If you are interested in these tools and in bringing them to Debian, please get in touch. I ve also had to package the Apache Derby database (Java); if someone out there cares about Derby, I d be more than happy to provide my packages as a start base for getting Derby into Debian. The packages need some work by someone who knows a thing or two about Derby and who can test and enhance the packaging of the server part.

A few weeks ago, I ve been busy at work updating the OFED packaging for OFED 1.5.2. We needed a version of the stack newer than what s (partially) available in Squeeze. As this has now been tested, we are all happy to contribute this work to Debian, courtesy of my customer, EDF. This morning, I ve pushed out packages for OFED 1.5.2 to the pkg-ofed SVN repository on Alioth. The packages should be made available in experimental at some point too, though a number of them will have to go through NEW. With this update, I ve cleaned up & updated the packaging where needed and also made two important changes that I need to highlight:

• package versions are now formed by the package version given in the BUILD_ID file shipped with OFED, postfixed with the OFED release they belong to. This gives, for instance, libibfoo 1.3.6-OFED-1.5.2-1. This makes it easier to identify OFED components and the OFED release they come from.
• InfiniBand driver modules for libibverbs have been renamed to ibverbs-driver-foo and the libibverbs modules are now installed to /usr/lib/ibverbs/drivers. This cleans up the packaging as the shared objects, that were previously shipped in libfoo packages, are not shared libraries but modules for libibverbs.

A number of libraries have changed their interfaces and bumped their soversions. libibcommon is gone, it has been folded into libibumad. The API/ABI between libibverbs and the IB driver modules has changed, so the new libibverbs comes with appropriate Breaks in place. Thanks to Beno t Mortier from pkg-ofed for his 2-day IB & OFED crash course that was instrumental in getting started with the OFED maze.

I ve just release pommed v1.35, adding support for the October 2010 MacBook Air machines. Kernel support for these machines has been submitted during the 2.6.37-rc cycle, so it will appear in 2.6.37 or 2.6.38 depending on maintainers, merge window and patch readiness. The machines have new keyboard assemblies (WellSpring IV and IVa) requiring new quirks in the HID layer, new trackpads, need sound & applesmc configurations of their own and the backlight module needs to know about them. So, make sure your kernel has all of that, otherwise your machine will be hard to use and pommed will not work at all. Update: #603395 filed against linux-2.6 with all the needed patches attached. It looks like we ll have all of this in Squeeze, thanks to our kernel team.

Apple released Remote 2.0, which doesn t work with forked-daapd. We are looking into it and trying to find out what the problem is. There are also more changes needed to support Remote 2.0 properly, so this will take some time. Remote 2.0 also introduces support for the iPad, with very different features and new requests. I don t have an iPad, so this won t be supported. For now, you should stick to the old version of Remote (1.3.3) until we get 2.0 working, at which point we ll probably pull support for the previous version in an attempt to maintain everybody s and the codebase sanity. In other news, I have a few changes pending that I want to get in before I cut a new release.

Hey, look, a release! I ve decided to start making tarball releases that make it easier to build forked-daapd, especially on embedded platforms. The tarballs come with the build system already generated (as should any real release) and they also contain pre-generated source files for the ANTLR3-based parsers used in forked-daapd. The pre-generated files will be used if antlr3 is not available for the build. I expect that pretty much everybody will end up using the generated files, given the feedback I ve had about ANTLR3. I hope this will also clear the misconception that forked-daapd depends on Java. I have expanded the installation instructions, making the distinction between building from the git tree and building from the tarballs. A few more fixes made it to this version since my last post, and Kai also put the final piece to the sort headers by adding sort headers to the song listings. Tarballs are available at http://alioth.debian.org/~jblache/forked-daapd/. Tarball integrity can be verified with GPG using the matching GPG signature file and my F5D65169 GPG key from the Debian keyring.

A number of things have happened in and around forked-daapd in the last month or so, so let s review the new features and the plan for Squeeze. On the features front, Kai Elwert has done a lot of work since July to implement missing features at the DAAP and DACP level to get better support for Remote:

• playlists can now be played from Remote;
• listings are sorted in a way that s closer to what you get from iTunes;
• sort headers are now generated when requested so Remote can display the A-Z quick access list.

Also, forked-daapd can now remember which speakers were selected before shutdown and attempts to automatically reselect these speakers the next time around. Speakers that were selected will be reselected if they appear at most 5 minutes after startup and the player isn t running at that time. On the Squeeze front, a viable snapshot of forked-daapd will release with Squeeze. This snapshot doesn t have all the Remote improvements listed above, although it s got the playlist support, can remember the speaker selection and includes bug fixes. My plan is to provide backports through backports.org once Squeeze is released and for as long as possible after that. There s always more in the works, so stay tuned =)

Thanks to the work of Mark Heily, both upstream and in Debian, this week saw the arrival of libdispatch in Debian. libdispatch is the userspace component of Apple s Grand Central Dispatch technology, designed to help write applications that really take advantage of multicore processors without having to fiddle directly with threads and locking (and getting it terribly wrong). To get that working on Linux, Mark wrote a userspace implementation of the pthread workqueues and a kqueue compatibility library (libkqueue). He also packaged the Blocks runtime (libblocksruntime) for Debian so as to build libdispatch with Blocks support. That means code using libdispatch must be built with CLang (or llvm-gcc-4.2) rather than gcc, as getting Blocks support in gcc in the near future looks highly improbable. On Mac OS X and FreeBSD, a kernel-based implementation of the pthread workqueues is used; we don t have that for Linux yet. The all-userspace implementation we have at the moment works really well; there are some caveats, though, but that s no reason for not starting to play with libdispatch right now! Try it out, it s great. Thanks again, Mark!

Following the update of antlr3 in unstable, I have now uploaded forked-daapd to Debian. For now, and until the new version of antlr3 is ready to migrate to testing, forked-daapd will only be available in experimental. I have also uploaded the ANTLR v3 C runtime (aka libantlr3c) to unstable. It s currently blocked from migrating to testing, until antlr3 is ready to migrate. Hopefully, the issues affecting antlr3 3.2 in unstable will be resolved soonish and all of this will migrate to testing and be part of Squeeze.

I ve been working on this for a few months now; it s taken quite some time because I wanted to do it right and deliver something better than what already existed in the OpenSource world as far as that particular technology was (not) concerned. So here it is:

AirTunes v2 UDP streaming forked-daapd can now do streaming to multiple AirTunes devices (as well as the local soundcard) and can be controlled from Remote (and, ideally, from anything that speaks DACP). I m releasing this early while the paint is still wet and not everything works on the DACP-side of things. Case in point, while playing a song or an album is supported, playing a playlist isn t. There s some work needed on this particular feature, and it ll get there eventually. There have been a couple of other changes in the codebase in recent weeks too; details after the jump. Implementing AirTunes v2 has been quite fun and, in addition to releasing the forked-daapd code today, I ve also taken the time to write down my findings about the protocol so others can experiment with it. This documentation is by no means exhaustive nor complete as far as the protocol goes; if you find out anything I haven t, let me know so I can update this document. Let s review the recent changes in the forked-daapd codebase over the past few weeks. AirTunes v2 You can now use Remote from your iSomething together with your AirTunes devices and forked-daapd to listen to your library. As mentioned above, playing a playlist is not supported; I have a few things to sort out before this can work and I suspect this will bring changes not only to the DACP interface but also to other parts of the code. So, stay tuned. forked-daapd supports synchronized playback to multiple AirTunes devices together with the local soundcard if so desired. You can expect the AirTunes devices to be in sync with each other, but the soundcard will usually run in circles around them. It looks like AirTunes plays sound a bit slower than 44.1 kHz; not a big deal, though, as AirTunes will catch up periodically and the desync, while it can be noticeable, isn t big (ie we re not talking seconds, not even one full second). Make sure your network can handle the load, as streaming to a single AirTunes device requires in excess of 1.6 Mbps (1.8 Mbps is actually closer to the real thing). Local audio output is supported via ALSA and OSS4, although note that the OSS4 code hasn t been very well tested. I intend to test and debug it fully at a later date, but feel free to beat me to it and send patches. Also note that, while there is code for FreeBSD in the player/streaming part of forked-daapd, it doesn t work well at all. The streaming code needs a timer with a higher resolution than kqueue/kevent has to offer, and I haven t found anything as capable as timerfd. So it s there, but it s broken. Read on about FreeBSD support. Metadata scanning Several fixes and enhancements have been made to the filescanner so it picks more and better metadata. Format-specific tagnames have been added to work around ffmpeg s limitations. Using a recent enough ffmpeg version (anything after mid-december 2009) will work best, especially for MP3 files. SQLite concurrency fixes The database code has been reworked to handle concurrency on the database. Issues with locked tables (or the whole database) were noticed pretty easily with big libraries when accessing forked-daapd during the library update after startup. On uniprocessor machines, this was even easier to hit. This, however, comes with new requirements for SQLite3: you ll now need SQLite3 v3.5.0 at least and it must be built with the unlock notify API enabled. Not all distributions ship SQLite3 with the unlock notify API enabled, so you may need to do a rebuild. Debian, for instance, doesn t; this is #579266. A word about FreeBSD support I m giving up on FreeBSD support. The streaming code doesn t work on FreeBSD due to timer issues, and I don t intend to fix it. I also don t intend to invest much time in porting future features to FreeBSD; if I can make it work, then fine, otherwise, too bad. Someone needs to step up if there s interest in having forked-daapd work on FreeBSD.

When I got my Mac Mini, one of the very first things I did was to test the Debian build of rEFIt on the machine, given the EFI firmware is x64 on the newer machines whereas it s ia32 on my MacBookPro; it was the very first time I could test the x64 build. And the results weren t good: some images couldn t be loaded, and it all seemed to be random. Disabling optimization in both GNU EFI and rEFIt for the x64 builds helped somewhat, at least it made rEFIt reliable enough to be usable. Now, with this new release, I started getting error messages, requiring a key press before the menu would be displayed. It s annoying but it s also very helpful as I now had an error message to work with, and even if the error message wasn t reliable, it would still point me to some part of the code. The message was: Invalid argument. On x64, the use of a call wrapper is mandatory because the EFI x64 calling convention doesn t match the calling convention of the ELF executables we re building and later disguising as EFI executables. The call wrapper has always been the prime suspect in this issue, it d be an obvious candidate for anyone looking at this issue. Today, digging into the issue with some new data, I realized what the problem was: when using the call wrapper, all arguments need to be of the UINT64 type. That s the type the call wrapper uses when extracting the arguments from its va_list. Introduce some more macros on top of the call wrapper, build, test: voila, it s fixed! That was tricky one. Let s sum it up for the search engines: all the arguments to the EFI interface called through uefi_call_wrapper() in GNU EFI need to be cast to UINT64. rEFIt 0.14-1, available in unstable in a couple hours. Next, I ll see if I can reenable optimization in both GNU EFI and rEFIt.

I ve just released pommed v1.32, a minor maintenance release for the 12 PowerBook G4. I ve just realized that I did not post an announce for pommed v1.31 a while back, which was also a maintenance release, adding support for the MacBookPro5,4 (15 June 2009) and the latest wireless keyboard.

After weeks of trying to get around to doing that, I am finally starting the process of replacing my 10-year old 1024 bits DSA key with a shiny new 4096 bits RSA key. The old key ID is F5D65169, the new key ID is FA1E5292; it s available from the keyservers or my website. I have put up a transition document, signed with both keys; if you ve signed my old key, grab it, verify it and if all is OK on your end, please sign the new key. You ll notice that one of the old UIDs did not make it to the new key; that s because I m not using that email address and don t actually intend to use it in the future. Thanks!